Rsyslog on RHEL 8

Rsyslog is the default syslog daemon on RHEL 8 and 9, and understanding its configuration is crucial for any system administrator.

To check if the rsyslog is running on your system:

systemctl status rsyslog

Rsyslog Basic Architecture and Components

• imuxsock: Inputs messages from /dev/log (local socket).
• imjournal: Reads entries from the journal when you want hybrid journald+rsyslog setups.
• Modules: Loadable plugins that add features (e.g., TCP, TLS, RELP).
• Rulesets: Define selectors (facility.level) and actions (write to file, forward, discard).
• Actions: Destinations or operations, such as writing to a file (omfile) or forwarding to a remote server (omfwd).

Basic Rsyslog Configuration

• Main config file: /etc/rsyslog.conf.
• Drop-in configs: /etc/rsyslog.d/*.conf.
• Sections in /etc/rsyslog.conf:
  • Module Load: Load built-in and extra modules.
  • Global Directives: Set default file formats, templates, and logging severity.
  • Rules: Use syntax facility.priority action; e.g., *.err @logserver

After editing the configuration files, you can run the following command to test the configuration syntax:

rsyslogd -N1

After changing the rsyslog configuration, you need to restart the daemon to put in place the new configuration/changes:

systemctl restart rsyslog

Filtering & Custom Log Files

Let’s provide an example: To capture info–level messages in a separate file:

1- Create /etc/rsyslog.d/info.conf with the following content:

*.info    /var/log/messages.info

Note: The basic structure for this configuration line is:

facility.priority log_file

The facility is one of the following keywords:
— auth, authpriv, cron, daemon, kern, lpr, mail, mark, news, security (same as auth), syslog, user, uucp, and local0 through local7

The priority is one of the following keywords, in ascending order:
— debug, info, notice, warning, warn (same as warning), err, error (same as err), crit, alert, emerg, panic (same as emerg).
The keywords’ error’, ‘warn’, and ‘panic’ are deprecated and should no longer be used. The priority defines the severity of the message.

The log_file is the absolute path to the file that will store the logs based on the facility.priority configuration!

2- Restart the rsyslog daemon and make a test using the program logger:

systemctl restart rsyslog
logger -p user.info "Test info log"
tail /var/log/messages.info

As we can see, with “logger” we created a test message with the “info” priority. The message has been sent to the file “/var/log/messages.info”.

Application Error Logging via Rsyslog

Let’s consider the Apache (httpd) application. To install Apache on RHEL:

dnf install -y httpd

Enable the service:

systemctl enable --now  httpd

Check the service status:

systemctl status httpd

The Apache’s main configuration file is “/etc/httpd/conf/httpd.conf”. By default, all errors are saved in the directory “logs”, into the file “error_log”:

So, in this case, we’ll direct Apache error logs through syslog:

1- In /etc/httpd/conf/httpd.conf, add:

# Basically, it means: Forward the errors logs from Apache to the local1 syslog facility:
ErrorLog "syslog:local1"

2- In /etc/rsyslog.d/httpd.conf, write:

# Basically, it means: Forward the logs with local1 facility and err priority to the specific file:
local1.err    /var/log/httpd-error.log

3- Restart both services (httpd and rsyslog):

systemctl restart httpd rsyslog

4- Trigger an error and verify the log:

logger -p local1.err "RSYSLOG TEST: hello from local1.err"
tail /var/log/httpd-error.log

Remote Logging

Another significant possibility is to forward critical logs to a centralized server. Let’s provide an example of that:

1- On the client, add to /etc/rsyslog.d/forward.conf:

*.alert    @@192.168.255.165:514

Note: In this case, 192.168.255.165 is the IP of the remote syslog server!

2- On the server, enable UDP/TCP reception in /etc/rsyslog.d/remote.conf:

module(load="imtcp")
input(type="imtcp" port="514")

On the server, allow TCP/514 on the firewall:

firewall-cmd --zone=public --permanent --add-port=514/tcp
firewall-cmd --reload

# To confirm the open ports:
firewall-cmd --zone=public --list-ports

3- Restart the rsyslog service on both client and server, and check if TCP/514 is listening on the server:

systemctl restart rsyslog    # on both client and server systems

netstat -putan | grep 514    # only on the server system

From the client, test the connectivity to the server on TCP/514:

nc -vz 192.168.255.165 514

The expected result should be:

From the client, let’s generate a test message:

logger -p user.alert -t REMOTE_TEST "Remote log forwarding test at $(date '+%Y-%m-%d %H:%M:%S')"

After sending the test message, on the server, filter the message in “/var/log/messages”:

grep "REMOTE_TEST" /var/log/messages

As we can see, the test message was tagged with “REMOTE_TEST” on the client, and it was forwarded to the remote server correctly through the Rsyslog daemon!

That’s it for now 🙂