Rsyslog is a high performance and secure logging system that has started its journey as a regular syslog daemon and evolved into one of the powerful logging systems in most of the Linux distributions.
Rsyslog is incredibly fast that can pump over one million messages per second to any local destination.
Rsyslog follows typical client-server paradigm i.e. it can be configured to run as a centralized log server and as a client in individual systems/devices to send their log files to the logging server.
This tutorial covers installation and configuration of a centralized syslog server using rsyslog followed by setting up a rsyslog client to send log events to rsyslog server.
Prerequisites
- You can SSH into Ubuntu 18/16 system using root or sudo enabled user.
How to Install & Setup Rsyslog Server in Ubuntu
Rsyslog is installed by default in Ubuntu 18/Ubuntu 16. You can verify if rsyslog is installed previously by using the following command:
# rsyslogd -v
However, if you find rsyslog is missing in your system, you can always install it by using the following command in the terminal.
# apt install rsyslog
Once installed, enable and run rsyslog service:
# systemctl enable rsyslog # systemctl start rsyslog # systemctl status rsyslog
Configure Rsyslog Server
By default, rsyslog server runs as a client mode and uses imjournal and imusock module to import log events from systemd journal and accepting syslog events generated by applications in the local system respectively.
Open the rsyslog configuration file to verify this.
# vi /etc/rsyslog.conf
... ... # The imjournal module bellow is now used as a message source instead of imuxsock. $ModLoad imuxsock # provides support for local system logging (e.g. via logger command) $ModLoad imjournal # provides access to the systemd journal ... ...
To enable rsyslog run in server mode, you need to enable protocol and port number on which it will receive log events from remote systems. Two protocols are used to facilitate reception of log events from remote system and are TCP and UDP.
The first one is secure and connection oriented scheme while the second one is not connection oriented may suffered from message losses.
However in some situations, you may want to use UDP protocol like receiving log events from a remote system in the local subnet where there are less chances of message losses.
Further few network devices able to communicate with UDP protocol only.
To enable log events reception in port number 514 using TCP protocol, enable the following two lines in rsyslog configuration file.
# vi /etc/rsyslog.conf
... ... # Provides TCP syslog reception $ModLoad imtcp $InputTCPServerRun 514 ... ...
Similarly enable UDP log event reception in port number 514. There is no harm in running both the protocol in the same port.
# vi /etc/rsyslog.conf
... ... # Provides UDP syslog reception $ModLoad imudp $UDPServerRun 514 ... ...
At this point, you need to define a template that will be analyzed by rsyslog server before receiving the incoming log events. The template instructs rsyslog server where to send the incoming log events. It can be a file in the local file system or another syslog server.
Make sure to place the template line after $ AllowedSender line:
... ... $template incoming-logs,"/var/log/%HOSTNAME%/%PROGRAMNAME%.log" ... ...
Next define a rule-set to process incoming log events. The format to define a rule is like follows:
Facility.Severity Level Destination
Where the facility can be one of the following:
- kern – Kernel log events (0)
- user – User-level log events (1)
- mail – Mail system log events (2)
- daemon – System daemon log events (3)
- auth – Security and authorization log events (4)
- syslog – Internally generated syslog events (5)
- lpr – Printing system log events (6)
- news – News daemon log events (7)
- uucp – Unix-to-Unix copy program (uucp) daemon log events (8)
- cron – cron daemon log events (9)
- authpriv – Security and authorization log events (10)
- ftp – FTP daemon log events (11)
- ntp – NTP log events (12)
- security – Audit daemon log events (13)
- console – Alert log events (14)
- solaris-cron – scheduling daemon log events (15)
- local0 – local7 – Locally defined application log events
The severity level along with their numerical codes are:
- Emergency (emerg, 0) – Panic messages signifying system is unusable (0)
- Alert (alert,1) – Action must be taken immediately (1)
- Critical (crit , 2) – Critical conditions (2)
- Error (err, 3) – Not so urgent messages (3)
- Warning (warning, 4) – Warning messages (4)
- Notice (notice, 5) – Normal but significant messages (5)
- Informational (info,6) – Informational messages (6)
- Debug (debug, 7) – Debug level messages (7)
For example, if you want to send all mail related log events to local file system then the following rule will do that for you:
mail.* /var/log/mail
Where * in mail.* signifies to include all severity level for facility mail.
Suppose you want rsyslog server to process log events related to all facility and severity levels then the rule becomes:
*.* /var/log/all
Now combine template and the rule set in the rsyslog configuration file after $ AllowedSender line.
# vi /etc/rsyslog.conf ... ... $template incoming-logs,"/var/log/%HOSTNAME%/%PROGRAMNAME%.log" *.* ?incoming-logs & ~ ... ...
The line “& ~” instruct rsyslog server to stop processing the log event as soon as it is written to the file. Save the file and restart rsyslog server.
# systemctl restart rsyslog
Verify if rsyslog server is listening in the configured port by using the following command:
# ss -tunelp | grep 514
udp UNCONN 0 0 *:514 *:* users:(("rsyslogd",pid=7486,fd=3)) ino:37269 sk:ffffa0c29cba5100 <->
udp UNCONN 0 0 :::514 :::* users:(("rsyslogd",pid=7486,fd=4)) ino:37270 sk:ffffa0c2a7828980 v6only:1 <->
tcp LISTEN 0 25 *:514 *:* users:(("rsyslogd",pid=7486,fd=5)) ino:37273 sk:ffffa0c29c9e26c0 <->
tcp LISTEN 0 25 :::514 :::* users:(("rsyslogd",pid=7486,fd=6)) ino:37274 sk:ffffa0c29fd51080 v6only:1 <->
The rsyslog server is now ready to receive log events from remote system in the port 514 using TCP/UDP protocol.
Configure Firewall in Rsyslog Server
If you are using uncomplicated firewall manager(UFW) and is enabled in the system then allow it to receive log events in the designated port(514).
To do that, just issue the following set of commands in the terminal.
# ufw allow 514/tcp # ufw allow 514/udp # ufw reload
How to Setup & Configure Rsyslog Client
You are already aware that by default rsyslog runs in client mode. Hence you just need to install rsyslog in the client machine and configure it to send log events to rsyslog server.
Start by installing rsyslog in the client system if it is not installed previously.
# apt install rsyslog # systemctl enable rsyslog
Now configure rsyslog in the client system to send all log events to remote rsyslog server that we configured in the previous step.
To do that, edit the rsyslog configuration file and add a line with following format in the rules section.
*.* @ip-address-of-rsysog-server:514
OR
*.* @@ip-address-of-rsysog-server:514
The @ before IP address of rsyslog server indicates rsyslog client to send log events to UDP port whereas @@ signifies to send the log events to TCP port.
# vi /etc/rsyslog.conf ... ... #### RULES #### ... ... *.* @10.160.0.5:514 ... ...
Restart rsyslog daemon:
# systemctl restart rsyslog
Head over to rsyslog server and navigate to the /var/log/ folder to find a folder by the name same as hostname of remote client. Change to the folder to list all the log files.
Conclusion
Rsyslog is an open source, client/server central log processing system. It’s a high-performance logging system for Linux operating system.
You can now setup a centralized log server using rsyslog that can receive and store log events from remote client. Further, this tutorial also covers how to forward log events to centralized log server using rsyslog client.
This article shows you the basic use of Rsyslog system, there are a ton of things you can do with it, if you wanted to dive more details into it, head over to the official documentation of Rsyslog.