Set Up Rsyslog and LogAnalyzer on CentOS Linux 5.5 for Centralized Logging
LogAnalyzer is a web based program that allows you to view event messages from a syslog source within your web browser. Rsyslog is a drop in replacement for the syslog daemon that among other things allows syslog messages to be saved in a MySQL database. Combining these two great programs and directing other network devices to forward syslog messages to a central server allows for a very powerful solution for searching and archiving event messages that occur throughout your network environment. In this example I will install rsyslog on a CentOS Linux 5.5 server to aggregate and collect syslog messages and configure LogAnalyzer on the same server to allow for a user friendly interface for viewing and searching through these messages.
First we need to install some required RPM’s. Since I am running LogAnalyzer, Rsyslog, and MySQL all on the same server I will install these required packages:
# yum install httpd php mysql php-mysql mysql-server wget rsyslog rsyslog-mysql
Now we’ll make sure MySQL and Apache are configured to start automatically and start them up:
# chkconfig mysqld on
# chkconfig httpd on
# service mysqld start
# service httpd start
By default the MySQL root database user is blank so for security we should set it now:
# mysqladmin -u root password NewPassword
Now let’s import the database schema for the rsyslog database into MySQL. You may need to adjust the path to your “createDB.sql” file below if the rsyslog version has been updated.
# mysql -u root -p < /usr/share/doc/rsyslog-mysql-3.22.1/createDB.sql
It is best practice to limit database access for applications, so now we’ll set up a user specifically for LogAnalyzer and rsyslog that we’ll use to access the newly created rsyslog database. For even greater security you may want to set up separate accounts for both rsyslog and LogAnalyzer, since LogAnalyzer is only viewing the rsyslog database fewer privileges like select should be needed. For my environment using the same user is adequate. Notice with MySQL you can make access very granular and specify to only allow the rsyslog user database access from the localhost. Also we’ll execute the “flush privileges” MySQL command to activate our permissions changes immediately.
# mysql -u root -p mysql
mysql> GRANT ALL ON Syslog.* TO rsyslog@localhost IDENTIFIED BY 'Password';
mysql> flush privileges;
mysql> exit
Now it is time to edit the”/etc/rsyslog.conf” file. We’ll include information that will allow us to log syslog messages from rsyslog into the MySQL database. The first line loads the MySQL driver. The second line allows us to specify to log messages from the “authpriv” facility with all severities, which includes most log in/out messages and switch user events. If I wanted to log all messages to MySQL I would specify *.*. I have identified the MySQL database server to log to as 127.0.0.1, Syslog is the name of the MySQL database, and finally I have specified my MySQL rsyslog username and password. To specify additional syslog facility/severity combinations add them to the front of the second line and separate each combination with a semicolon (mail.*;authpriv.* :ommysql…). Remember that when you specify a severity that is the minimum level which will be logged, anything with a higher severity will also be logged. Add the code to the top of the file:
$ModLoad ommysql
authpriv.* :ommysql:127.0.0.1,Syslog,rsyslog,Password
Now it’s time to shut down and disable the existing syslog daemon and enable and start up rsyslog:
# chkconfig syslog off
# service syslog stop
# chkconfig rsyslog on
# service rsyslog start
It is now time to go out to the web and download LogAnalyzer. To find information on the latest release go to http://loganalyzer.adiscon.com/downloads.
Or to download directly to your Linux server the version I am using enter this (wget is required):
# cd ~
# wget http://download.adiscon.com/loganalyzer/loganalyzer-3.0.0.tar.gz
Unzip and untar the LogAnalyzer files:
# tar zxvf loganalyzer-3.0.0.tar.gz
Now it is time to move various files and subdirectories to your Apache web document root. In this example I am assuming that this is still the Apache default of “/var/www/html”.
# mv loganalyzer-3.0.0/src /var/www/html/loganalyzer
# mv loganalyzer-3.0.0/contrib/* /var/www/html/loganalyzer/
Change to new LogAnalyzer web subdirectory, modify the file permissions on two scripts, and run the configure.sh script. This will create a blank config.php file which will be have information added during the web portion of the configuration.
# cd /var/www/html/loganalyzer
# chmod u+x configure.sh secure.sh
# ./configure.sh
Great tutorial. I had tried to set this up before and got lost at some point. This was very easy to follow. Two things to mention, when setting up the data source in loganalyzer you should make sure the table is “SystemEvents” and not “systemevents”. You can edit this in the config.php file at the bottom later if you miss it.
Also, for me the remote logging to the central rsyslog server does not work until I edit /etc/sysconfig/rsyslog and modify rsyslog options to be “SYSLOGD_OPTIONS=”-r514 -m 0″ and then restart the service. We are using the same versions so I’m not sure if I missed something or you did.
Thanks for the tips, James!
add this to the /etc/rsyslog to make it become a central rsyslog server:
#UDP log
$ModLoad imudp
$UDPServerRun 514
#TCP log
$ModLoad imtcp
$InputTCPServerRun 514
on the client, add port 514 to end of the rsyslog server
/etc/rsyslog
authpriv.* @192.168.10.100:514 #udp
or
authpriv.* @@192.168.10.100:514 #tcp
Sorry for my delay, but thanks for your tips Guang. Definitely a better and more up to date method for configuring rsyslog parameters.
Hello Aaron,
First of all great tutorial, it’s really easy to follow and getting rsyslog and loganalyzer to work is done in a few minutes. However, I’m struggling to get remote logging enabled. I’m not sure if I need something like remote privileges for mysql?
I have tried the tips from James (adding -r514 tot syslogd options) and I also tried the settings from Guang, but no matter what I try, no logging from remote server. Do you offer commercial support to troubleshoot, because I’m out of options and I badly want some logging to be active :-)
Hi Gijsbert,
I would recommend first verifying that remote rsyslog logging works to a file and then move on to checking that messages are logged into MySQL. As long as the facility and priority settings are configured correctly remote syslog messages should show up in the files along with the local messages on the rsyslog server. You shouldn’t need any remote access configured with MySQL as long as it and the rsyslog server are on the same computer.
You could also try my options settings in “/etc/sysconfig/rsyslog”:
SYSLOGD_OPTIONS="-r -t514 -m 0"
I think it is basically the same as James’, however I can’t find any information on what the -t switch does. Also you might try to disable the host firewalls on both devices temporarily to ensure this isn’t the source of your problem.
Best Wishes,
Aaron
hi awalrath,
i’m follow all your step one by one, but then i got this message after complete setup lognalyzer “No syslog records found – Error Details: Could not find the configured database”. I have tried so many time but git a same result. can you help me on this.
Hi azuan…
I have the same error after sevrel installation. How did you solve your problem?
Thanks
The database is called “Syslog” and the table is called “SystemEvents” (both case sensitive)
Heya Aaron,
Great post, worked with out a problem after adding the -r514 to “/etc/sysconfig/rsyslog”
Use this to monitor a bunch of pretty active servers and has made life a bit easier for the guys.
Cheers
Ben
Hey
Just to ask is the remote client must install rsyslog also??
Hi techkaki –
Rsyslog on the client is optional, if you wanted you could use standard syslog for either the server or the client. Rsyslog on the client will give you several advantages, however. It will use TCP to send messages to the rsyslog server for more reliable delivery. It can also spool messages locally if the rsyslog server is down, and send the non-transmitted messages when the server comes back online.
I’ve been meaning to update the instructions because they are a little dated and there are now better ways to configure the rsyslog server and the rsyslog/syslog client, although most of this is covered in the excellent user comments.
Aaron
Heya Techkaki,
No you should not. I can confirm that you can use syslogd which comes centos.
Its just a matter of making sure the rsyslog is the server to which your sending your syslogs to.
Hope that helps
Cheers
Ben
how come my remote machine fail to log message to rsyslog server.
On rsyslog server, i already set options settings in “/etc/sysconfig/rsyslog”:
SYSLOGD_OPTIONS=”-r -t514 -m 0″
on remote machine, i already configure the /etc/syslog.conf setting as below:-
*.* @hostname_or_IP_of_syslog_server
Hi techkaki,
Your configuration on your syslog clients should be okay. Actually I am planning on writing a new version of this article to include the newer recommendations regarding configuration.
You could try this in the /etc/sysconfig/rsyslog:
SYSLOGD_OPTIONS=”-r514 -m 0″
I believe the -t turns on TCP message reception, and the standard old syslog daemon doesn’t support sending messages via TCP (only UDP).
Instead of the change above you could append this to your /etc/rsyslog.conf:
# Provides UDP syslog reception
$ModLoad imudp.so
$UDPServerRun 514
# Provides TCP syslog reception
$ModLoad imtcp.so
$InputTCPServerRun 514
That will activate both TCP and UDP message reception. Don’t forget to set the appropriate firewall rules.
Aaron
i looking forward with your new tutorial about rsyslog. In your new tutorial, if can, please provide more example on how to logs remote machines logs to rsyslog server. For example, logs windows logs, firewall logs, centos logs, wifi logs and debian logs.
Am I install and configure rsyslog on UBUNTU as above given method or something is different
Hi abdulmalik,
I am running Ubuntu 10.10 on some of my client PCs and it looks like rsyslog is installed by default. The configuration should basically be the same, on the rsyslog server the best option is to uncomment the following directives in the /etc/rsyslog.conf file:
# Provides UDP syslog reception
$ModLoad imudp.so
$UDPServerRun 514
# Provides TCP syslog reception
$ModLoad imtcp.so
$InputTCPServerRun 514
One big difference I see is that on Ubuntu the directives for logging to actual files is broken out in under the /etc/rsyslog.d/*.conf files.
Cheers,
Aaron
Thank You very much,
I have configured rsyslog server and snare agent for windows system but following fields are missing in log analyzer
Event Type, Event Source, Event ID and Event User. Please guide me, how resolve this problem.
When I select veiw “eventlog feild” follwoing fields in loganalyzer are empty,
Event Type, Event Source, Event ID and Event User. Please guide me, how resolve this problem
Thank You
Hi abdulmalik,
I’m stuck at Windows client. I do not know how to send logs to my rsyslog server, could you please guide me.
The other good Windows syslog client (in addition to the very good Snare as abdulmalik mentioned) is the Datagram SyslogAgent.
http://syslogserver.com/syslogagent.html
It is based on the old NTsyslog. What I like about it is that you can choose different facilities for different event log sources, I believe you can only select one syslog facility for them all with the free Snare client. Also, Datagram’s agent optionally will check if the syslog/rsyslog server is up and will spool the syslog messages locally until the server is back online. AFAIK this support is not available with Snare unless you are using it with the payware Event Reporter.
Cheers,
Aaron
You can download event forwarding agent from following url,
If you want your Windows 2000 – Server 2008 machines to send logs to your new central log server then check out eventlog-to-syslog. Just download it, unzip it, copy evtsys.exe to C:\Windows\system32\ and execute
C:\Windows\system32>evtsys.exe -i -h ip.address.of.myserv
https://engineering.purdue.edu/ECN/Resources/Documents/UNIX/evtsys/
You can also use snare agent for windows, download from following url,
http://www.intersectalliance.com/projects/index.html
In networking settings change port to 514 and putt syslog server IP and lease check syslog header
I have configured Rsyslog with RELP and Log analyzer, now facing problem in eventlog feild veiw there are some fields are missing like eventlog type, event user, source event and event Id, can anyone help in this regards, here is my rsyslog.conf file,
# /etc/rsyslog.conf Configuration file for rsyslog.
#
# For more information see
# /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html
#
# Default logging rules can be found in /etc/rsyslog.d/50-default.conf
#################
#### MODULES ####
#################
$ModLoad imuxsock # provides support for local system logging
$ModLoad imklog # provides kernel logging support (previously done by rklogd)
#$ModLoad immark # provides –MARK– message capability
$KLogPath /proc/kmsg
# provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514
# provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514
###########################
#### GLOBAL DIRECTIVES ####
###########################
#
# Use traditional timestamp format.
# To enable high precision timestamps, comment out the following line.
#
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
# Filter duplicated messages
$RepeatedMsgReduction on
#
# Set the default permissions for all log files.
#
$FileOwner syslog
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022
$PrivDropToUser syslog
$PrivDropToGroup syslog
#
# Include all config files in /etc/rsyslog.d/
#
$IncludeConfig /etc/rsyslog.d/*.conf
# Buffering stuff:
$WorkDirectory /var/rsyslog/work # default location for work (spool) files
$ActionQueueType LinkedList # use asynchronous processing
$ActionQueueFileName dbq # set file name, also enables disk mode
$ActionResumeRetryCount -1 # infinite retries on insert failure
lease note i am not using event reporter
One thing you may want to try is test sending your Windows logs to a regular text file on your rsyslog server and see if the missing fields show up in there. There is a possibility that there is an incompatibility between how rsyslog is saving the messages into MySQL and how Snare is formatting the messages. Unfortunately I may be of limited help because I am no longer using Snare for my Windows syslog client, currently I am using the Datagram SyslogAgent:
http://syslogserver.com/syslogagent.html
You may want to give it a try and see if it works any better for you.
Cheers,
Aaron
I have a problem. I follow the tutorial, but when i try to enter the http://localhost/loganalyzer page i get this message
You don’t have permission to access /loganalyzer on this server.
Someone can help me.
Many Thanks
Hi Jason,
Sorry for my delay, I am actually not running loganalyzer at the moment and wanted to check on an old server if I had any trouble like you did. One thing you may want to check is whether SELinux is set to enforcing. If so there may be an issue with the context label on the loganalyzer files, and may prevent apache displaying the loganalyzer web pages. You can temporarily set it to permissive with “setenforce 0”. I have noticed that SELinux seems to have a problem if a directory is moved into the /var/www/html apache root, but if the files are extracted from the archive directory to the web root directly everything seems to work fine.
On my old install of loganalyzer I don’t have any special permissions set on the files or directory, so I don’t believe that would be the source of your problem.
Cheers,
Aaron
I had the same problem.
You have to disable the SELinux. To do it open /etc/selinux/config using your favorite text editor and change the line
from
SELINUX=enforcing
to
SELINUX=disabled
Hope it will solve your problem.
resetting permissions with
chcon -Rv –type=httpd_sys_content_t /var/www/html
got me in without turning off selinux
Ok, thanks… I cant solve the problem… I never work with loganalyzer.
Great tutorial and informations. I have installed loganalyzer and works just fine. Thanks!
If you want a cross-platform tool which can collect on Windows and Linux, take a look at nxlog which is a great open source tool I’m using with success. http://nxlog
Its a great article, I was able to setup rsyslog and log analyer,
I have 2 questions:
1. How can I configure loganalyzer to populate by reading multiple log files located under different location.
2. I have cisco switches as client, I need to point them to log server. I am not sure whether I can install rsyslog package on those switches, any idea on this ??/
Thanks.
its gr8 article i have read & configured it . Now i want to configure the logs of cisco switch to this server. Can you please guide in this.
Thanks
I have a problem in loganalyzer in the index.php “no syslog records found – Error Details:”
No syslog records found
Please help me…!!