Home > Linux, Monitoring, syslog > Set Up Rsyslog and LogAnalyzer on CentOS Linux 5.5 for Centralized Logging

Set Up Rsyslog and LogAnalyzer on CentOS Linux 5.5 for Centralized Logging

LogAnalyzer is a web based program that allows you to view event messages from a syslog source within your web browser.  Rsyslog is a drop in replacement for the syslog daemon that among other things allows syslog messages to be saved in a MySQL database.  Combining these two great programs and directing other network devices to forward syslog messages to a central server allows for a very powerful solution for searching and archiving event messages that occur throughout your network environment.  In this example I will install rsyslog on a CentOS Linux 5.5 server to aggregate and collect syslog messages and configure LogAnalyzer on the same server to allow for a user friendly interface for viewing and searching through these messages.

First we need to install some required RPM’s.  Since I am running LogAnalyzer, Rsyslog, and MySQL all on the same server I will install these required packages:

# yum install httpd php mysql php-mysql mysql-server wget rsyslog rsyslog-mysql

Now we’ll make sure MySQL and Apache are configured to start automatically and start them up:

# chkconfig mysqld on
# chkconfig httpd on
# service mysqld start
# service httpd start

By default the MySQL root database user is blank so for security we should set it now:

# mysqladmin -u root password NewPassword

Now let’s import the database schema for the rsyslog database into MySQL.  You may need to adjust the path to your “createDB.sql” file below if the rsyslog version has been updated.

# mysql -u root -p < /usr/share/doc/rsyslog-mysql-3.22.1/createDB.sql

It is best practice to limit database access for applications, so now we’ll set up a user specifically for LogAnalyzer and rsyslog that we’ll use to access the newly created rsyslog database.  For even greater security you may want to set up separate accounts for both rsyslog and LogAnalyzer, since LogAnalyzer is only viewing the rsyslog database fewer privileges like select should be needed.  For my environment using the same user is adequate.  Notice with MySQL you can make access very granular and specify to only allow the rsyslog user database access from the localhost.  Also we’ll execute the “flush privileges” MySQL command to activate our permissions changes immediately.

# mysql -u root -p mysql
mysql> GRANT ALL ON Syslog.* TO rsyslog@localhost IDENTIFIED BY 'Password';
mysql> flush privileges;
mysql> exit

Now it is time to edit the”/etc/rsyslog.conf” file.  We’ll include information that will allow us to log syslog messages from rsyslog into the MySQL database.  The first line loads the MySQL driver.  The second line allows us to specify to log messages from the “authpriv” facility with all severities, which includes most log in/out messages and switch user events.  If I wanted to log all messages to MySQL I would specify *.*.  I have identified the MySQL database server to log to as 127.0.0.1, Syslog is the name of the MySQL database, and finally I have specified my MySQL rsyslog username and password.  To specify additional syslog facility/severity combinations add them to the front of the second line and separate each combination with a semicolon (mail.*;authpriv.* :ommysql…).  Remember that when you specify a severity that is the minimum level which will be logged, anything with a higher severity will also be logged.  Add the code to the top of the file:

$ModLoad ommysql
authpriv.* :ommysql:127.0.0.1,Syslog,rsyslog,Password

Now it’s time to shut down and disable the existing syslog daemon and enable and start up rsyslog:

# chkconfig syslog off
# service syslog stop
# chkconfig rsyslog on
# service rsyslog start

It is now time to go out to the web and download LogAnalyzer.  To find information on the latest release go to http://loganalyzer.adiscon.com/downloads.

Or to download directly to your Linux server the version I am using enter this (wget is required):

# cd ~
# wget http://download.adiscon.com/loganalyzer/loganalyzer-3.0.0.tar.gz

Unzip and untar the LogAnalyzer files:

# tar zxvf loganalyzer-3.0.0.tar.gz

Now it is time to move various files and subdirectories to your Apache web document root.  In this example I am assuming that this is still the Apache default of “/var/www/html”.

# mv loganalyzer-3.0.0/src /var/www/html/loganalyzer
# mv loganalyzer-3.0.0/contrib/* /var/www/html/loganalyzer/

Change to new LogAnalyzer web subdirectory, modify the file permissions on two scripts, and run the configure.sh script.  This will create a blank config.php file which will be have information added during the web portion of the configuration.

# cd /var/www/html/loganalyzer
# chmod u+x configure.sh secure.sh
# ./configure.sh

Advertisements

Pages: 1 2 3

Categories: Linux, Monitoring, syslog Tags: , ,
  1. James
    December 1, 2010 at 7:58 pm

    Great tutorial. I had tried to set this up before and got lost at some point. This was very easy to follow. Two things to mention, when setting up the data source in loganalyzer you should make sure the table is “SystemEvents” and not “systemevents”. You can edit this in the config.php file at the bottom later if you miss it.

    Also, for me the remote logging to the central rsyslog server does not work until I edit /etc/sysconfig/rsyslog and modify rsyslog options to be “SYSLOGD_OPTIONS=”-r514 -m 0″ and then restart the service. We are using the same versions so I’m not sure if I missed something or you did.

    • December 1, 2010 at 11:52 pm

      Thanks for the tips, James!

  2. Guang
    December 14, 2010 at 4:37 am

    add this to the /etc/rsyslog to make it become a central rsyslog server:

    #UDP log
    $ModLoad imudp
    $UDPServerRun 514
    #TCP log
    $ModLoad imtcp
    $InputTCPServerRun 514

    on the client, add port 514 to end of the rsyslog server

    /etc/rsyslog

    authpriv.* @192.168.10.100:514 #udp

    or
    authpriv.* @@192.168.10.100:514 #tcp

    • December 23, 2010 at 4:09 am

      Sorry for my delay, but thanks for your tips Guang. Definitely a better and more up to date method for configuring rsyslog parameters.

  3. Gijsbert
    December 22, 2010 at 11:07 pm

    Hello Aaron,

    First of all great tutorial, it’s really easy to follow and getting rsyslog and loganalyzer to work is done in a few minutes. However, I’m struggling to get remote logging enabled. I’m not sure if I need something like remote privileges for mysql?

    I have tried the tips from James (adding -r514 tot syslogd options) and I also tried the settings from Guang, but no matter what I try, no logging from remote server. Do you offer commercial support to troubleshoot, because I’m out of options and I badly want some logging to be active :-)

    • December 23, 2010 at 4:05 am

      Hi Gijsbert,

      I would recommend first verifying that remote rsyslog logging works to a file and then move on to checking that messages are logged into MySQL. As long as the facility and priority settings are configured correctly remote syslog messages should show up in the files along with the local messages on the rsyslog server. You shouldn’t need any remote access configured with MySQL as long as it and the rsyslog server are on the same computer.

      You could also try my options settings in “/etc/sysconfig/rsyslog”:

      SYSLOGD_OPTIONS="-r -t514 -m 0"

      I think it is basically the same as James’, however I can’t find any information on what the -t switch does. Also you might try to disable the host firewalls on both devices temporarily to ensure this isn’t the source of your problem.

      Best Wishes,

      Aaron

  4. azuan alias
    March 22, 2011 at 3:21 am

    hi awalrath,

    i’m follow all your step one by one, but then i got this message after complete setup lognalyzer “No syslog records found – Error Details: Could not find the configured database”. I have tried so many time but git a same result. can you help me on this.

    • Gino
      September 30, 2011 at 2:31 pm

      Hi azuan…

      I have the same error after sevrel installation. How did you solve your problem?

      Thanks

      • Bob
        January 11, 2012 at 9:34 pm

        The database is called “Syslog” and the table is called “SystemEvents” (both case sensitive)

  5. Ben
    March 23, 2011 at 6:30 am

    Heya Aaron,
    Great post, worked with out a problem after adding the -r514 to “/etc/sysconfig/rsyslog”

    Use this to monitor a bunch of pretty active servers and has made life a bit easier for the guys.

    Cheers
    Ben

  6. May 5, 2011 at 9:36 am

    Hey

    Just to ask is the remote client must install rsyslog also??

    • May 6, 2011 at 2:13 am

      Hi techkaki –

      Rsyslog on the client is optional, if you wanted you could use standard syslog for either the server or the client. Rsyslog on the client will give you several advantages, however. It will use TCP to send messages to the rsyslog server for more reliable delivery. It can also spool messages locally if the rsyslog server is down, and send the non-transmitted messages when the server comes back online.

      I’ve been meaning to update the instructions because they are a little dated and there are now better ways to configure the rsyslog server and the rsyslog/syslog client, although most of this is covered in the excellent user comments.

      Aaron

  7. Ben
    May 6, 2011 at 2:10 am

    Heya Techkaki,
    No you should not. I can confirm that you can use syslogd which comes centos.
    Its just a matter of making sure the rsyslog is the server to which your sending your syslogs to.

    Hope that helps

    Cheers
    Ben

    • May 19, 2011 at 7:18 pm

      how come my remote machine fail to log message to rsyslog server.
      On rsyslog server, i already set options settings in “/etc/sysconfig/rsyslog”:

      SYSLOGD_OPTIONS=”-r -t514 -m 0″

      on remote machine, i already configure the /etc/syslog.conf setting as below:-
      *.* @hostname_or_IP_of_syslog_server

      • May 20, 2011 at 3:24 am

        Hi techkaki,

        Your configuration on your syslog clients should be okay. Actually I am planning on writing a new version of this article to include the newer recommendations regarding configuration.

        You could try this in the /etc/sysconfig/rsyslog:

        SYSLOGD_OPTIONS=”-r514 -m 0″

        I believe the -t turns on TCP message reception, and the standard old syslog daemon doesn’t support sending messages via TCP (only UDP).

        Instead of the change above you could append this to your /etc/rsyslog.conf:

        # Provides UDP syslog reception
        $ModLoad imudp.so
        $UDPServerRun 514

        # Provides TCP syslog reception
        $ModLoad imtcp.so
        $InputTCPServerRun 514

        That will activate both TCP and UDP message reception. Don’t forget to set the appropriate firewall rules.

        Aaron

  8. May 20, 2011 at 5:09 am

    awalrath :
    Hi techkaki,
    Your configuration on your syslog clients should be okay. Actually I am planning on writing a new version of this article to include the newer recommendations regarding configuration.
    You could try this in the /etc/sysconfig/rsyslog:
    SYSLOGD_OPTIONS=”-r514 -m 0″
    I believe the -t turns on TCP message reception, and the standard old syslog daemon doesn’t support sending messages via TCP (only UDP).
    Instead of the change above you could append this to your /etc/rsyslog.conf:
    # Provides UDP syslog reception
    $ModLoad imudp.so
    $UDPServerRun 514
    # Provides TCP syslog reception
    $ModLoad imtcp.so
    $InputTCPServerRun 514
    That will activate both TCP and UDP message reception. Don’t forget to set the appropriate firewall rules.
    Aaron

    i looking forward with your new tutorial about rsyslog. In your new tutorial, if can, please provide more example on how to logs remote machines logs to rsyslog server. For example, logs windows logs, firewall logs, centos logs, wifi logs and debian logs.

  9. abdulmalik
    May 28, 2011 at 4:07 am

    Am I install and configure rsyslog on UBUNTU as above given method or something is different

    • May 28, 2011 at 6:31 pm

      Hi abdulmalik,

      I am running Ubuntu 10.10 on some of my client PCs and it looks like rsyslog is installed by default. The configuration should basically be the same, on the rsyslog server the best option is to uncomment the following directives in the /etc/rsyslog.conf file:

      # Provides UDP syslog reception
      $ModLoad imudp.so
      $UDPServerRun 514

      # Provides TCP syslog reception
      $ModLoad imtcp.so
      $InputTCPServerRun 514

      One big difference I see is that on Ubuntu the directives for logging to actual files is broken out in under the /etc/rsyslog.d/*.conf files.

      Cheers,

      Aaron

  10. abdulmalik
    May 28, 2011 at 7:26 pm

    Thank You very much,

    I have configured rsyslog server and snare agent for windows system but following fields are missing in log analyzer
    Event Type, Event Source, Event ID and Event User. Please guide me, how resolve this problem.

  11. abdulmalik
    May 29, 2011 at 3:59 am

    When I select veiw “eventlog feild” follwoing fields in loganalyzer are empty,
    Event Type, Event Source, Event ID and Event User. Please guide me, how resolve this problem

    Thank You

  12. May 30, 2011 at 7:07 am

    Hi abdulmalik,

    I’m stuck at Windows client. I do not know how to send logs to my rsyslog server, could you please guide me.

    • May 30, 2011 at 7:28 pm

      The other good Windows syslog client (in addition to the very good Snare as abdulmalik mentioned) is the Datagram SyslogAgent.

      http://syslogserver.com/syslogagent.html

      It is based on the old NTsyslog. What I like about it is that you can choose different facilities for different event log sources, I believe you can only select one syslog facility for them all with the free Snare client. Also, Datagram’s agent optionally will check if the syslog/rsyslog server is up and will spool the syslog messages locally until the server is back online. AFAIK this support is not available with Snare unless you are using it with the payware Event Reporter.

      Cheers,

      Aaron

  13. abdulmalik
    May 30, 2011 at 7:44 am

    You can download event forwarding agent from following url,
    If you want your Windows 2000 – Server 2008 machines to send logs to your new central log server then check out eventlog-to-syslog. Just download it, unzip it, copy evtsys.exe to C:\Windows\system32\ and execute

    C:\Windows\system32>evtsys.exe -i -h ip.address.of.myserv

    https://engineering.purdue.edu/ECN/Resources/Documents/UNIX/evtsys/

    You can also use snare agent for windows, download from following url,

    http://www.intersectalliance.com/projects/index.html

    In networking settings change port to 514 and putt syslog server IP and lease check syslog header

  14. abdulmalik
    May 30, 2011 at 6:27 pm

    I have configured Rsyslog with RELP and Log analyzer, now facing problem in eventlog feild veiw there are some fields are missing like eventlog type, event user, source event and event Id, can anyone help in this regards, here is my rsyslog.conf file,

    # /etc/rsyslog.conf Configuration file for rsyslog.
    #
    # For more information see
    # /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html
    #
    # Default logging rules can be found in /etc/rsyslog.d/50-default.conf

    #################
    #### MODULES ####
    #################

    $ModLoad imuxsock # provides support for local system logging
    $ModLoad imklog # provides kernel logging support (previously done by rklogd)
    #$ModLoad immark # provides –MARK– message capability

    $KLogPath /proc/kmsg

    # provides UDP syslog reception
    $ModLoad imudp
    $UDPServerRun 514

    # provides TCP syslog reception
    $ModLoad imtcp
    $InputTCPServerRun 514

    ###########################
    #### GLOBAL DIRECTIVES ####
    ###########################

    #
    # Use traditional timestamp format.
    # To enable high precision timestamps, comment out the following line.
    #
    $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

    # Filter duplicated messages
    $RepeatedMsgReduction on

    #
    # Set the default permissions for all log files.
    #
    $FileOwner syslog
    $FileGroup adm
    $FileCreateMode 0640
    $DirCreateMode 0755
    $Umask 0022
    $PrivDropToUser syslog
    $PrivDropToGroup syslog

    #
    # Include all config files in /etc/rsyslog.d/
    #
    $IncludeConfig /etc/rsyslog.d/*.conf

    # Buffering stuff:
    $WorkDirectory /var/rsyslog/work # default location for work (spool) files
    $ActionQueueType LinkedList # use asynchronous processing
    $ActionQueueFileName dbq # set file name, also enables disk mode
    $ActionResumeRetryCount -1 # infinite retries on insert failure

    lease note i am not using event reporter

    • May 30, 2011 at 7:50 pm

      One thing you may want to try is test sending your Windows logs to a regular text file on your rsyslog server and see if the missing fields show up in there. There is a possibility that there is an incompatibility between how rsyslog is saving the messages into MySQL and how Snare is formatting the messages. Unfortunately I may be of limited help because I am no longer using Snare for my Windows syslog client, currently I am using the Datagram SyslogAgent:

      http://syslogserver.com/syslogagent.html

      You may want to give it a try and see if it works any better for you.

      Cheers,

      Aaron

  15. Jason
    June 3, 2011 at 2:43 pm

    I have a problem. I follow the tutorial, but when i try to enter the http://localhost/loganalyzer page i get this message

    You don’t have permission to access /loganalyzer on this server.

    Someone can help me.

    Many Thanks

    • June 7, 2011 at 11:39 pm

      Hi Jason,

      Sorry for my delay, I am actually not running loganalyzer at the moment and wanted to check on an old server if I had any trouble like you did. One thing you may want to check is whether SELinux is set to enforcing. If so there may be an issue with the context label on the loganalyzer files, and may prevent apache displaying the loganalyzer web pages. You can temporarily set it to permissive with “setenforce 0”. I have noticed that SELinux seems to have a problem if a directory is moved into the /var/www/html apache root, but if the files are extracted from the archive directory to the web root directly everything seems to work fine.

      On my old install of loganalyzer I don’t have any special permissions set on the files or directory, so I don’t believe that would be the source of your problem.

      Cheers,

      Aaron

    • Russell
      November 22, 2011 at 2:58 pm

      I had the same problem.
      You have to disable the SELinux. To do it open /etc/selinux/config using your favorite text editor and change the line
      from
      SELINUX=enforcing
      to
      SELINUX=disabled

      Hope it will solve your problem.

    • zac
      February 11, 2012 at 5:54 pm

      resetting permissions with
      chcon -Rv –type=httpd_sys_content_t /var/www/html
      got me in without turning off selinux

  16. Jason
    June 8, 2011 at 1:24 pm

    Ok, thanks… I cant solve the problem… I never work with loganalyzer.

  17. December 29, 2011 at 9:16 am

    Great tutorial and informations. I have installed loganalyzer and works just fine. Thanks!

  18. January 25, 2012 at 6:14 pm

    If you want a cross-platform tool which can collect on Windows and Linux, take a look at nxlog which is a great open source tool I’m using with success. http://nxlog

  19. bobby
    February 28, 2012 at 2:46 pm

    Its a great article, I was able to setup rsyslog and log analyer,
    I have 2 questions:
    1. How can I configure loganalyzer to populate by reading multiple log files located under different location.
    2. I have cisco switches as client, I need to point them to log server. I am not sure whether I can install rsyslog package on those switches, any idea on this ??/

    Thanks.

  20. July 3, 2012 at 2:43 pm

    its gr8 article i have read & configured it . Now i want to configure the logs of cisco switch to this server. Can you please guide in this.

    Thanks

  21. Melvin
    June 8, 2013 at 3:36 pm

    I have a problem in loganalyzer in the index.php “no syslog records found – Error Details:”

    No syslog records found

    Please help me…!!

  1. March 4, 2011 at 10:18 am
  2. August 12, 2011 at 1:58 pm

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: