Logcheck Guide

1. Getting Started With logcheck

logcheck is an updated version of logsentry (from the sentrytools package), which is a tool to analyze the system logs. Additionally, logcheck comes with a built-in database of common, not-interesting log messages to filter out the noise. The general idea of the tool is that all messages are interesting, except the ones explicitly marked as noise. logcheck periodically sends you an e-mail with a summary of interesting messages.

Installing logcheck

Important: It is strongly recommended to remove logsentry if you have it installed on your system. Additionally, you should remove /etc/logcheck to avoid permission and file collision problem.

Code Listing 1.1: Removing logsentry

(Uninstall the logsentry package)
# emerge -C logsentry
(Remove leftover files)
# rm -rf /etc/logcheck

Now you can proceed with the installation of logcheck.

Code Listing 1.2: Installing logcheck

# emerge -av app-admin/logcheck

Basic configuration

logcheck creates a separate user "logcheck" to avoid running as root. Actually, it will refuse to run as root. To allow it to analyze the logs, you need to make sure they are readable by logcheck. Here is an example for syslog-ng:

Code Listing 1.3: /etc/syslog-ng/syslog-ng.conf snippet

options {
        owner(root);

        (Make log files group-readable by logcheck)
        group(logcheck);
        perm(0640);
};

Now reload the configuration and make sure the changes work as expected.

Code Listing 1.4: Reload syslog-ng configuration

# /etc/init.d/syslog-ng reload
(Make sure /var/log/messages has correct permissions)
# ls -l /var/log/messages
-rw-r----- 1 root logcheck 1694438 Feb 12 12:18 /var/log/messages

You should now adjust some basic logcheck settings in /etc/logcheck/logcheck.conf.

Code Listing 1.5: Basic /etc/logcheck/logcheck.conf setup

# Controls the level of filtering:
# Can be Set to "workstation", "server" or "paranoid" for different
# levels of filtering. Defaults to server if not set.
(The workstation level includes server, and server includes paranoid.
The paranoid level filters almost no messages)
REPORTLEVEL="server"

# Controls the address mail goes to:
# *NOTE* the script does not set a default value for this variable!
# Should be set to an offsite "emailaddress@some.domain.tld"
(Make sure you can receive the logcheck e-mails. Testing is strongly
recommended)
SENDMAILTO="root"

# Controls if syslog-summary is run over each section.
# Alternatively, set to "1" to enable extra summary.
# HINT: syslog-summary needs to be installed.
(If you get a lot of similar messages in the logs, you
may want to install app-admin/syslog-summary and enable
this setting)
SYSLOGSUMMARY=0

You also have to tell logcheck which log files to scan (/etc/logcheck/logcheck.logfiles).

Code Listing 1.6: Basic /etc/logcheck/logcheck.logfiles setup

(This is an example for syslog-ng)
/var/log/messages

Finally, enable the logcheck cron job.

Code Listing 1.7: Enable logcheck cron job

(Edit the cron file and follow the instructions inside)
# nano -w /etc/cron.hourly/logcheck.cron

Note: For more information about cron read the Cron Guide.

Congratulations! Now you will be regularly getting important log messages by email. An example message looks like this:

Code Listing 1.8: Example logcheck message

System Events
=-=-=-=-=-=-=
Feb 10 17:13:53 localhost kernel: [30233.238342] conftest[25838]: segfault at 40 ip 40061403 sp bfc443c4 error 4
in libc-2.10.1.so[4003e000+142000]
Feb 11 12:31:21 localhost postfix/pickup[18704]: fatal: could not find any active network interfaces
Feb 11 12:31:22 localhost postfix/master[3776]: warning: process //usr/lib/postfix/pickup pid 18704 exit status 1
Feb 11 12:31:22 localhost postfix/master[3776]: warning: //usr/lib/postfix/pickup: bad command startup -- throttling

2. Troubleshooting

General tips

You can use the logcheck's -d switch to display more debugging information. Example:

Code Listing 2.1: Debugging logcheck

# su -s /bin/bash -c '/usr/sbin/logcheck -d' logcheck
D: [1281318818] Turning debug mode on
D: [1281318818] Sourcing - /etc/logcheck/logcheck.conf
D: [1281318818] Finished getopts c:dhH:l:L:m:opr:RsS:tTuvw
D: [1281318818] Trying to get lockfile: /var/lock/logcheck/logcheck.lock
D: [1281318818] Running lockfile-touch /var/lock/logcheck/logcheck.lock
D: [1281318818] cleanrules: /etc/logcheck/cracking.d/kernel
...
D: [1281318818] cleanrules: /etc/logcheck/violations.d/su
D: [1281318818] cleanrules: /etc/logcheck/violations.d/sudo
...
D: [1281318825] logoutput called with file: /var/log/messages
D: [1281318825] Running /usr/sbin/logtail2 on /var/log/messages
D: [1281318825] Sorting logs
D: [1281318825] Setting the Intro
D: [1281318825] Checking for security alerts
D: [1281318825] greplogoutput: kernel
...
D: [1281318825] greplogoutput: returning 1
D: [1281318825] Checking for security events
...
D: [1281318825] greplogoutput: su
D: [1281318825] greplogoutput: Entries in checked
D: [1281318825] cleanchecked - file: /tmp/logcheck.uIFLqU/violations-ignore/logcheck-su
D: [1281318825] report: cat'ing - Security Events for su
...
D: [1281318835] report: cat'ing - System Events
D: [1281318835] Setting the footer text
D: [1281318835] Sending report: 'localhost 2010-08-09 03:53 Security Events' to root
D: [1281318835] cleanup: Killing lockfile-touch - 17979
D: [1281318835] cleanup: Removing lockfile: /var/lock/logcheck/logcheck.lock
D: [1281318835] cleanup: Removing - /tmp/logcheck.uIFLqU

The contents of this document, unless otherwise expressly stated, are licensed under the CC-BY-SA-2.5 license. The Gentoo Name and Logo Usage Guidelines apply.

Page updated October 12, 2010

Summary: This guide shows you how to analyze system logs with logcheck.

Paweł Hajdan
Author

Joshua Saddler
Editor

Donate to support our development efforts.