= Using syslogd and logrotate to manage the server's log files = First note that on our machines the newer syslogd-ng is used as a more powerful replacement for the traditional syslogd. See `man syslog-ng` and `man syslog` for all details. == Facilities and severity levels == syslogd uses '''selectors''' to identify the program (called the '''facility''') that is sending a log message and the message's severity '''level'''. So a selector has the general form ''facility.level''. For example, the selector ''mail.warning'' specifies that a program that somehow deals with mail (a mail reader, a mail transfer agent, a random mail producer, ...) wants to log a warning message. Both facility names and levels must be chosen from a short list of predefined values. '''''No, you cannot invent your own facilities or levels'''''. You have to use the predefined levels, and you have to group your programs into the predefined facility groups. (But syslog-ng allows you to filter messages by regular expression matches, which you could use to filter out all messages that exclusively belong to a program called "therollingstones1965" by prepending this unique string to all messages of this and only this program.) Facilities are defined for the kernel, for common groups of utilities, and for locally written programs. Everything else is classified under the generic facility ''user''. The following facilities exist as defined in `/usr/include/sys/syslog.h`: {{{ /* facility codes */ #define LOG_KERN (0<<3) /* kernel messages */ #define LOG_USER (1<<3) /* random user-level messages */ #define LOG_MAIL (2<<3) /* mail system */ #define LOG_DAEMON (3<<3) /* system daemons */ #define LOG_AUTH (4<<3) /* security/authorization messages */ #define LOG_SYSLOG (5<<3) /* messages generated internally by syslogd */ #define LOG_LPR (6<<3) /* line printer subsystem */ #define LOG_NEWS (7<<3) /* network news subsystem */ #define LOG_UUCP (8<<3) /* UUCP subsystem */ #define LOG_CRON (9<<3) /* clock daemon */ #define LOG_AUTHPRIV (10<<3) /* security/authorization messages (private) */ #define LOG_FTP (11<<3) /* ftp daemon */ #define LOG_LOCAL0 (16<<3) /* reserved for local use */ #define LOG_LOCAL1 (17<<3) /* reserved for local use */ #define LOG_LOCAL2 (18<<3) /* reserved for local use */ #define LOG_LOCAL3 (19<<3) /* reserved for local use */ #define LOG_LOCAL4 (20<<3) /* reserved for local use */ #define LOG_LOCAL5 (21<<3) /* reserved for local use */ #define LOG_LOCAL6 (22<<3) /* reserved for local use */ #define LOG_LOCAL7 (23<<3) /* reserved for local use */ }}} (So the names of the facilities are ''kern'', ''user'', and so on.) The following severity levels exist, in descending order of severity: {{{ #define LOG_EMERG 0 /* system is unusable */ #define LOG_ALERT 1 /* action must be taken immediately */ #define LOG_CRIT 2 /* critical conditions */ #define LOG_ERR 3 /* error conditions */ #define LOG_WARNING 4 /* warning conditions */ #define LOG_NOTICE 5 /* normal but significant condition */ #define LOG_INFO 6 /* informational */ #define LOG_DEBUG 7 /* debug-level messages */ }}} (So the names of the levels are ''emerg'', ''alert'', and so on.) The severity level of a message specifies its importance. In the `syslog-ng.conf` file, levels indicate the ''minimum'' importance that a message must have in order to be logged upon a call to the `syslog()` function. For example, if `syslog-ng.conf` specifies that ''mail.info'' messages be logged to a file, then ''mail.warning'' messages will go there also unless there is a separate rule for these higher level messages. == Configuring syslogd-ng == The configuraton file for syslogd-ng is `/etc/sylog-ng/syslog-ng.conf`. You must be root to edit this file. Suppose that we want to use the syslogd-ng to log messages for our own server, no matter what their severity level is. We decide to choose the (predefined!) `local0` facility for our server program. (This is tantamount to say that our server belongs to a group of programs called `local0`.) We have to specify a destination file for the log, set up a filter that filters out all messages that originate from `local0` programs, and connect the source of all log messages via the filter to the destination: {{{ # file /etc/sylog-ng/syslog-ng.conf destination myserverlog { file("/var/log/myserverlog" owner("root") group("adm") perm(0640)); }; filter f_myserver { facility(local0); }; # all priorities match! log { source(src); filter(f_myserver); destination(myserverlog); }; }}} Of course you will ask now what the source is. According to one of the first lines in `/etc/sylog-ng/syslog-ng.conf`: {{{ # This is the default behavior of sysklogd package # Logs may come from unix stream, but not from another machine. # source src { unix-stream("/dev/log"); internal(); file("/proc/kmsg" log_prefix("kernel: ")); }; }}} (As a user, you do not have worry about the source. The `syslog()` function will write its messages to the `/dev/log` stream.) Note that with syslog-ng you can write very complex filter rules to filter out only the messages that you really want. These rules even allow regular expression matching. A common use is not only to filter by facility, but also by level. Now restart syslogd-ng: {{{ # kill -HUP `/bin/cat /var/run/syslog-ng.pid` }}} == Logging from the command line == You can test the newly created logging selector by the `logger` program. You do not have to be root to do this! (Of course, any user level program should be able to log its messages.) We log a demo message from a ''local0'' facility with the ''info'' level: {{{ $ logger -p local0.info "This is a test line" }}} {{{ # tail -f /var/log/myserverlog Nov 18 20:00:55 mpino1301 logger: This is a test line }}} Note that there is no information about the facility and the level in the log file. (It is implicitly clear by the rules above that `myserverlog` only contains `local0` messages.) If we only wanted to log ''info'' messages, we could create a separate destination file `myserverlog.info` together with a corresponding filter rule to filter out ''info'' messages of ''local0'' facilities. == Logging from C programs == Here is a short C program (`syslogdemo.c`) that opens the log for the `local0` facility, writes a log messages with the `warning` level into the log, then writes a log messages with the `err` level into the log, and finally closes the log again. {{{ #include int main(int argc, char**argv) { openlog(argv[0], /* string constant prepended to every message */ LOG_PID, /* option: include PID with each message */ LOG_LOCAL0 /* facility to log */); syslog(LOG_WARNING, "Just logged %s", "this warning."); syslog(LOG_ERR, "There were %d floating point errors.", 4711); closelog(); } }}} The manual page of `syslog` will give you all the details (but there are not many). The log file now ends by the follwing lines: {{{ Nov 18 20:30:14 mpino1301 ./syslogdemo[21388]: Just logged this warning. Nov 18 20:30:14 mpino1301 ./syslogdemo[21388]: There were 4711 floating point errors. }}} Note that these lines contain the name of the program (`syslogdemo`) and its PID (21388). Also note that neither the facility name nor the severity level are contained in these lines. By the rules in `syslog-ng.conf` above, this information is implicitly contained in the file name. If we only wanted to log `warning` messages, we could introduce a separate destination file `myserverlog.warning` together with a corresponding filter: {{{ # file /etc/sylog-ng/syslog-ng.conf destination myserverlogwarn { file("/var/log/myserverlog.warning" ... }; filter f_myserverwarn { facility(local0) and level(warning); }; log { source(src); filter(f_myserverwarn); destination(myserverlogwarn); }; }}} With these ''additional'' lines in `/etc/sylog-ng/syslog-ng.conf`, ''both'' files `myserverlog` and `myserverlog.warning` would be filled by the above C program! == Using logrotate to keep the log file small == At some point in time, the server's log file may become too large. Then it is time to use the `logrotate` utility. Edit the file `/etc/logrotate.conf` and insert the following lines. This configuration rotates `/var/log/myserverlog` once every week, keeping five versions of the file (one current and four older ones). All noncurrent versions of the log file are compressed. The log file is not rotated if it is empty. Error notifications are sent to `bast@mpi-inf.mpg.de`. {{{ /var/log/myserverlog { weekly rotate 5 compress notifempty errors bast@mpi-inf.mpg.de } }}} `logrotate` is normally run out of `cron` once a day. `man logrotate` gives you all the details.