ONLamp.com    
 Published on ONLamp.com (http://www.onlamp.com/)
 See this if you're having trouble printing code examples


FreeBSD Basics

Where the Log Files Live

11/08/2000

In today's article, I'd like to tie together some of the concepts we've learned so far from the previous articles in the series. Let's apply our newfound skills to see what we can find out about FreeBSD and system logs.

You know there are logs on your FreeBSD system somewhere; you've probably also heard that it is a good thing to read these logs on a regular basis. You may have even heard horror stories about logs filling up a user's hard drive. So how do we go about finding these mysterious logs? Let's start by taking a look at the layout of our FreeBSD system using the trusty old command:

man hier

Need some help viewing manpages?

Read The Friendly Manpage! -- A Tutorial

Read The Friendly Manpage! -- Part Two

We'll then search for the word "log" within this manpage by typing:

/log

Our first hit shows that the multi-purpose logs live in /var:

/var/ multi-purpose log, temporary, transient, and spool files

If you repeat the search by repeating the "n" key twice, you'll see that there are 2 subdirectories of /var that contain logs: cron and log.

cron/

  log     cron log files; see cron(8)

log/	 misc. system log files

If you repeat the search one more time by pressing the "n" key, you'll get a "Pattern not found" message, so it looks like we've found all the logs that came with our directory structure.

We're interested in the system log files, so let's take a look at the contents of /var/log:

ls /var/log
cron            messages        setuid.today
dmesg.today     ppp.logs        setuid.yesterday
dmesg.yesterday security        slip.log
lpd-errs        sendmail.st     wtmp
maillog         sendmail.st.0

For more on permissions, see:

An Introduction to Unix Permissions

An Introduction to Unix Permissions -- Part Two

Your output may vary slightly depending upon your version of FreeBSD, which ports you have installed on your FreeBSD system, and how long it's been since you've been in this directory. Being the curious type, you'll probably want to have a peek at each log to see what it contains. But before you start looking at files you didn't create, you'll want to first check that you have permission to view their contents by typing:

ls -l
total 324
drwxr-xr-x  3 root  wheel  1024 Nov  5 00:00 ./
drwxr-xr-x 18 root  wheel   512 Sep 26 10:53 ../
-rw-------  1 root  wheel 81964 Nov  5 09:15 cron
-rw-r-----  1 root  wheel  3435 Nov  3 02:06 dmesg.today
-rw-r-----  1 root  wheel  3382 Nov  2 02:06 dmesg.yesterday
-rw-rw-r--  1 root  wheel     0 Jul 28 09:10 lpd-errs
-rw-rw-r--  1 root  wheel 16821 Nov  5 08:41 maillog
-rw-rw-r--  1 root  wheel 78888 Nov  5 08:40 messages
-rw-------  1 root  wheel 80332 Oct 30 14:17 ppp.log
-rw-------  1 root  wheel     0 Jul 28 09:10 security
-rw-rw-r--  1 root  wheel   616 Nov  5 08:41 sendmail.st
-rw-rw-r--  1 root  wheel   616 Nov  4 19:33 sendmail.st.0
-rw-r-----  1 root  wheel  7791 Nov  3 02:06 setuid.today
-rw-r-----  1 root  wheel  6587 Nov  2 02:06 setuid.yesterday
-rw-------  1 root  wheel     0 Jul 28 09:10 slip.log
-rw-r--r--  1 root  wheel  2684 Nov  2 21:12 wtmp

It looks like a regular user only has permission to view about half of the log files. If that user lives in the wheel group, they can view a few more, but only the superuser can view all of the system log files.

One last thing before looking at these files: You did not make these files, so you don't know what type of data they contain. Remember, you never open up an unknown file without first testing it with the file utility. Let's check the whole directory at once:

file *
cron:             ASCII text
dmesg.today:      English text
dmesg.yesterday:  English text
lpd-errs:         empty
maillog:          ASCII text
messages:         English text
ppp.log:          mail text
security:         empty
sendmail.st:      data
sendmail.st.0:	  data
setuid.today:     ASCII text
setuid.yesterday: ASCII text
slip.log:         empty
wtmp:             data

Any file that has a type called data is usually non-printable. This means that you shouldn't try to send the sendmail* or wtmp files to your terminal using the more or cat commands or your favorite editor. It looks like the lpd-errs, security and slip.log files are empty. The other files contain text, so they can be viewed by any user who has "r" permission to that file. Some of these files are quite large; if you are only concerned with the last bit, that is, the most recent part of the log, use the tail command like so:

tail maillog

This will display the last 10 lines of the maillog file; you'll note that maillog has very long lines that will wrap around your screen.

Now you know which log files you can safely look at and satisfy your curiosity regarding their contents. But who put that information into those log files, and how can you specify what type of information you'd like to see in your log files? Sounds like we need to use the apropos command to see which manpages will shed some light on this subject. If you type:

apropos system log

you'll receive a couple of screens full of possible manpages. Let's narrow our search a bit by adding some quotation marks:

apropos "system log"

The " " tell apropos that you only want manpages that contain both words; the original search told apropos to return manpages that contained either word. This last search yielded a lot fewer results:

logger(1) - make entries in the system log
newsyslog(8) - maintain system log files to manageable sizes
syslog(3), vsyslog(3), openlog(3), closelog(3), setlogmask(3) - control system log

We seem to be getting closer; it appears that FreeBSD uses the word "syslog" instead of system logs. Let's try:

apropos syslog
newsyslog(8) - maintain system log files to manageable sizes
syslog(3), vsyslog(3), openlog(3), closelog(3), setlogmask(3) - control system log
syslog.conf(5) - syslogd 8 configuration file
syslogd(8) - log systems messages
Sys::Syslog(3), openlog(3), closelog(3), setlogmask(3), syslog(3) - Perl interface to the UNIX syslog|(3) calls

And we've hit paydirt; syslogd is the daemon responsible for logging system messages, and syslog.conf is its configuration file.

So, do you have permission to muck about with this syslog.conf file? To find out, type:

ls -l /etc/syslog.conf
-rw-r--r--  1 root  wheel  903 Jul 28 09:10 /etc/syslog.conf

Looks like anyone can read it, but only the superuser will be able to modify it. Let's just take a peek at it as a regular user using the more command:

more /etc/syslog.conf

View the results from this command here.

Some parts of this file are self-explanatory, but we'll have to take this file's advice and read the syslog.conf(5) manpage before we start making any changes to it. I'll cover the highlights; you'll have to do a man 5 syslog.conf to get all the details.

Each line in syslog.conf has two fields: the selector field (on the left), which specifies the type of message, and an action field (on the right), which specifies the action to be taken if a message matches the selection criteria. The selector field is separated from the action field by one or more tabs.

The selector field itself is divided into two parts separated by a period, like this:

facility.level

where facility is what generated the message and level is the severity of the message. The possible values for facilities and levels are explained in syslog(3). The following tables provide a summary of the facilities and levels:

Table 1: Facilities

Facility Name

What Program It Represents

AUTH

The authorization system: login(1), su(1), getty(8), etc.

AUTHPRIV

The same as AUTH, but logged to a file readable only by selected individuals.

CRON

The cron daemon: cron(8).

DAEMON

System daemons, such as routed(8), that are not provided for explicitly by other facilities.

FTP

The file transfer protocol daemons: ftpd(8), tftpd(8).

KERN

Messages generated by the kernel. These cannot be generated by any user processes.

LPR

The line printer spooling system: lpr(1), lpc(8), lpd(8), etc.

MAIL

The mail system.

NEWS

The network news system.

SECURITY

Security subsystems.

SYSLOG

Messages generated internally by syslogd(8).

USER

Messages generated by random user processes. This is the default facility identifier if none is specified.

UUCP

The uucp system. ipfw(4).

*

Specifies all facilities or programs except mark.

MARK

A special facility used by syslogd.



Table 2: Levels

Level Name

What It Represents

EMERG

A panic condition. This is normally broadcast to all users.

ALERT

A condition that should be corrected immediately, such as a corrupted system database.

CRIT

Critical conditions, e.g., hard device errors.

ERR

Errors.

WARNING

Warning messages.

NOTICE

Conditions that are not error conditions, but should possibly be handled specially.

INFO

Informational messages.

DEBUG

Messages that contain information normally of use only when debugging a program.

NONE

Special level to disable the facility.

Let's use these tables to decipher what type of messages are sent to the console, that is, those irritating bold-white messages that show up on your first terminal. The corresponding line in /etc/syslog.conf reads as:

*.err;kern.debug;auth.notice;mail.crit  /dev/console

Note that the selector field is a bunch of "facility.levels" tied together with semicolons. Reading from left to right, this line tells syslogd to send the following types of messages to the console:

You should be able to use the same logic to see what types of messages are sent to which logs in the next five lines:

*.notice;kern.debug;lpr.info;mail.crit;news.err /var/log/messages
security.*    /var/log/security
mail.info     /var/log/maillog
lpr.info      /var/log/lpd-errs
cron.*        /var/log/cron

However, to understand the rest of the file, we need to know the five possible actions listed in the following table:

Table 3: Actions

Syntax of Action

What It Does

/pathname

Messages are added to the end of the specified file.

@hostname

Messages are forwarded to the syslogd(8) program on the specified computer.

user1,user2,etc.

Messages are written to those users if they are logged in.

*

Messages are written to all logged-in users.

|command

Pipes the message to the specified command.

Also note that two of the lines in the file begin with an exclamation mark and don't have an action field after them like so:

!startslip
*.*              /var/log/slip.log
!ppp
*.*              /var/log/ppp.log

Occasionally you'll want to log the messages of a program that isn't covered by one of the built-in facilities. To add this program to /etc/syslog.conf, type the name of the program's executable on a line by itself with a ! in front of the name. On the next line, input the selector and action fields as you normally would.

You'll find that man 5 syslog.conf has excellent examples covering just about every scenario you'll ever come across when configuring logging. If you do ever edit your syslog.conf file, you will have to send a HUP signal to syslogd to force it to read the changes. To do this:

more /var/run/syslog.pid

and make note of the number you receive; this is the PID (process ID) being used by the syslog daemon. Then, as the superuser:

kill -1 number

where number is the number you received above.

The last thing I want to cover today is the newsyslog utility. When you originally looked in your /var/log directory, you may have received a listing of a lot of files that ended in .0, .1, etc., and some of these files may have also been compressed (they had a .gz extension). This is a result of the workings of newsyslog, which we mentioned briefly in the Getting Cron to Do Our Bidding article. Let's take a quick look in this utility's manpage:

man newsyslog

NAME

newsyslog - maintain system log files to manageable sizes

Newsyslog is a program that should be scheduled to run periodically by cron(8). When it is executed it archives log files if necessary. If a log file is determined to require archiving, newsyslog rearranges the files so that "logfile" is empty, "logfile.0" has the last period's logs in it, "logfile.1" has the next to last period's logs in it, and so on, up to a user-specified number of archived logs. Optionally, the archived logs can be compressed to save space.

In other words, if a logfile becomes too large, newsyslog will rename it with a .0 extension, possibly zip it, and create a new file with the original log name. For example:

If you continue to read through the manpage for newsyslog, you'll learn how to tweak its configuration file (/etc/newsyslog.conf) so you can schedule when files will be renamed and compressed.

If you ever need to view the contents of a log that has already been compressed by newsyslog, you can use the zmore utility like so:

zmore maillog.0.gz
If you need to remove old log files to save space, it is safe to delete a log that ends with a either a number or a .gz from the /var/log directory. If you need to do this often, there is no need to create a cronjob; newsyslog will do this automatically. It will keep as many or as few backlogs as you desire and rotate through them when they reach a specified size. I would not recommend deleting the other logs, though, as syslogd expects to be able to find the logfiles in the paths that you've specified in /etc/syslog.conf. So, in the above example, it is safe to delete maillog.0.gz and maillog.1.gz, but don't delete maillog.

If you ever inadvertently delete an original logfile, you can create it using the touch utility:

cd /var/log
rm maillog	(oops)
touch maillog

This will create an empty maillog file that syslogd can write to.

This should get you started working with logs on your FreeBSD system. In next week's article we'll dig a little deeper and take a look at processes, PIDs, and the ps utility.

Dru Lavigne is a network and systems administrator, IT instructor, author and international speaker. She has over a decade of experience administering and teaching Netware, Microsoft, Cisco, Checkpoint, SCO, Solaris, Linux, and BSD systems. A prolific author, she pens the popular FreeBSD Basics column for O'Reilly and is author of BSD Hacks and The Best of FreeBSD Basics.


Read more FreeBSD Basics columns.

Discuss this article in the Operating Systems Forum.

Return to the BSD DevCenter.

 

Copyright © 2009 O'Reilly Media, Inc.