The Watchful Eye of FAM
by Q Ethan McCallum12/16/2004
In the past, people have often asked how to track changes in a file or
directory from native code. For example, "When this file appears, I want my app
to kick off the process. How would I do that?" If I'd had FAM back then, I could have
forgone loops of alternating select() and sleep()
calls and called it a day.
FAM is the File Alteration Monitor: it watches files and directories for you, alerting your code to events such as removal, change, and execution. FAM is a building block API that lends itself to several possibilities: a file manager could use it to track directory contents; another app could kick off a batch process when a file arrives from a remote host; and I provide an example here that uses FAM to track a log file for real-time processing.
SGI created FAM and develops it under Irix
and Linux. It's not quite POSIX-portable, but it reportedly works under several
other operating systems. While SGI has released the FAM daemon under the GPL,
the libfam client library uses the GNU Lesser General Public
License (aka the GNU Library License). Apps that simply use FAM to
watch files don't fall under the standard GPL's virality clause.
In this article I'll explain how to configure a system for FAM (including a brief troubleshooting guide), show you how to watch files, and describe a quirk of watching directories. I'll wrap up with a tour of the aforementioned log-watcher tool.
I tested the sample code under Fedora Core 2 using
fam-2.6.10-9.FC2, fam-devel-2.6.10-9.FC2,
portmap-4.0-59, and xinetd-2.3.13-2. Different OSes
may require different packages.
FAM's client library is written in C. Though I use C++ in the examples, mostly as a programming preference, people with moderate C experience should have no trouble understanding the code. Please download the examples and follow along.
|
Related Reading 
Linux Cookbook |
How FAM Works
FAM implements the Observer design pattern: your code (the observer) binds to FAM (the subject) and requests notification of changes (events) in certain files or directories. I call these files and directories watch targets, for lack of a better term.
Functionally, the FAM daemon tracks watch targets and listens for requests from client code. Applications use the client API to register targets and receive event notifications. The client and daemon communicate via RPC (Remote Procedure Calls).
FAM tracks watch targets using a kernel monitor if possible, falling back to
select() calls otherwise. (The DNotify kernel monitor is
standard in Linux 2.4 and later.) While select() may be less
efficient than a kernel monitor, FAM's layer of abstraction spares the
application developer from coding for both situations.
Configuring Your System for FAM
FAM's RPC ties require that your system have the portmapper installed, as
well as a superserver such as inetd or xinetd. When asked
to monitor a file on an NFS mount, the local FAM daemon will proxy the request
to the remote server's daemon. Set local_only=false in
/etc/fam.conf to disable this behavior.
The example code includes a sample xinetd configuration for FAM.
The directive bind=127.0.0.1 means that xinetd will accept FAM requests only from the local host. Comment out this line if you want to
provide information for remote FAM clients.
(Properly securing FAM, xinetd, and the portmapper is beyond the scope of this article. These instructions are just enough to make FAM run on your machine.)
Compile and run the stub program step1 to test your FAM setup. Don't worry about the inner workings of step1 yet; just make sure that it compiles and runs successfully. If step1 fails for any reason, refer to the sidebar Troubleshooting a FAM Setup.
Watching Files
The stub program step2 demonstrates a simple FAM client: it's a "talking" tool that announces events on the filenames provided on the command line. I've sacrificed fancy design for readability.
Lines 45 and 46 declare some global variables. fc is a
FAMConnection object, or FAM file descriptor. fr is
a FAMRequest, or FAM request ID. You need just one
FAMRequest per application if you're watching only files (but not
directories). The event loop inside main() checks the Boolean
runFam. I'll return to this momentarily.
FAMOpen() connects to the FAM service:
FAMOpen( fc ) ;Call this once per program.
FAMMonitorFile() registers filenames of interest:
FAMMonitorFile( fc , file , fr , NULL ) ;fc and fr are the FAMConnection and
FAMRequest objects defined earlier. file is the
filename to register. The last parameter is user-defined data included in the
event object (described below). It's useful in more complex apps that create
FAM-related objects in one scope but use them in another. This example doesn't
use such state data, so this value is NULL.
If there are no files to watch--that is, the program can't access them, or
FAMMonitorFile() yields an error--step2 exits because it
has nothing to do (lines 123 to 150).
A FAMEvent object encapsulates an event on a watched target.
Its member variables include the watch target's name, an event code, and the
user-data parameter set in FAMMonitorFile().
FAMNextEvent() catches FAM events--that is, changes to watch
targets--and populates the provided FAMEvent pointer,
fe, with that information:
FAMNextEvent( fc , fe ) ;The FAMEvent.code member holds event types encoded as symbolic
constants: FAMChanged represents a changed file,
FAMDeleted a deleted file, and so on. React to events of interest
by catching those codes in a switch() block:
// event loop
while( true ){
// wait for an event and store it in "fe"
FAMNextEvent( fc , fe ) ;
switch( fe->code ){
case FAMChanged:
// react to a file change ...
break ;
case FAMDeleted:
// react to a file deletion ...
break ;
// ... other FAM events ...
}
}If you have no interest in a certain event, don't catch it in the
switch(). FAMNextEvent() blocks until it receives an
event. As an alternative, you can use FAMPending() to ensure that an
event is ready before calling FAMNextEvent().
FAMPending() doesn't block, and it returns 1 if
there's at least one event to process. In turn, if FAMPending()
detects a waiting event, FAMNextEvent() will return
immediately:
// event loop using FAMPending()
while( true ){
// ... some other event loop tasks ...
if( 1 == FAMPending( fc ) ){
// this is guaranteed to not block now
FAMNextEvent( fc , fe ) ;
switch( fe->code ){
// ... same as above, using
// FAMNextEvent() on its own
}
}
// .. other event loop tasks
}FAMPending() is more suitable for single threads of execution,
such as part of a general event loop. If your event loop consists of
alternating FAMPending() and sleep() calls, though,
you may as well just let FAMNextEvent() do the waiting for
you.
The function sighandler_SIGINT() (lines 70 to 82) catches
SIGINT signals (control-C). It calls
FAMCancelMonitor() to cancel the FAM monitoring and sets
runFam to false to end the event loop:
void sighandler_SIGINT( int sig ){
...
FAMCancelMonitor( fc , fr ) ;
runFam = false ;
...
}FAMCancelMonitor() generates an event that triggers
FAMNextEvent() one last time, which forces the testing of
runFam's value. Finally, FAMClose() terminates the
app's connection to the FAM daemon.
Supply step2 with the names of some files and watch it in action:
$ ./step2 file1 file2 file3The first item of interest is that FAM requires fully qualified paths of watch targets. Attempts to register relative paths yield (misleading) "permission denied" errors. Perhaps a better command line would be:
$ ./step2 ${PWD}/file1 ${PWD}/file2 ${PWD}/file3I encourage you to experiment with step2's monitored files until
you understand which actions trigger event notifications. For example, you can
touch files to update their timestamps, delete them, change their
permissions with chmod, and so on. It's especially interesting to
watch the events that occur for hard links to a watched file.
Lessons that I've learned from watching step2's output include:
FAM watches files based on name, not inode. Renaming a watched file thus results in a
FAMDeletedevent, and you don't receive any more notification under the new filename. (This also explains why theFAMDeletedandFAMCreatedevents exist.)Even a change in permissions (
chmod) appears as a file change. That means FAM tracks some filesystem metadata in addition to the file itself.Each file registered using
FAMMonitorFile()throws aFAMExistsandFAMEndExistevent. As explained below, this is slightly different--and more useful--when watching directories.Formally shutting down the monitor with
FAMCancelMonitor()sends aFAMAcknowledgeevent.Client code can register interest in any target that is accessible, though not necessarily readable, to the program's effective user ID. FAM doesn't grant client code read access for those watch targets, though.
FAM doesn't follow symbolic links of watched files, though it does catch changes on hard links. For example, removing a hard link to a file yields a file-changed event, because that file's link count has decreased.
Watching Directories
Watching a directory means reporting events on its immediate children as
well as the directory itself. It's very similar to watching files, except that
you call FAMMonitorDirectory() instead of
FAMMonitorFile(). The stub program step3 demonstrates
this.
Calling FAMMonitorDirectory() on a file doesn't fail. (The same
goes for calling FAMMonitorFile() for a directory.) FAM will still
report some events, though. Have your code stat() or
lstat() the target to determine which FAM call to use.
Registering a directory with FAM will report several
FAMExists events: one for the directory itself, plus one for each
element contained therein. (This includes hidden elements, except for the
special directory entries . and ...) A single
FAMEndExist event marks the end of this list. Such information can
be useful for statistics, such as tracking the number of a directory's child
elements. step3 catches these in the main event loop (lines 216 to 287),
but you can also catch them when the directory is first registered (the
commented-out lines 168 to 186). The latter approach is helpful when the FAM code
does not run in its own thread.
Watching several directories in the same app presents a conundrum: the
FAMEvent.filename member refers to the filename relative to the
watched directory, yet FAMEvent has no member for the directory
itself.
This is one case where the userdata parameter of
FAMMonitorDirectory() comes in handy. Lines 143 to 155 of
step3 store a C++ string pointer, which the event loop
later retrieves from the FAMEvent object:
while( runFam ){
int rc = FAMNextEvent( fc , fe ) ;
if( 1 != rc ){
std::cerr << "FAMNextEvent returned error"
<< std::endl ;
continue ;
}
std::string* dir =
reinterpret_cast< std::string* >( fe->userdata ) ;
// ... switch() block, same as before ...
}The reinterpret_cast turns the void* into a usable
std::string. Straight C code would use a plain cast instead.
step3 demonstrates a loop based on FAMPending(),
though it would work better for straight FAMNextEvent() calls
because there's nothing else in the event loop.
To operate on the file, concatenate the directory and filename to create the
fully qualified pathname. C developers could use strncat(), while
C++ offers std::ostringstream objects.
Watching a Log File
For text-based log files, the notion of an update means the addition of new entries (lines). A FAM-based application could track such a log file and provide real-time processing of its entries.
The stub program app is an example of such a tool. The FAM event loop is similar to that of step2, but another class does the heavy lifting, which I'll explain shortly.
app fires the target->process() member function for
each file-change event:
// event loop:
while( runFam ){
FAMNextEvent( fc , fe ) ;
if( FAMChanged == fe->code ){
target->process() ;
}
}(In a real app, of course, the event loop would run in a separate thread.)
target is a Handler object. The
Handler class is a pure-virtual (or interface),
which means it only provides member function declarations. The provided
implementation thereof, EchoHandler, simply prepends
Line: to each line that it reads from the watched file. Feel free
to swap in your own implementation: inherit from Handler and write
a process() member function.
Most of EchoHandler::process() is C++ magic to keep track of
the current place in the file. I won't explain that here, as this article isn't
about the C++ iostreams library. If you're curious, peruse the
source file's comments.
That's a Wrap
FAM provides developers a means to track changes in files and directories. Its client API is fairly straightforward and clean, which makes it easy to fold into other applications. You could certainly write your own framework to watch files; but since FAM exists, you have one less reason to do this.
Resources
Download this article's sample code. This includes the source code for the various programs (step1, step2, step3, and app), and the sample xinetd and FAM configurations.
The FAM website provides online documentation and source code.
The original "Gang of Four" Design Patterns book describes the Observer design pattern. Other circles know this pattern as Publish/Subscribe.
Troubleshooting a FAM SetupUse the following steps to resolve any errors reported by the step1 stub tool.
|
Q Ethan McCallum grew from curious child to curious adult, turning his passion for technology into a career.
Return to the Linux DevCenter.
