Linux DevCenter    
 Published on Linux DevCenter (http://www.linuxdevcenter.com/)
 See this if you're having trouble printing code examples


The Watchful Eye of FAM

by Q Ethan McCallum
12/16/2004

In the past, people have often asked how to track changes in a file or directory from native code. For example, "When this file appears, I want my app to kick off the process. How would I do that?" If I'd had FAM back then, I could have forgone loops of alternating select() and sleep() calls and called it a day.

FAM is the File Alteration Monitor: it watches files and directories for you, alerting your code to events such as removal, change, and execution. FAM is a building block API that lends itself to several possibilities: a file manager could use it to track directory contents; another app could kick off a batch process when a file arrives from a remote host; and I provide an example here that uses FAM to track a log file for real-time processing.

SGI created FAM and develops it under Irix and Linux. It's not quite POSIX-portable, but it reportedly works under several other operating systems. While SGI has released the FAM daemon under the GPL, the libfam client library uses the GNU Lesser General Public License (aka the GNU Library License). Apps that simply use FAM to watch files don't fall under the standard GPL's virality clause.

In this article I'll explain how to configure a system for FAM (including a brief troubleshooting guide), show you how to watch files, and describe a quirk of watching directories. I'll wrap up with a tour of the aforementioned log-watcher tool.

I tested the sample code under Fedora Core 2 using fam-2.6.10-9.FC2, fam-devel-2.6.10-9.FC2, portmap-4.0-59, and xinetd-2.3.13-2. Different OSes may require different packages.

FAM's client library is written in C. Though I use C++ in the examples, mostly as a programming preference, people with moderate C experience should have no trouble understanding the code. Please download the examples and follow along.

Related Reading

Linux Cookbook
By Carla Schroder

How FAM Works

FAM implements the Observer design pattern: your code (the observer) binds to FAM (the subject) and requests notification of changes (events) in certain files or directories. I call these files and directories watch targets, for lack of a better term.

Functionally, the FAM daemon tracks watch targets and listens for requests from client code. Applications use the client API to register targets and receive event notifications. The client and daemon communicate via RPC (Remote Procedure Calls).

FAM tracks watch targets using a kernel monitor if possible, falling back to select() calls otherwise. (The DNotify kernel monitor is standard in Linux 2.4 and later.) While select() may be less efficient than a kernel monitor, FAM's layer of abstraction spares the application developer from coding for both situations.

Configuring Your System for FAM

FAM's RPC ties require that your system have the portmapper installed, as well as a superserver such as inetd or xinetd. When asked to monitor a file on an NFS mount, the local FAM daemon will proxy the request to the remote server's daemon. Set local_only=false in /etc/fam.conf to disable this behavior.

The example code includes a sample xinetd configuration for FAM. The directive bind=127.0.0.1 means that xinetd will accept FAM requests only from the local host. Comment out this line if you want to provide information for remote FAM clients.

(Properly securing FAM, xinetd, and the portmapper is beyond the scope of this article. These instructions are just enough to make FAM run on your machine.)

Compile and run the stub program step1 to test your FAM setup. Don't worry about the inner workings of step1 yet; just make sure that it compiles and runs successfully. If step1 fails for any reason, refer to the sidebar Troubleshooting a FAM Setup.

Watching Files

The stub program step2 demonstrates a simple FAM client: it's a "talking" tool that announces events on the filenames provided on the command line. I've sacrificed fancy design for readability.

Lines 45 and 46 declare some global variables. fc is a FAMConnection object, or FAM file descriptor. fr is a FAMRequest, or FAM request ID. You need just one FAMRequest per application if you're watching only files (but not directories). The event loop inside main() checks the Boolean runFam. I'll return to this momentarily.

FAMOpen() connects to the FAM service:

FAMOpen( fc ) ;

Call this once per program.

FAMMonitorFile() registers filenames of interest:

FAMMonitorFile( fc , file , fr , NULL ) ;

fc and fr are the FAMConnection and FAMRequest objects defined earlier. file is the filename to register. The last parameter is user-defined data included in the event object (described below). It's useful in more complex apps that create FAM-related objects in one scope but use them in another. This example doesn't use such state data, so this value is NULL.

If there are no files to watch--that is, the program can't access them, or FAMMonitorFile() yields an error--step2 exits because it has nothing to do (lines 123 to 150).

A FAMEvent object encapsulates an event on a watched target. Its member variables include the watch target's name, an event code, and the user-data parameter set in FAMMonitorFile(). FAMNextEvent() catches FAM events--that is, changes to watch targets--and populates the provided FAMEvent pointer, fe, with that information:

FAMNextEvent( fc , fe ) ;

The FAMEvent.code member holds event types encoded as symbolic constants: FAMChanged represents a changed file, FAMDeleted a deleted file, and so on. React to events of interest by catching those codes in a switch() block:

// event loop
while( true ){

  // wait for an event and store it in "fe"
  FAMNextEvent( fc , fe ) ;

  switch( fe->code ){

    case FAMChanged:
      // react to a file change ...
      break ;

    case FAMDeleted:
      // react to a file deletion ...
      break ;

    // ... other FAM events ...

  }

}

If you have no interest in a certain event, don't catch it in the switch(). FAMNextEvent() blocks until it receives an event. As an alternative, you can use FAMPending() to ensure that an event is ready before calling FAMNextEvent(). FAMPending() doesn't block, and it returns 1 if there's at least one event to process. In turn, if FAMPending() detects a waiting event, FAMNextEvent() will return immediately:

// event loop using FAMPending()
while( true ){

  // ... some other event loop tasks ...

  if( 1 == FAMPending( fc ) ){
    // this is guaranteed to not block now
    FAMNextEvent( fc , fe ) ;

    switch( fe->code ){

      // ... same as above, using
      // FAMNextEvent() on its own

    }

  }

  // .. other event loop tasks
}

FAMPending() is more suitable for single threads of execution, such as part of a general event loop. If your event loop consists of alternating FAMPending() and sleep() calls, though, you may as well just let FAMNextEvent() do the waiting for you.

The function sighandler_SIGINT() (lines 70 to 82) catches SIGINT signals (control-C). It calls FAMCancelMonitor() to cancel the FAM monitoring and sets runFam to false to end the event loop:

void sighandler_SIGINT( int sig ){
  ...
  FAMCancelMonitor( fc , fr ) ;
  runFam = false ;
  ...
}

FAMCancelMonitor() generates an event that triggers FAMNextEvent() one last time, which forces the testing of runFam's value. Finally, FAMClose() terminates the app's connection to the FAM daemon.

Supply step2 with the names of some files and watch it in action:

$ ./step2 file1 file2 file3

The first item of interest is that FAM requires fully qualified paths of watch targets. Attempts to register relative paths yield (misleading) "permission denied" errors. Perhaps a better command line would be:

$ ./step2 ${PWD}/file1 ${PWD}/file2 ${PWD}/file3

I encourage you to experiment with step2's monitored files until you understand which actions trigger event notifications. For example, you can touch files to update their timestamps, delete them, change their permissions with chmod, and so on. It's especially interesting to watch the events that occur for hard links to a watched file.

Lessons that I've learned from watching step2's output include:

Watching Directories

Watching a directory means reporting events on its immediate children as well as the directory itself. It's very similar to watching files, except that you call FAMMonitorDirectory() instead of FAMMonitorFile(). The stub program step3 demonstrates this.

Calling FAMMonitorDirectory() on a file doesn't fail. (The same goes for calling FAMMonitorFile() for a directory.) FAM will still report some events, though. Have your code stat() or lstat() the target to determine which FAM call to use.

Registering a directory with FAM will report several FAMExists events: one for the directory itself, plus one for each element contained therein. (This includes hidden elements, except for the special directory entries . and ...) A single FAMEndExist event marks the end of this list. Such information can be useful for statistics, such as tracking the number of a directory's child elements. step3 catches these in the main event loop (lines 216 to 287), but you can also catch them when the directory is first registered (the commented-out lines 168 to 186). The latter approach is helpful when the FAM code does not run in its own thread.

Watching several directories in the same app presents a conundrum: the FAMEvent.filename member refers to the filename relative to the watched directory, yet FAMEvent has no member for the directory itself.

This is one case where the userdata parameter of FAMMonitorDirectory() comes in handy. Lines 143 to 155 of step3 store a C++ string pointer, which the event loop later retrieves from the FAMEvent object:

while( runFam ){
  
  int rc = FAMNextEvent( fc , fe ) ;

  if( 1 != rc ){
    std::cerr << "FAMNextEvent returned error"
      << std::endl ;
    continue ;
  }

  std::string* dir =
    reinterpret_cast< std::string* >( fe->userdata ) ;

  // ... switch() block, same as before ...
}

The reinterpret_cast turns the void* into a usable std::string. Straight C code would use a plain cast instead.

step3 demonstrates a loop based on FAMPending(), though it would work better for straight FAMNextEvent() calls because there's nothing else in the event loop.

To operate on the file, concatenate the directory and filename to create the fully qualified pathname. C developers could use strncat(), while C++ offers std::ostringstream objects.

Watching a Log File

For text-based log files, the notion of an update means the addition of new entries (lines). A FAM-based application could track such a log file and provide real-time processing of its entries.

The stub program app is an example of such a tool. The FAM event loop is similar to that of step2, but another class does the heavy lifting, which I'll explain shortly.

app fires the target->process() member function for each file-change event:

// event loop:

while( runFam ){

  FAMNextEvent( fc , fe ) ;

  if( FAMChanged == fe->code ){
        target->process() ;
  }
}

(In a real app, of course, the event loop would run in a separate thread.)

target is a Handler object. The Handler class is a pure-virtual (or interface), which means it only provides member function declarations. The provided implementation thereof, EchoHandler, simply prepends Line: to each line that it reads from the watched file. Feel free to swap in your own implementation: inherit from Handler and write a process() member function.

Most of EchoHandler::process() is C++ magic to keep track of the current place in the file. I won't explain that here, as this article isn't about the C++ iostreams library. If you're curious, peruse the source file's comments.

That's a Wrap

FAM provides developers a means to track changes in files and directories. Its client API is fairly straightforward and clean, which makes it easy to fold into other applications. You could certainly write your own framework to watch files; but since FAM exists, you have one less reason to do this.

Resources

Q Ethan McCallum grew from curious child to curious adult, turning his passion for technology into a career.


Return to the Linux DevCenter.

Copyright © 2009 O'Reilly Media, Inc.