ONLamp.com    
 Published on ONLamp.com (http://www.onlamp.com/)
 See this if you're having trouble printing code examples


Company-Wide Instant Messaging with Jabberd

by Oktay Altunergil
10/06/2005

Instant messaging has quickly become part of everyday life in the last decade, with Mirabilis's ICQ leading the way and a host of others following. However, the idea of being notified of the presence of other people when they are online and being able to communicate with them instantly dates further back. Even before ICQ came out, some versions of Unix included simple tools to poll a remote server periodically to see whether a particular user was logged in and notify the user via a small GUI application of their presence. From then on, a host of applications including talk were available for this type of communication. This worked considerably well when the internet-using population comprised university students and people who worked at government organizations and corporations. Because almost everyone logged in to the network using some kind of a host machine with a real shell on the system, people could communicate with their friends and colleagues using only the simple tools Unix always provided.

With the internet becoming available to the masses from the comfort of their homes (albeit with no shell account to speak of) and becoming a much more hostile environment, causing server administrators to deny other hosts to poll their servers inquiring about the users logged in, the old method of instant communication became impractical. The need arose to provide an easier and more versatile way to communicate, and various companies came out with their own implementation of the instant-messaging solution, often with proprietary protocols and server software doing the work.

Today's instant-messaging landscape looks much different from ten years ago. Major players in the field are multimillion-dollar corporations that see instant messaging as just another business opportunity--another way to deliver their own messages to millions. For better or worse, the efforts of the corporations have paid off and instant messaging has become very commonplace, with people starting to exchange instant-messaging IDs instead of phone numbers or even email addresses. As with many other areas of technology built around proprietary offerings, someone sooner or later would offer an open standard. In instant messaging, the leader here is Jabber.

Related Reading

Linux Server Hacks
100 Industrial-Strength Tips and Tools
By Rob Flickenger

Jabber is an open, XML-based protocol for instant messaging and presence. One of the more visible applications of this protocol is the creation of an instant-messaging network, though the protocol itself allows the development of and use with many other applications.

Apart from the philosophical benefits of Jabber, one great advantage is the absolute openness of both the Jabber protocol itself and a variety of open source software projects (some also conform to the FSF's description of free software) that implement the protocol. Without this, few companies would have the resources to accomplish the solutions described in this article.

Scenario

Consider a hypothetical midsize company with multiple branch offices in multiple locations. The company employs between 200 and 400 employees, and the security of internal communications is of extreme importance. The employees use a mixture of Windows and various flavors of Linux and BSD on their workstations, and expect to be able to use the messaging system when home or on the road. Another crucial requirement is that the system authenticate against the existing Microsoft Active Directory domain controller. This will allow the central management of employees' access to the messaging system.

The instant-messaging needs of this hypothetical corporation dictate:

Although they're not part of the requirements, I'll also throw in a few freebies just for good karma. These should go a long way toward ensuring a successful deployment of the system and acceptance by users, at least in the introduction stage.

As a side note, this solution also intentionally leaves out some features that your organization might desire. Logging is one, file transfers is another. At the time of this article, there is no single preferred method for securely exchanging files with low overhead and universal client support. Various Jabber Enhancement Proposals (JEPs) address that; these will be part of the Jabber protocol in the future.

Alternatives to Using Jabber

There are many alternatives to using Jabber as an instant-messaging solution.

Publicly available instant-messaging networks

One option is to use a publicly available instant-messaging network. Those include MSN Messenger, Yahoo Messenger, AOL Instant Messenger, and ICQ. Although arguments against using these public networks in a corporate environment should be pretty self-evident, in practice many corporations either embrace the use of a particular public instant-messaging network or do not have policies against using these services, and employees end up using their existing user accounts on these networks for work purposes.

Some of the more obvious drawbacks to using a public network in a corporate setting are;

Hosted, subscription-based solutions

Many players in the public instant-messaging space also provide enterprise versions of their software, in which they host a corporation's instant-messaging domain on the provider's servers. These usually include additional features for security and automation. While this is better than using a public network, there are still some drawbacks:

Nonhosted commercial solutions

Another instant-messaging provider business model is to allow a corporation to install the provider's software on the corporation's own servers, giving the organization complete control over security and availability. If your corporation's needs exactly match the provider's service capabilities, this might be a good solution for you. There are a few different types of companies in this space. Some have their own proprietary protocols and server and client software. Others use existing protocols to implement their own software solution. There are even a few providers with products based on the open Jabber protocol. However, as with any other software provided by a third party, there are some limitations to this type of solution:

Required Software and Tools

My preference is running a Jabber server. How does that work?

Jabber server software--jabberd2

The first and most important decision to make is which Jabber server software to use. My examples assume that you are using the open source jabberd2 implementation. jabberd2 is a complete rewrite of the original Jabber server software, and it is likely to be the most complete reference implementation of the Jabber protocols in the long run. At any rate, it already supports all the features listed in the requirements section. Other open source and proprietary Jabber server implementations are available. Some of these--especially ejabberd--show promise, so at least evaluate the open source versions before deciding on your own server software to use.

In order to support all the desirable functions, you must compile jabberd2 to include some optional components. The requirements outlined earlier dictate support for a database back end for roster information (in this example, MySQL), LDAP to authenticate via Active Directory, and finally SSL to ensure the secure transfer of all messages between clients. There are a lot more options you can add while compiling jabberd2, and it really doesn't hurt to include a few extra components you think you might need in the future. However, a configure statement such as this should suffice:

./configure --prefix=/opt/jabberd2 --enable-ldap --enable-ssl \
    --enable-mysql

This article is not as a step-by-step walk-through, so you may have to add more to the above configure line to have the software compile cleanly on your system, especially if your library and header files are in locations other than what the configure script considers standard.

Notice the choice of /opt/jabberd2 as the prefix for the installation. This single, central location makes it easier to configure and upgrade the software after the installation. For the rest of this article, I will assume that you have installed the software here. Please refer to the jabberd2 documentation in order to complete the installation.

Server hardware

As with any server hardware, factors such as the server software and the number of simultaneous users influence the necessary specifications. jabberd2 is not very imposing in this respect and will scale well without a problem up to a couple of thousand registered users. If you opt for other Jabber server implementations, especially a Java-based version, do your homework before deciding what kind of hardware to use. For the requirements of the hypothetical company, a Pentium III system with 800-plus MHz and 512MB of system memory will suffice. In reality, you can probably get away with an even less powerful server, but this configuration is a safe minimum based on my own experience. You are also welcome to use an existing server that provides other services, although for security reasons I recommend using a dedicated server.

Other server-side software

Based on the requirements, the IM server also needs a database back end. jabberd2 supports many different database systems. I've chosen to use MySQL to store user details, roster information, and messages. This requires no advanced database features; a recent version of MySQL should work fine as long as you have the libraries and drivers for that version installed.

OpenLDAP and OpenSSL provide LDAP and SSL, respectively. Install those separately only if the configure statement during the install does not go through. If configure completes without an error, you probably have both of them installed on your system already, as is common with most Linux and *BSD variants.

Aside from the usual software packages listed above, there a few more necessary packages. Make sure that you have Perl installed along with the Perl modules Net::LDAP, Net::LDAP::Control, and DBI, with DBD::MySQL as the driver.

The final package to install is djb's daemontools suite. If you aren't familiar with daemontools don't worry. I will discuss it shortly.

Client-side software

Fortunately, a huge variety of Jabber client software is already available from multiple vendors. All of them will work with any Jabber server software (with varying levels of support) thanks to the open and standard nature of the Jabber protocol. There is at least one usable client for every desktop operating systems. The client software for the hypothetical company should at a minimum support for SSL connections. Windows users may look into Exodus, a fairly complete implementation with extensive administrative features built in, while Unix users can't go wrong with anything from among Gossip, Gaim, Gabber, Gajim, and Psi. (Some of these Unix clients are also available on Windows nowadays.)

Postinstall Configuration

jabberd2 comes with extensive documentation, which should make getting a basic server up and running a fairly pain-free task. Because such documentation and other sources of quality installation how-tos exist, I will concentrate only on the differences from a plain Jabber server install.

MySQL storage back-end configuration

The storage mechanism for jabberd2 stores the user account information, queued messages, and roster lists. It can also store usernames and passwords to provide authentication, but the plans for this configuration are different. Configure the storage mechanism of MySQL as described in the jabberd2 documentation. The schema for the database is part of the jabberd2 distribution package and includes the definitions for the tables active, authreg, disco-items, logout, motd-message, motd-times, privacy-default, privacy-items, private, queue, roster-groups, roster-items, vacation-settings, and vcard.

Although LDAP already contains part of the necessary user information, including the users' full name and department, using LDAP as the storage mechanism is not very straightforward. The first problem is the LDAP schema. To function, Jabber needs some extra object parameters mapped on Active Directory. You could create a new schema, perhaps inheriting as much of the usable data as possible from Active Directory, or extending the existing Active Directory schema to be able to use LDAP for storage as well as authentication. However, not only is it much easier to use MySQL for the storage, but LDAP also is actually unsuited for operations that require fast and concurrent additions of data into the system. Its design primarily promotes central and infrequent modification of data and very fast read-only access, which makes it perfect for tasks such as centralized authentication.

Configuration for authentication via Active Directory

Authentication in this system will come from the Active Directory server using the LDAP authentication mechanism of jabberd2. That way, users will be able to keep using their Active Directory domain username and passwords to authenticate, while users who are no longer with the company will lose their access as soon as someone disables their account in Active Directory.

In order to set up jabberd2 for LDAP authentication, you must fill out the following section of the c2s.xml configuration file. If you installed jabberd2 in /opt/jabberd2, the configuration files will be in the /opt/jabberd2/etc/jabberd directory.

<!-- LDAP module configuration -->
<ldap>
  <!-- LDAP server host and port (default: 389) -->
  <host>YOUR.ACTIVE.DIRECTORY.SERVER.ADDRESS</host>
  <port>389</port>
  <binddn>YOUR LDAP DN</binddn>
  <bindpw>YOUR LDAP PASSWORD</bindpw>
  <!-- LDAP attribute that holds the user ID (default: uid) -->
  <uidattr>sAMAccountName</uidattr>

  <!-- base DN of the tree. You should specify a DN for each
       authentication realm declared in the <local/> section above,
       by using the realm attribute. -->
  <basedn>OU=Company Name,DC=companyname,DC=com</basedn>
</ldap>

Here is a quick rundown of the variables you need to set in the XML file.

If you aren't sure what values to put in the LDAP configuration, ask your Active Directory administrator.

Also make sure that the <authreg> section of the same XML file has <module>ldap</module> configured and that all other module selections are commented out.

Creating and updating user accounts and populating rosters

Before users can log in and start using the system, their user account must be available in the system and they should have other users listed in their roster. Here's an external script that writes information directly to the database tables that jabberd2 uses for this purpose.

Although Active Directory comes with a default schema to take care of a corporation's directory needs, each corporation uses it differently. If your business rules require a different schema, modify the Perl script provided here to match your setup.

Sometimes directories include role accounts that are not those of actual employees. In the example environment, these entities do not have their mail field set in their active directory entry. The first assumption is that all real users who need a Jabber account have a mail entry.

The second assumption regards the levels in the organizational structure hierarchy. Most companies have levels within levels of department groups. For instance, a technical support group may have two groups for two separate office locations within it. In order to make the script simpler, I assumed that no subdepartments exist in the hierarchy.

Please keep in mind that this script is a tool to aid you in writing your own script. Although it worked in my environment, providing support for the script itself is outside the scope of this article. You can also download this script directly.

#!/usr/bin/perl -w
#   Oktay Altunergil - http://altunergil.org  Aug, 2nd 2005
#
#   This script is released in the hopes that it will be useful. Please make
#   sure you modify this script for your own purposes before running it. 
#
#
#   This script searches the windows domain for valid users and creates jabber
#   vcard/roster entries for them. It also adds each user to each other's
#   roster and bypasses the  invitation to add user steps.
#
#   assumptions:    - every legitimate user has a 'mail' entry
#           - the directory tree looks like:
#                       Root
#               ----------- 
#       Dept1:                  Dept2:          
#       ---------               ---------
#       Employee1               Employee2
#       Employee3               Employee4
#
use strict;
use Net::LDAP;
use Net::LDAP::Control;
use DBI;

my ($debug, $sth, $entry, $dept, $s, $query, $rosteritem, $rostergroup,
    @roster, $inneritem, $res);

my $basedn     = "YOUR BASE DN HERE";
my $ldapserver = "YOUR LDAP SERVER HERE";
my $disname    = "YOUR DN HERE";
my $ldappass   = "PASSWORD FOR THE ABOVE DN HERE";

my $dbuser     = "YOUR MYSQL USERNAME";
my $dbpass     = "YOUR MYSQL PASSWORD";
my $dbname     = "MYSQL DATABASE NAME";

my $dbh  = DBI->connect("DBI:mysql:database=$dbname;host=localhost",
                        $dbuser, $dbpass); 

my $ldap = Net::LDAP->new( $ldapserver, debug => 0 ) or die "$@";
my $mesg = $ldap->bind ($disname, password => $ldappass, version => 3 )
    or die "Cannot bind to ldap server. $!";

die $mesg->error if $mesg->code;

# query the department names
my $search = $ldap->search(
             base   => "$basedn",
             scope  => "one",
             filter => "(&(objectClass=organizationalUnit) )" );

# fetch the department names
foreach $entry ($search->entries) {
   
    $dept = $entry->get_value('name');
    print "\n\n Dept: $dept \n\n";

    fetchperson("ou=$dept,$basedn", "one", "$dept");

    # query the first department level names
    $s = $ldap->search(
        base   => "ou=$dept,$basedn",
        scope  => "one",
        filter => "( & (objectClass=organizationalUnit) )" );
    die $s->error if $s->code;
}

$ldap->unbind;

# delete everything in roster-items and roster-groups
# start with a clean page
$query = "DELETE FROM `roster-items`";
$dbh->do("$query") or die ("Cannot delete roster-items!\n");
$query = "DELETE FROM `roster-groups`";
$dbh->do("$query") or die ("Cannot delete roster-groups!\n");
    
# now that the @roster array is ready, populate the rosters
foreach $rosteritem (@roster)
{
    print STDERR "\n\n$rosteritem->{jid} is in $rosteritem->{role}" if $debug;
    foreach $inneritem (@roster) {
    if($debug) { print STDERR "\n|_____$inneritem->{jid}"
        unless $rosteritem->{'jid'} eq $inneritem->{'jid'} ; }
    
    
    # the following two statements add the roster-groups (Dept Names or Buddy
    # Groups etc) and also take care of adding each user to every users'
    # rosters without the need to authorize them manually
    $query = "INSERT INTO `roster-items` SET
            `collection-owner` = " . $dbh->quote($rosteritem->{'jid'}) . ",
            jid    = " . $dbh->quote($inneritem->{'jid'}) . ",
            name   = " . $dbh->quote($inneritem->{'name'}) . " ,
            `to`   = 1,
            `from` = 1,
            ask    = 0 ";
    $sth = $dbh->prepare($query);
    $sth->execute or die("Cannot execute query! ($!)");

    $query = "INSERT INTO `roster-groups` SET
            `collection-owner` = " . $dbh->quote($rosteritem->{'jid'}) . ",
            jid = " . $dbh->quote($inneritem->{'jid'}) . ",
            `group` = " . $dbh->quote($inneritem->{'rostergroup'}) ;
    $sth = $dbh->prepare($query);
    $sth->execute or warn("Cannot add roster group! ($!)");
    }
}


#### SUBS ####

sub updatevcard {
    $res = shift;
    $rostergroup = shift;

    my $collection_owner = $res->get_value('sAMAccountName') . '@' .
        'jabber.yourdomain.com';
    my $email            = $res->get_value('mail');
    my $org_orgname      = $res->get_value('l');
    my $org_orgunit      = $res->get_value('department');
    my $tel              = $res->get_value('telephoneNumber');
    my $title            = $res->get_value('title');
    my $role             = $res->get_value('description');
    my $nickname         = $res->get_value('givenName');
    my $adr_locality     = $res->get_value('physicalDeliveryOfficeName');
    my $adr_region       = $res->get_value('st');
    my $fn               = $res->get_value('displayName');

    $query               = "REPLACE INTO vcard SET 
            `collection-owner` = " . $dbh->quote($collection_owner) . ",
            fn                 = " . $dbh->quote($fn) . ",
            nickname           =  " . $dbh->quote($nickname) . ",
            email              = " . $dbh->quote($email) . ",
            title              = " . $dbh->quote($title) . ",
            role               = " . $dbh->quote($role) . ", 
            `adr-locality`     = " . $dbh->quote($adr_locality) . ",
            `adr-region`       = " . $dbh->quote($adr_region) . ",
            `org-orgname`      = " . $dbh->quote($org_orgname) . ",
            `org-orgunit`      = " . $dbh->quote($org_orgunit) . ",
            `desc`             = " . $dbh->quote($rostergroup) ; 

    $sth = $dbh->prepare($query) or die("Cannot prepare database query! ($!)"); 
    $sth->execute or die("Cannot execute database query! ($!) \n $query");
    $sth->finish;
    print STDERR $collection_owner . "\n" if $debug;

    push @roster , {"jid" => $collection_owner, "name" => $fn,
                    "rostergroup" => $rostergroup}; 
}

sub fetchperson {
    my $searchdn    = shift;
    my $scope       = shift;
    my $rostergroup = shift;
    my ($pers, $person);

    # query the user names
    my $us = $ldap->search(
        base   => "$searchdn",
        scope  => "$scope",
        filter => "( & (objectClass=person) (mail=*) )" );
    die $us->error if $us->code;

    # fetch the usernames
    foreach $pers ($us->entries) {
        $person = $pers->get_value('cn');
        print "\t\t|___" . $person . "\n";
        updatevcard($pers,$rostergroup);
    }
}

As you can see, the script completely removes all roster data before updating it with the latest information from LDAP. In some cases, such as when you want to be able to allow your users to make persistent changes to their vCards themselves, this might not be desirable. In that case, you would have to modify the script to make it smarter.

Encrypting traffic

To make your communication secure, configure your Jabber server to encrypt all traffic between the server and the clients using SSL. Other options for encryption use SASL/TSL to authenticate users securely and encrypt the traffic; however, having a dedicated port for SSL is easier to configure and is supported by most clients. Before using SSL, you must create a certificate file. Because the goal is to use Jabber internally, a self-signed certificate will be sufficient. Perform the following steps on the shell while logged in as root to create the certificate file:

# openssl req -new -x509 -newkey rsa:1024 -days 3650 -keyout privkey.pem \
    -out server.pem
# openssl rsa -in privkey.pem -out privkey.pem
# cat privkey.pem >> server.pem
# rm privkey.pem
# mv server.pem /opt/jabberd2/etc/jabberd/server.pem
# chown root:jabberd /opt/jabberd2/etc/jabberd/server.pem
# chmod 640 /opt/jabberd2/etc/jabberd/server.pem

The relevant configuration file for SSL encryption is c2s.xml, specifically the <local> section. Depending on whether you would like to enable SASL/TLS, this configuration will look different. In either case, make sure to disallow unencrypted communication.

Regardless of the encryption method used, provide the location of the newly created SSL certificate file:

<pemfile>/opt/jabberd2/etc/jabberd/server.pem</pemfile>
Encryption with SSL

First, disable the default Jabber port (5222) to ensure that your clients will not accidentally connect using an unencrypted stream. To do this, set the port parameter to 0:

<port>0</port>

Also make sure to enable the port dedicated to SSL connections, usually 5223:

<ssl-port>5223</ssl-port>
Encryption with SASL/TLS

This method uses the default Jabber port to start an encrypted channel, so it does not need an extra dedicated port. However, it's important to ensure that this port does not also allow unencrypted connections. To accomplish this, force jabberd2 to always use SASL/TLS by adding the following single statement to the <local> section of c2s.xml:

<require-starttls/>

Unlike with the dedicated SSL port method above, be sure to enable port 5222 when using SASL/TLS, because all communication flows through this port.

<port>5222</port>

If you would like to support older clients that do not implement SASL/TLS, it might still be a good idea to enable the SSL port even when SASL/TLS is enabled on the server.

Please note that jabberd2 now allows the encryption of communication between its server components as well. It's a good idea to configure such encryption to ensure better security.

PGP/GPG-based peer-to-peer encryption

Some client implementations also support PGP/GPG-based peer-to-peer encryption of messages. Although this is nice to have, it is insufficient security for a corporate environment. PGP/GPG-based encryption of traffic requires a lengthy configuration process on each client and is difficult to enforce. It also involves the exchange of public keys between each party that will use this encryption method. However, employees may still use such peer-to-peer encryption as an additional layer of security if they wish.

Security considerations

Like all servers, Jabber servers also need to be secured and tightened before going online. A few simple steps can make your server more secure. First of all, disable all unnecessary services on the server. If you're using a default install of Linux or FreeBSD, services such as HTTP, FTP, SSH, and DNS might start up automatically. If you can dedicate Jabber to its own server, it makes sense to disable these services. Furthermore, because this is a server in a corporate environment, you can use a firewall to deny access to any port on the server other than the Jabber port from all IPs outside the corporate WAN. More security-conscious companies may also block the Jabber port to the outside world if they would like their employees to use the instant-messaging system only while they are in the office. However, a Jabber server that is accessible to authorized users from anywhere in the world is potentially more useful.

Because running services on the default ports for Jabber (5222 and 5223) does not require root privileges, jabberd2 can easily run as a user with very limited privileges. Therefore it's beneficial to create a separate user for this purpose and make jabberd2 run as this user. I like to use the jabberd user and group.

Authenticating users via Active Directory uses the username and password info passed to the Jabber server from the client, so there's no need to use a separate LDAP account for authentication. However, the script needs to use an LDAP account to connect to Active Directory and retrieve the user and roster entries. Even then, it only needs to be able to read data from Active Directory, not write to it, so a read-only user account is sufficient.

Because this solution is open only to the employees, you can disable registrations altogether. Remember, the script will import all valid users, and they will authenticate on the domain server, so there is no concept of a conventional registration process in the workflow. By default, jabberd2 allows users to register themselves. To disable this feature, find the <register> section in c2s.xml and comment out the <enable/> parameter.

By default, jabber servers have the ability to communicate with each other, allowing a username on a server such as jabber.org to communicate with a user on jabber.turk-php.com without needing a username on both servers. This feature is great for public servers, but it introduces a foreign environment to a system running in a corporate setting. Protecting against any issues regarding server-to-server communication is as easy as not starting the sm component. In such a setup, the resolver component is also not necessary, but it is not a security problem to have it running.

High availability

Like all software, jabberd2 too sometimes crashes and fails. Although each new release finds and fixes bugs that cause crashes, it's still important to make sure that the server never goes offline for a long period of time. You can ensure this using the daemontools suite of tools, which monitors daemons and restarts them in the event that they die.

daemontools has a pretty straightforward installation procedure. If it's not already installed on your server, either find a package for your operating system or refer to the installation documentation and install the software.

Once installed, daemontools requires a startup script for each component that it will monitor in order to use when restarting that component. Listed below is the script for the c2s component as a reference. Paste this into a text file and save it as run. This script goes into the /service/c2s directory on your server, where /service is the default location daemontools looks for services it needs to monitor.

#!/bin/sh
exec setuidgid jabberd /opt/jabberd2/bin/c2s -D \
    -c /opt/jabberd2/etc/jabberd/c2s.xml 2>&1

After you create a directory and a script for each component (c2s, router, and sm), daemontools will take care of restarting these components should they die.

jabberd2 currently does not provide clustering features. ejabberd, based on erlang and its distributed database mnesia, is capable of clustering out of the box and is worth considering if high availability and load balancing are priorities.

Conclusion

Instant messaging in the enterprise can be a very valuable and efficient tool when used properly, but its deployment needs good planning and execution. Little convenience features such as populating rosters automatically go a long way toward improving the system's acceptance.

Although I have tried to cover most of the instant-messaging needs and requirements of a typical corporation, your business will probably have different requirements that will require further research and planning. Still, this article should be helpful in deciding whether a solution based on the open Jabber protocol might suit your needs.

Happy hacking!

References and links

Oktay Altunergil works for a national web hosting company as a developer concentrating on web applications on the Unix platform.


Return to ONLamp.com.

Copyright © 2009 O'Reilly Media, Inc.