ONLamp.com    
 Published on ONLamp.com (http://www.onlamp.com/)
 See this if you're having trouble printing code examples


The lighttpd Web Server

by Bill Lubanovic
04/05/2007

Until recently, Apache didn't have a serious open source rival. In Netcraft's latest web server survey, we can see one emerging. As always, Apache has the top spot, Microsoft's IIS is second, and the ever-popular unknown is third. Fourth is Sun's Java Web Server (formerly known as ONE, formerly iPlanet, formerly Netscape). But at number five, serving about 1.4 million sites, is something called lighttpd. Where did that come from? We'll look into lighttpd's history, basic installation and configuration, and some visions of the future.

You can pronounce it lite-tee-pee-dee or, with less spitting, lighty. However you say/spray it, you'll find lots of information on its web site, wiki, blog, or forums. lighttpd was designed to be a high-performance web server with light resource use. It uses much less memory than Apache and is generally faster. lighttpd quietly powers many heavy-duty sites, including YouTube, Wikipedia, Meebo, and A List Apart; you'll often see it used in Apache's place with popular tools such as Ruby on Rails and Trac.

What's Wrong with Apache?

Despite its popularity, sometimes Apache isn't the best solution. Apache provides different Multi-Processing Models (MPMs) for use in different runtime environments. The prefork model--the most popular on Linux--creates a number of Apache processes at startup time and manages them in a pool. The alternative worker model uses multiple threads instead of processes. Although threads are lighter than processes, you can't use them unless your whole server is threadsafe. Although Apache and mod_php are threadsafe, this is not guaranteed for all the third-party modules they might use. The PHP site discourages use of Apache 2 with a threaded MPM; this may have slowed the movement of developers from Apache 1.3 to 2.0. But the prefork model has its own problems: each process (Apache + PHP + third-party modules) consumes a lot of memory (30MB is not unusual). When you multiply this by the number of simultaneous Apache processes, your available RAM can disappear fast.

lighttpd in the Mist

Some web sites serve thousands of files in parallel, and are limited by memory and the maximum number of threads or processes. Dan Kegel has detailed the issues encountered when handling thousands of simultaneous connections in his page on the C10K problem. In 2003, a German MySQL developer named Jan Kneschke became interested in this problem and believed he could write a faster web server than Apache by focusing on the right techniques. He designed lighttpd as a single process with a single thread and non-blocking I/O. Instead of select, he used the fastest event handler in the target system: poll, epoll, kqueue, or /dev/poll. He chose zero-copy system calls like sendfile rather than read and write. Within a few months, lighttpd was serving static files faster than Apache.

The next step was to handle dynamic (CGI) applications, particularly PHP. Kneschke dusted off FastCGI, invented by Open Market in the go-go early Internet days as a way to improve CGI performance. Rather than having the web server start the same external CGI program on each call, FastCGI was essentially a daemon to prestart the CGI application and handle communication between it and the web server. This was faster, but Perl and PHP were later absorbed into Apache as modules, which were even faster and gave them access to Apache's internal HTTP processing steps. FastCGI for Apache became neglected, but after it was added to lighttpd and hooked up with PHP, its performance met or exceeded that of Apache with mod_php. As an extra, the lighttpd implementation added automatic load balancing.

The lighttpd ecosystem has since expanded with modules to manage virtual hosts, redirection, URL rewriting, authentication, and other webby things. For most purposes, anything you can do in Apache you can do in lighttpd. In the next few sections, we'll show how to install and configure lighttpd, with sidelong glances at Apache.

Installing lighttpd

Let's install lighttpd and poke it with some metaphorical sticks. The wiki's installation page gives examples of binary or source installations for various Linux distributions. For hairy-chested developers (it doesn't have to be your own hair), a full source install goes like this:

    # wget http://www.lighttpd.net/download/lighttpd-1.4.13.tar.gz
    # tar xvzf lighttpd-1.4.13.tar.gz
    # cd lighttpd-1.4.13
        # ./configure
    # make
    # make install

This will install lighttpd under /usr/local. If the build failed, check that the prerequisite pcre and zlib development packages are installed on your system.

If you want to start and stop lighttpd manually, you're done. To install lighttpd as a service like Apache, edit and install the init script:

    # sed -e 's/FOO/lighttpd/g' doc/rc.lighttpd > lighttpd.init
    # chmod a+rx lighttpd.init
    # cp lighttpd.init /etc/init.d/lighttpd
    # cp -p doc/sysconfig.lighttpd /etc/sysconfig/lighttpd
    # install -Dp ./doc/lighttpd.conf /etc/lighttpd/lighttpd.conf
        # chkconfig lighttpd on

Basic Configuration

The syntax of lighttpd's configuration file may be the most visible difference from Apache. The wiki's configuration page examples look more like Perl (or PHP or Python) than Apache's XMLish httpd.conf. For a simple web site with static files, you need to specify the same things you would for Apache: the document root, the access and error logfile names, and the Linux user and group names for the server process. Here are Apache (httpd.conf) and lighttpd (lighttpd.conf) equivalents:

Apache:

DocumentRoot /var/www/html
CustomLog /var/www/logs/access
ErrorLog /var/www/logs/error
User www
Group www

lighttpd:

server.document-root = "/var/www/html"
accesslog.filename = "/var/www/logs/access"
server.errorlog = "/var/www/logs/error"
server.username = "www"
server.groupname = "www"
server.modules = ( "mod_accesslog" )

lighttpd has an include mechanism similar to Apache's, so the lighttpd.conf file doesn't have to grow. To use an optional module, you need to enable it and set its options. Apache enables with LoadModule, but lighttpd just includes the uncommented module name in the server.modules array. The only one you'd need so far is mod_accesslog.

Authentication and Authorization

lighttpd does not support .htaccess files, so you need to specify all settings in the lighttpd.conf file, or files that it includes. It understands Apache user files for basic and digest authentication, but group file support is not yet implemented. Here's how to password-protect a top-level directory called special:

Apache:

<Location /special>
  AuthName "My Special Directory"
  AuthType Basic
  AuthUserFile /var/www/passwords/users
  Order deny,allow
  require valid-user
</Location>

lighttpd:

auth.backend = "htpasswd"
auth.backend.htpasswd.userfile = "/var/www/passwords/users"
auth.require = ( "/special/" =>
  (
  "method"   => "basic",
  "realm"    => "My Special Directory",
  "require"  => "valid-user"
  )
)

Virtual Hosts

Here's another task for your hardworking and unappreciated web server: manage two sites called scratch.example.com and sniff.example.com:

Apache:

NameVirtualHost *
<VirtualHost *>
  ServerName "scratch.example.com"
  DocumentRoot "/var/www/hosts/scratch/docs"
<VirtualHost>
<VirtualHost *>
  ServerName "sniff.example.com"
  DocumentRoot "/var/www/hosts/sniff/docs"
<VirtualHost>

lighttpd:

$HTTP["host"] == "scratch.example.com" {
  server.document-root = "/var/www/hosts/scratch/docs/" }
$HTTP["host"] == "sniff.example.com" {
  server.document-root = "/var/www/hosts/sniff/docs/" }

That's doing it the hard way. If you manage many hosts, you'll save typing with a virtual host module:

Apache:

LoadModule vhost_alias_module modules/mod_vhost_alias.so
VirtualDocumentRoot /var/www/hosts/%1/docs

lighttpd:

server.modules = ( ..., "mod_evhost", ... )
evhost.path-pattern = "/var/www/hosts/%3/docs"

Server-Side Includes (SSI)

A baby step toward dynamic content, it's easy to enable SSI for files with the .shtml suffix:

Apache:

AddHandler server-parsed .shtml

lighttpd:

server.modules = ( ..., "mod_ssi", ... )
ssi.extension = ( "shtml" )

PHP

lighttpd optimizes static file throughput by offloading CPU-intensive dynamic content to another process. Rather than processing PHP internally, as Apache does with mod_php, lighttpd hands it off to FastCGI. These configuration snippets turn dull, lifeless .php files into vivacious PHP scripts. For racier details than we can show on this family-rated site, see this page.

Apache:

LoadModule php5_module modules/libphp5.so
AddType application/x-httpd-php .php

lighttpd:

server.modules = ( ..., "mod_fastcgi", ... )
fastcgi.server =
  ( ".php" =>
    ( "localhost" =>
      (
      "socket" => "/tmp/php-fastcgi.socket",
      "bin-path" => "/usr/local/bin/php"
      )
    )
  )

lighttpd Goodies

lighttpd includes Apache-equivalent modules for compression, directory listings, user directories, SSL, WebDAV, and URL rewriting and redirection. You can read about these on the web site. Other interesting modules are unique to lighttpd.

If you want to be a mini YouTube, you can stream thousands of Flash movies in parallel with the mod_flv_streaming module. YouTube serves its own static files with lighttpd, although it doesn't use this module yet.

If you had such a site with lots of Flash files, how would you protect against hotlinking? lighttpd's solution, applicable to any file type, is mod_secdownload. You write a function (examples in the link include PHP and Ruby) to generate a special URL, and the module cracks the URL to permit access to a given file for a given amount of time.

Lighty One Five Oh

Kneschke is now pushing toward version 1.5.0, which will have increased performance and flexibility. A new I/O subsystem improves file throughput with threading (threads make sense here) and async I/O--either POSIX, Linux native, or the userland gthread wrapper in glib 2.0.

The mod_proxy_core unifies three backend modules: mod-proxy, mod-fastcgi, and mod-scgi. By separating the protocols from the actual processing, this adds load balancing (four kinds!), fail-over, keep-alive, and internal queuing to the basic proxy function.

A recent addition called mod_magnet is expected to play a large role in lighttpd's future. It gives access to the different phases of HTTP request and response, including tasks like URL rewriting and content generation. An interesting choice is its use of the embedded scripting language Lua rather than a complex grammar like Apache's mod_rewrite. We'll see if developers like this or will stick with Apache's familiar, though sometimes difficult, rewrite rules.

Whither Lighty?

Kneschke expects the future of lighttpd to follow two use cases:

After 1.5.0, mod_magnet will offer more dynamic server configuration. This could bring in some Apache developers who had rejected porting to lighttpd for lack of .htaccess support. I'm looking forward to planned support for Comet--sort of a reverse Ajax, in which the server updates the client when it has new data. This has applications for web dashboards, chats, and other highly interactive applications. With Ajax and Comet, web applications can look and feel more like traditional GUI applications. But even without these new features, lighttpd is already a strong competitor for Apache--especially where memory is limited or the workload consists of many static files. Light is good.

Bill Lubanovic started developing software with UNIX in the 70s, GUIs in the 80s, and the Web in the 90s. He now does web visualization work for a wind energy company.


Return to ONLamp.com.

Copyright © 2009 O'Reilly Media, Inc.