BSD DevCenter
oreilly.comSafari Books Online.Conferences.


Big Scary Daemons

BSD Ports Collection Basics


In the early days of Unix, installing a program from source code was a hit-or-miss proposition at best. The now-pervasive "configure" hadn't appeared yet. Every systems administrator had to understand the platform a program had been written for, and its differences from their platform, before they had a hope of porting a chunk of code. The duplication of effort was truly monstrous. The FreeBSD ports system started addressing this back in 1995; the system has scaled well, and is still very popular among the BSD community.

In FreeBSD parlance, a "port" is simply a set of instructions and patches for compiling a piece of software. Ports have spread from FreeBSD to NetBSD and OpenBSD, where they've been changed to fit those environments. (Since NetBSD uses "ports" to mean a hardware platform, they call them "packages.") The basic concepts remain the same, however.

Most FreeBSD users understand the basics of using a port: You go to the port's directory, type make install, and the computer magically pulls the source code out of the ether, confirms it hasn't been tampered with, uncompresses, patches, builds, and installs. The overwhelming majority of the time, this system works perfectly.

For most users, the most difficult problem is finding the desired port. Right now FreeBSD has over four thousand ports, scattered over forty-odd categories. I highly recommend pib (/usr/ports/sysutils/pib), the Ports Index Browser. It's a simple tk-based tool that reads /usr/ports/INDEX and presents a searchable, browsable interface.

Previously in Big Scary Daemons:

Running Commercial Linux Software on FreeBSD

Building Detailed Network Reports with Netflow

Visualizing Network Traffic with Netflow and FlowScan

Monitoring Network Traffic with Netflow

Information Security with Colin Percival

If you think tk is too heavy, or just want to quickly find a single port, you can find most of them with grep. When you know the port's name, you can use:

grep -i ^portname /usr/ports/INDEX

The ^ tells grep that the word appears at the beginning of the line, and considerably trims your search. A grep -i ^gtk INDEX gives 29 results, where not using the ^ gives 387.

Some ports don't work seamlessly. Either FreeBSD has changed, or the original source code has changed, or the program has features the port doesn't cover. When that happens, you can contact the port maintainer or search the freebsd-questions archive to see if anyone else has had the problem. Worst case, you need to break out your programming skills and try to diagnose the problem.

In any case, it's a good idea to understand the basics of the ports system. You need to solve the problem anyway, and if you can send patches back to the FreeBSD project, you'll help a lot of people as well as yourself.

If you want to learn about the ports system, you have four major sources of information.

First, the FreeBSD handbook contains a step-by-step tutorial on how to use the ports system. It covers just what's needed to get started.

For more in-depth information than the handbook, check ports (7). This describes the basic environment variables and make targets that control port-building behavior.

The porter's handbook describes how to create a port for FreeBSD.

Finally, the big enchilada: Any of the prior three might be out of date, depending on how overloaded the documentation team is that week. The actual source code that controls the entire process is /usr/ports/Mk/ If you find something in that doesn't match the documentation, the documents are incorrect. (Patches can be sent via send-pr.) In a way, is one of the easiest pieces of FreeBSD code to read and understand; it's far simpler than, say, the IP stack or the FTP client. Remember, there's a difference between "easy" and "easiest part of the system."

One term you'll find scattered throughout the ports system is "distfile." A distfile is simply a file of source code. They're usually combined with tar, and then compressed with gzip, zip, compress, or bzip.

A port itself is fairly simple. The ports tree layout has recently changed, however, so we'll cover it briefly.

Every port contains a Makefile. This isn't your usual source code Makefile; instead, the port Makefile tells FreeBSD how to build the source code it downloads. It includes such information as which version of make to use, which compiler it needs, and dependencies.

The file distinfo contains distfile checksums. When the port was last updated, the port maintainer confirmed the checksum of a known good distfile and recorded it here.

The file pkg-comment contains a one-line description of the port.

A longer description of the port, usually including a URL for further information, is found in pkg-descr.

The pkg_plist file has a list of all files the port installs.

The optional files directory contains any patches needed to make the code compile on FreeBSD.

Lastly, the optional scripts directory contains scripts that can be run at various parts of the installation. Scripts can be used for any pre- or post-processing that the port needs, for example, to change permissions on a downloaded distfile so that patch(1) can run properly.

Here's an overview of what happens when you go to a port's directory and type make install.

The process starts with make fetch. If the source code isn't in /usr/ports/distfiles, your system goes to the MASTER_SITE specified in the Makefile and gets it.

Then, make checksum confirms that the distfile's hashed signature matches the one that the port has on file. This is meant as a security measure; if the source code contains a Trojan horse loaded by an attacker, this process will detect it and stop the build with a warning about a checksum mismatch.

Of course, make checksum also stops compilation if the distfile has been deliberately changed. Software authors might make a minor change in their code, but give the source file the same name when they make it available for download. The FreeBSD port might or might not work after this change. If you're sure that the distfile has not been compromised and want to try it anyway, you can override this with make NO_CHECKSUM=YES.

The make depends stage checks to see if the port requires any other software to install. For example, an X window manager requires an X server. Recursing through the various dependencies, make depends builds them as needed.

To create a work directory and uncompress the distfile, use make extract.

A port contains patches, if necessary, to allow the software to build under FreeBSD; make patch applies them.

Any configure script in the source code is run by make configure.

The make build stage compiles the checked, extracted, and patched software.

Finally, make install installs the software and records its presence under /var/db/pkg.

Whenever you use a make target, all previous stages are run. For example, a make extract runs through make fetch, make checksum, and make extract.

Now that you know what normally happens when working with ports, next time we'll look at some ways to fine-tune a port.

Michael W. Lucas

Read more Big Scary Daemons columns.

Discuss this article in the Operating Systems Forum.

Return to the BSD DevCenter.

Sponsored by: