Published on (
 See this if you're having trouble printing code examples

Big Scary Daemons

Cleaning Up Ports


Despite all the praise the FreeBSD ports system gets, it has limitations. One of these limitations is actually related to one of FreeBSD's other strengths -- the upgrade system. The two interact in a very clumsy way.

The port installation process records every file a port includes and all of the port's dependencies. For example, many ports require the PNG graphics library. The port records the exact version of PNG it used, such as version 1.0.11. This is generally the same version found in the ports tree when you installed your first piece of software. The FreeBSD upgrade process can upgrade the version of the software supported by the ports tree. For example, PNG was recently upgraded to version 1.0.12. The library version number didn't increase, but the new library has minor bug fixes. I'd like to upgrade. If I do that, however, the 30-odd ports that have the exact PNG version hard-coded in them will have incorrect records.

My laptop has 187 packages. I upgrade my ports tree on a weekly basis. This gets messy very quickly, and impossible soon after. Fortunately, Akinori MUSHA has stepped up to the plate and developed a tool to untangle this mess: portupgrade(1) and friends.

The tools included in portupgrade implement some new features in the FreeBSD ports system. First is pkgdb and portdb, which build databases to index /var/db/pkg and the ports tree. This accelerates searching and manipulating this information. These tools also rewrite the various /var/db/pkg files to maintain consistency. Finally, there are wrappers for the various pkg_* commands that handle rewriting both the databases and the plain-text records when you add or remove software. You get all this by installing /usr/ports/sysutils/portupgrade.

Before you start using portupgrade, back up /var/db/pkg! I've never had any trouble with this program, but any tool that directly alters system records has boundless potential to make your life difficult. If you completely destroy your package records, you want to be able to restore them.

# tar -czvf dbpkg.tgz /var/db/pkg

Run pkgdb -F to do your initial setup and do some basic clean up. If you have a lot of ports, and have upgraded your ports tree several times, set aside a solid block of time to do this the first time. Once you have a consistent database, maintenance is quick and easy; it's only this first install that is time-consuming.

#pkgdb -F
Checking the origin of Hermes-1.3.2
Checking the origin of Mesa-3.4.1
Checking the origin of Mesa-3.4.2_1
Checking the origin of ORBit-0.5.10_5
Checking the origin of XFree86-aoutlibs-

When it comes across a package with changed dependencies, pkgdb will ask you if you want to correct them.

Stale dependency: esound-0.2.22 -> libaudiofile-0.2.2:
libaudiofile-0.2.1_1 ? ([y]es/[n]o/[a]ll) [yes]

So, esound has recorded libaudiofile version 0.2.2 as a dependency, but version 0.2.1_1 is installed. I probably did a pkg_add -f at some point in the past, thinking that it would work despite the minor difference in version numbers. (It did.) It's asking if I want to update my entry in /var/db/pkg/esound-0.2.22/+CONTENTS to point to the version of libaudiofile that's actually installed. If I take the default, it will correct the entry. If I type "a", it will not only correct the libaudiofile entry for esound but also for every other package that uses libaudiofile. This is very handy for packages that have many dependent packages, such as PNG.

In the previous example, pkgdb was able to make an intelligent guess about the dependency. Other times, it cannot guess at all. For example, I built XFree86 from's CVS repository. imake is part of XFree86, so it has no record in /var/db/pkg. This is OK; the port-building process checks for the existence of a program, not for its record in /var/db/pkg. When a port's installation record is placed in /var/db/pkg, however, it assumes that the port is there if it found the program. (This is arguably a shortcoming in the ports system, but there's no easy fix for this.)

What this means is that when a port lists imake as a dependency, pkgdb chokes.

Stale dependency: Xaw3d-1.5 -> imake-4.1.0:
New dependency? (? to help): ?

Here I have a choice. I can either assign a new dependency, or I can tell the package that this is no longer a dependency. While Xaw3d does require imake, I know that on this particular system it will never find an imake package installed. I want to delete the dependency. If you hit "?" for help, you'll see:

[Enter] to skip, [Ctrl]+[D] to delete, [.][Enter] to abort, [Tab] to complete
New dependency? (? to help):

Control-D it is.
Delete this? ([y]es/[n]o/[a]ll) [yes]

If I choose "yes," it will delete just this dependency. If I choose "all" it will delete this dependency whenever it appears. A lot of ports use imake; I want to delete them all. You'll see this show up later:

Stale dependency: xneko-4.4 -> XFree86-libraries-4.1.0:
Delete this? ([y]es/[n]o/[a]ll) [yes] Deleted.

Other times, there really is a dependency elsewhere that pkgdb can't guess at.

Stale dependency: plugger-3.3_1 -> timidity++-esound-2.10.4:
New dependency? (? to help): ?
 [Enter] to skip, [Ctrl]+[D] to delete, [.][Enter] to abort, [Tab] to complete
New dependency? (? to help): timidity++-2.10.4

The tab completion makes this doable. I know that timidity++-esound is using some version of timidity, but I don't want to go digging around to find out the whole version number. I can type in the first few letters and hit tab, and get the rest of the package name filled in.

Finally, at the end of the whole process pkgdb will warn you about duplicates.

Duplicated origin: graphics/Mesa3 - Mesa-3.4.1 Mesa-3.4.2_1
Remove any of them? [no]

Related Reading

UNIX Power ToolsUNIX Power Tools
By Jerry Peek, Tim O'Reilly & Mike Loukides
Table of Contents
Full Description

Here I have a problem. It's not uncommon when you install software from packages to have the package require another package. You might have a slightly different version of the package already installed. If you're installing via package, however, it automatically installs dependencies even if a slightly different version is already installed. At some point this happened to my system. My laptop has two different versions of Mesa, but one has overwritten the other. It obviously cannot be too bad, as my system still works. (Eventually, I should entirely remove Mesa and install a known good version.) Still, I need to clean up the package database. I arbitrarily removed the older version.

Now you have a database of package information, /var/db/pkg/pkgdb.db. You'll also have a ports database, /usr/ports/index.db and /usr/ports/index.dbo. The various portupgrade tools use these databases to work their magic. Whenever you upgrade your ports tree, upgrade the ports database and /usr/ports/INDEX. You can do this easily with portsdb -Uu.

Now that we have the tools installed, let's see what we can do. To see what you have that's obsolete, use portversion (1). This works in the same way as pkg_info but is far faster.

# portversion
Hermes                      =
Mesa                        =
ORBit                       <
XFree86-aoutlibs            <
Xaw3d                       =
aalib                       =

A < means that the installed version is older than what is available in the ports tree. You can easily run portversion | grep '<' to see which ports are obsolete. I do this every so often just to see what's installed on my system. For example, my Apache package is outdated. I'm no longer using Apache on my laptop, however, so I can remove it. The standard pkg_delete command still works, but if I use it my package database will fall out of synch with my system. Use pkg_deinstall(1) instead.

# pkg_deinstall apache
---< Deinstalling 'apache-1.3.20'
[Updating the pkgdb in /var/db/pkg ... - 182 packages found (-1 +0) (...) done]

The nice thing is, I no longer need to give the full version number! (That's bugged me about the FreeBSD pkg_* tools for quite some time, but never enough to do anything about it.) The portupgrade tools include globbing functions, which means that they can do pattern matching.

I want to upgrade certain other ports. What's more, I want to upgrade these while keeping the database synchronized, and updating the now-correct entries in /var/db/pkg. For example, pkgversion tells me that gd is out of date. It's simple enough to upgrade:

# portupgrade gd
===>  Cleaning for gettext-0.10.35
===>  Cleaning for gmake-3.79.1
===>  Cleaning for libtool-1.3.4_2
===>  Cleaning for jpeg-6b
===>  Cleaning for png-1.2.0
===>  Cleaning for freetype2-2.0.5
===>  Cleaning for gd-1.8.4_4
If you want to compile in X support use 
'make -DWITH_X11' instead
===>  Extracting for gd-1.8.4_4

You'll see the familiar make output for a while. If you keep watching, though, you'll see it make a detour to deinstall the existing package, update the database, and keep going.

... ---> Deinstalling 'gd-1.8.4_3'
pkg_delete: package 'gd-1.8.4_3' is required by these other packages and may not be deinstalled (but I'll delete it anyway):
[Updating the pkgdb in /var/db/pkg ... - 180 packages found (-1 +0) (...) done]
===> Installing for gd-1.8.4_4

Once the install finishes, it updates the package database again. Everything is synchronized.

Now, let's look at something difficult. My docproj port is out of date. The docproj port is a port that doesn't include anything in and of itself, but has dependencies on every tool needed to edit the FreeBSD Documentation Project. If you regularly build the docs tree, you need this tool to be up-to-date. Also, you don't want to recursively rebuild the whole thing just for a minor update; docproj has some huge dependencies.

If I wanted to save some time, I could use portupgrade -P to tell portupgrade to use packages for this. Time is something my computer has no shortage of, so I'll use ports. We need to tell portupgrade to recursively upgrade packages, however, so it will upgrade the packages required by docproj. You can do this with the -R flag. Let's give it a try.

# portupgrade -R docproj
This port will try to ensure that the tools used by the FreeBSD
Documentation Project are installed on your system so you can convert documentation from SGML to other formats.

One of these components is JadeTeX, which depends on TeTeX. The source for TeTeX is larger than 30 MB, and may be a very long download.

If you do not want to produce PostScript and PDF formats from the documentation, you do not need JadeTeX, and you should set the variable JADETEX to "no". If you do want to produce PostScript and PDF output then set JADETEX to "yes".

For example:

    make JADETEX=no
    make JADETEX=yes

*** Error code 1

** Command failed: make clean build
** Fix the problem and try again.
** The following packages were not installed or upgraded (-:skipped /
        ! textproc/docproj (docproj-1.4)        (unknown build error)

Ugh. This port requires customization. You can edit /usr/ports/textproc/docproj/Makefile to add JADETEX=no somewhere. It will now run transparently.

When you've finished, you can check how things stand with portversion. You might find that other port dependencies have changed -- for example, if port A depends on port B, but port C also depends on port B, running portupgrade A will not edit the dependencies of port C. Be prepared to run pkgdb -F on occasion to find and fix these problems.

If you want to have the system handle these things itself, you can use portupgrade -rR; this will upgrade both dependent and depending ports. If you always want the latest software on your system, this is the way to go. portupgrade includes a variety of other tools that simplify software management on FreeBSD, but this should be enough to get you started.

You might find that you have to rebuild a single port multiple times from the same distfile. If a port is revised, portversion will catch it and mark it as outdated. You'll want to keep the distfiles for the ports you have installed, but delete outdated distfiles. portsclean -D will do this. Similarly, you might want to remove shared libraries that are not referenced in the package database. I don't want to use that option -- remember, I installed XFree86 from scratch, not from ports. This function would remove my X setup, ouch! But portsclean -L would do this, and it would help deal with problems such as leftover files from my defective Mesa install.

Finally, I have to say that this tool seems fairly robust. As I was finishing this article, I ran these programs to check my examples. I noticed one obsolete package, and simply had to try to upgrade it.

# portupgrade -R portupgrade

Yep, it works. I'd say this tool is a definite winner.

Michael W. Lucas

Read more Big Scary Daemons columns.

Return to the BSD DevCenter.

Copyright © 2009 O'Reilly Media, Inc.