Despite all the praise the FreeBSD ports system gets, it has limitations. One of these limitations is actually related to one of FreeBSD's other strengths -- the upgrade system. The two interact in a very clumsy way.
The port installation process records every file a port includes and all of the port's dependencies. For example, many ports require the PNG graphics library. The port records the exact version of PNG it used, such as version 1.0.11. This is generally the same version found in the ports tree when you installed your first piece of software. The FreeBSD upgrade process can upgrade the version of the software supported by the ports tree. For example, PNG was recently upgraded to version 1.0.12. The library version number didn't increase, but the new library has minor bug fixes. I'd like to upgrade. If I do that, however, the 30-odd ports that have the exact PNG version hard-coded in them will have incorrect records.
My laptop has 187 packages. I upgrade my ports tree on a weekly
basis. This gets messy very quickly, and impossible soon after.
Fortunately, Akinori MUSHA has stepped up to the plate and developed a
tool to untangle this mess:
portupgrade(1) and friends.
The tools included in
portupgrade implement some new features in the
FreeBSD ports system. First is
portdb, which build
databases to index
/var/db/pkg and the ports tree. This accelerates
searching and manipulating this information. These tools also rewrite
/var/db/pkg files to maintain consistency. Finally, there
are wrappers for the various
pkg_* commands that handle rewriting both
the databases and the plain-text records when you add or remove
software. You get all this by installing
Before you start using
portupgrade, back up
/var/db/pkg! I've never
had any trouble with this program, but any tool that directly alters
system records has boundless potential to make your life difficult.
If you completely destroy your package records, you want to be able to
# tar -czvf dbpkg.tgz /var/db/pkg
pkgdb -F to do your initial setup and do some basic clean up. If
you have a lot of ports, and have upgraded your ports tree several
times, set aside a solid block of time to do this the first time.
Once you have a consistent database, maintenance is quick and easy;
it's only this first install that is time-consuming.
#pkgdb -F Checking the origin of Hermes-1.3.2 Checking the origin of Mesa-3.4.1 Checking the origin of Mesa-3.4.2_1 Checking the origin of ORBit-0.5.10_5 Checking the origin of XFree86-aoutlibs-22.214.171.124 ...
When it comes across a package with changed dependencies,
ask you if you want to correct them.
Stale dependency: esound-0.2.22 -> libaudiofile-0.2.2: libaudiofile-0.2.1_1 ? ([y]es/[n]o/[a]ll) [yes]
esound has recorded
libaudiofile version 0.2.2 as a dependency,
but version 0.2.1_1 is installed. I probably did a
pkg_add -f at some
point in the past, thinking that it would work despite the minor
difference in version numbers. (It did.) It's asking if I want to
update my entry in
/var/db/pkg/esound-0.2.22/+CONTENTS to point to the
libaudiofile that's actually installed. If I take the
default, it will correct the entry. If I type "a", it will not
only correct the
libaudiofile entry for
esound but also for every other
package that uses
libaudiofile. This is very handy for packages that
have many dependent packages, such as PNG.
In the previous example,
pkgdb was able to make an intelligent guess
about the dependency. Other times, it cannot guess at all. For
example, I built XFree86 from xfree86.org's CVS repository.
part of XFree86, so it has no record in
/var/db/pkg. This is OK; the
port-building process checks for the existence of a program, not for
its record in
/var/db/pkg. When a port's installation record is
/var/db/pkg, however, it assumes that the port is there if
it found the program. (This is arguably a shortcoming in the ports
system, but there's no easy fix for this.)
What this means is that when a port lists
imake as a dependency,
Stale dependency: Xaw3d-1.5 -> imake-4.1.0: New dependency? (? to help): ?
Here I have a choice. I can either assign a new dependency, or I can
tell the package that this is no longer a dependency. While Xaw3d
imake, I know that on this particular system it will
never find an
imake package installed. I want to delete the
dependency. If you hit "?" for help, you'll see:
[Enter] to skip, [Ctrl]+[D] to delete, [.][Enter] to abort, [Tab] to
New dependency? (? to help):
Control-D it is.
Delete this? ([y]es/[n]o/[a]ll) [yes]
If I choose "yes," it will delete just this
dependency. If I choose "all" it will delete this dependency whenever
it appears. A lot of ports use
imake; I want to delete them all.
You'll see this show up later:
Stale dependency: xneko-4.4 -> XFree86-libraries-4.1.0: Delete this? ([y]es/[n]o/[a]ll) [yes] Deleted.
Other times, there really is a dependency elsewhere that
pkgdb can't guess at.
Stale dependency: plugger-3.3_1 -> timidity++-esound-2.10.4:
New dependency? (? to help): ?
[Enter] to skip, [Ctrl]+[D] to delete, [.][Enter] to abort, [Tab] to complete
New dependency? (? to help): timidity++-2.10.4
The tab completion makes this doable. I know that
timidity++-esound is using some version of
timidity, but I don't want
to go digging around to find out the whole version number. I can type
in the first few letters and hit tab, and get the rest of the package
name filled in.
Finally, at the end of the whole process
pkgdb will warn you about
Duplicated origin: graphics/Mesa3 - Mesa-3.4.1 Mesa-3.4.2_1 Remove any of them? [no]
Here I have a problem. It's not uncommon when you install software from packages to have the package require another package. You might have a slightly different version of the package already installed. If you're installing via package, however, it automatically installs dependencies even if a slightly different version is already installed. At some point this happened to my system. My laptop has two different versions of Mesa, but one has overwritten the other. It obviously cannot be too bad, as my system still works. (Eventually, I should entirely remove Mesa and install a known good version.) Still, I need to clean up the package database. I arbitrarily removed the older version.
Now you have a database of package information,
You'll also have a ports database,
/usr/ports/index.dbo. The various
portupgrade tools use these
databases to work their magic. Whenever you upgrade your ports tree,
upgrade the ports database and
/usr/ports/INDEX. You can do this
Now that we have the tools installed, let's see what we can do. To
see what you have that's obsolete, use
portversion (1). This works in
the same way as
pkg_info but is far faster.
# portversion Hermes = Mesa = ORBit < XFree86-aoutlibs < Xaw3d = aalib =
< means that the installed version is older than what is available
in the ports tree. You can easily run
portversion | grep '<' to see
which ports are obsolete. I do this every so often just to see what's
installed on my system. For example, my Apache package is outdated.
I'm no longer using Apache on my laptop, however, so I can remove it.
pkg_delete command still works, but if I use it my
package database will fall out of synch with my system. Use
# pkg_deinstall apache
---< Deinstalling 'apache-1.3.20'
[Updating the pkgdb in /var/db/pkg ... - 182 packages found (-1 +0) (...) done]
The nice thing is, I no longer need to give the full version number!
(That's bugged me about the FreeBSD
pkg_* tools for quite some time,
but never enough to do anything about it.) The
tools include globbing functions, which means that they can do pattern
I want to upgrade certain other ports. What's more, I want to upgrade
these while keeping the database synchronized, and updating the
now-correct entries in
/var/db/pkg. For example,
pkgversion tells me
gd is out of date. It's simple enough to upgrade:
# portupgrade gd ===> Cleaning for gettext-0.10.35 ===> Cleaning for gmake-3.79.1 ===> Cleaning for libtool-1.3.4_2 ===> Cleaning for jpeg-6b ===> Cleaning for png-1.2.0 ===> Cleaning for freetype2-2.0.5 ===> Cleaning for gd-1.8.4_4 If you want to compile in X support use 'make -DWITH_X11' instead ===> Extracting for gd-1.8.4_4 ....
You'll see the familiar
make output for a while. If you keep
watching, though, you'll see it make a detour to deinstall the
existing package, update the database, and keep going.
---> Deinstalling 'gd-1.8.4_3'
pkg_delete: package 'gd-1.8.4_3' is required by these other packages and may not be deinstalled (but I'll delete it anyway):
[Updating the pkgdb in /var/db/pkg ... - 180 packages found (-1 +0) (...) done]
===> Installing for gd-1.8.4_4
Once the install finishes, it updates the package database again. Everything is synchronized.
Now, let's look at something difficult. My
docproj port is out of
docproj port is a port that doesn't include anything in and
of itself, but has dependencies on every tool needed to edit the
FreeBSD Documentation Project. If you regularly build the docs tree,
you need this tool to be up-to-date. Also, you don't want to
recursively rebuild the whole thing just for a minor update;
has some huge dependencies.
If I wanted to save some time, I could use
portupgrade -P to tell
portupgrade to use packages for this. Time is something my computer
has no shortage of, so I'll use ports. We need to tell
recursively upgrade packages, however, so it will upgrade the packages
docproj. You can do this with the
-R flag. Let's give it
# portupgrade -R docproj
This port will try to ensure that the tools used by the FreeBSD
Documentation Project are installed on your system so you can convert documentation from SGML to other formats.
One of these components is JadeTeX, which depends on TeTeX. The source for TeTeX is larger than 30 MB, and may be a very long download.
If you do not want to produce PostScript and PDF formats from the
documentation, you do not need JadeTeX, and you should set the
JADETEX to "no". If you do want to produce PostScript and
PDF output then set
JADETEX to "yes".
*** Error code 1
** Command failed: make clean build
** Fix the problem and try again.
** The following packages were not installed or upgraded (-:skipped /
! textproc/docproj (docproj-1.4) (unknown build error)
Ugh. This port requires customization. You can edit
/usr/ports/textproc/docproj/Makefile to add
JADETEX=no somewhere. It
will now run transparently.
When you've finished, you can check how things stand with
You might find that other port dependencies have changed -- for
example, if port A depends on port B, but port C also depends on port
portupgrade A will not edit the dependencies of port C.
Be prepared to run
pkgdb -F on occasion to find and fix these
If you want to have the system handle these things itself, you can use
portupgrade -rR; this will upgrade both dependent and depending
ports. If you always want the latest software on your system, this is
the way to go.
portupgrade includes a variety of other tools that
simplify software management on FreeBSD, but this should be enough to
get you started.
You might find that you have to rebuild a single port multiple times
from the same distfile. If a port is revised,
portversion will catch
it and mark it as outdated. You'll want to keep the distfiles for the
ports you have installed, but delete outdated distfiles.
-D will do this. Similarly, you might want to remove shared
libraries that are not referenced in the package database. I don't
want to use that option -- remember, I installed XFree86 from scratch,
not from ports. This function would remove my X setup, ouch! But
portsclean -L would do this, and it would help deal with problems
such as leftover files from my defective Mesa install.
Finally, I have to say that this tool seems fairly robust. As I was finishing this article, I ran these programs to check my examples. I noticed one obsolete package, and simply had to try to upgrade it.
# portupgrade -R portupgrade
Yep, it works. I'd say this tool is a definite winner.
Michael W. Lucas
Read more Big Scary Daemons columns.
Return to the BSD DevCenter.
Copyright © 2009 O'Reilly Media, Inc.