BSD DevCenter
oreilly.comSafari Books Online.Conferences.

advertisement


CVSup Infrastructure
Pages: 1, 2

First off, here's how the FreeBSD CVSup mirror system works.



There's a master CVS repository on freefall.freebsd.org. This is the absolutely authoritative source of FreeBSD code. Users cannot, under any circumstances, update their systems from freefall. It's also a hard-working machine. As the authoritative repository, it must check to see if files have been changed by some program other than cvsupd. This is done with stat (2) which, while not particularly expensive used one at a time, devours disk resources when it has to check every single file that has ever been in FreeBSD. Every time someone updates from freefall, disk usage climbs.

The CVSup server on freefall has only one client, cvs-master.freebsd.org. Its purpose is to serve as a main source for official mirrors. "The master mirror is the most efficient by far," says Polstra. "It doesn't have to do much disk I/O [those stat() calls], and it doesn't have to do too much thinking." The files are only touched by cvsupd, and cvsupd knows perfectly well what files it's used. As such, cvs-master can support many main mirrors. Since it's a key part of the FreeBSD infrastructure, however, access to cvs-master is tightly restricted to mirror sites only.

These final mirrors are what us lowly users can access. Mirror machines pull their updates from cvs-master. Because cvs-master updates every 6 minutes, code becomes available on a user mirror not more than 66 minutes after it appears on freefall. That's not bad for a worldwide data distribution system.

If you want to be a part of all this, it's not that hard.

To run an official mirror, you need to first check your hardware. Polstra recommends at least a 400MHz Pentium II, 256MB of RAM, and a good, fast hard disk. This would support 8 to 10 users most of the time, unless there's a recent release. (A release touches every file in the repository, increasing the amount of work needed to update, hence boosting the time required.)

One particular mirror server used to have a single disk, 128MB of RAM, and a PII-400 MHz CPU. It could handle up to eight clients simultaneously, but interactive performance was horrendous when the system was under load. That same system has been upgraded to two Ultra2 SCSI disks -- one for the system files and one for the repository, and 512MB of RAM. Its limit has been upped to 20 clients at a time, and interactive performance is always excellent.

Adding RAM is the best way to increase a CVSup mirror's performance. Updates will happen more quickly -- users will stay connected for a shorter time, so the system load will drop and it can handle more users. This is something of the opposite of the "death spiral" an overloaded machine can suffer from.

Second, you need to check your bandwidth. One mirror chosen at random transferred about 1.1 megabits of traffic out and a quarter-megabit in during a randomly-chosen day. You can't quite serve this over an ISDN line, but a T1 has plenty of capacity to spare.

If you have the hardware and the bandwidth to donate, contact Polstra at jdp@freebsd.org. Include your location, as he tries to balance mirrors by geography. If your area is short on mirrors and you fit his other requirements, he might take you up on it. John chooses the mirror sites carefully, balancing the needs against the locations offered and the capacity. "I would be unlikely to allow 30 mirror sites in, say, Iceland," he says.

Next time, we'll look at the software involved in setting up a mirror. Hopefully, we can force John to soon change the naming scheme on the mirrors to something like cvsup14.yourstatehere.freebsd.org. And even if we can't do that, you can set up a local mirror to meet the needs of your organization.

Michael W. Lucas


Read more Big Scary Daemons columns.

Return to the BSD DevCenter.





Sponsored by: