Understanding Archiversby Dru Lavigne
In the next few articles, I'd like to take a look at backups and archiving utilities.
If you're like I was when I started using Unix, I was intimidated
by the words
dump, and a quick peek at their respective man pages did not alleviate my fears.
So I quickly convinced myself that I really didn't need to learn how those utilities worked. After all, I didn't even own a tape drive on my home FreeBSD system. Yes, I knew that backups were really, really important, but surely I could just copy the files I needed as I needed them.
I've since learned that copying files is actually the hard way to do a backup and is not particularly conducive to me backing up everything I should on a regular basis. In today's article, I'd like to introduce the concept of archiving, which archiving utilities are available, and some of the differences between the archiving utilities. In the next few articles, I'll continue by demonstrating the usage of each of these archiving utilities.
I'm currently logged in as the user "dru". I'll
cd into my home
directory and take a look at its contents:
cd ls . .xinitrc perlscripts .. articles this .cshrc ip.c tricks .history jpegs unix .mailrc lynx_bookmarks.html .ssh2 pdfs
Let's say I want to backup the contents of my perlscripts directory to another directory I'll call backup. If I try:
cp perlscripts backup cp: perlscripts/ is a directory (not copied).
I'll see that the copy operation fails, since perlscripts is a directory.
However, if I remember to use the
r or recursive switch:
cp -r perlscripts backup
the copy command will be successful. A new directory named backup will be created for me and the contents of perlscripts will be copied to it.
That seemed easy enough, but it may not be the best way to do a backup. For starters, let's do a long listing of the original directory and the new backup directory:
ls -l perlscripts total 6 drwxr-xr-x 2 dru wheel 512 Oct 4 07:29 . drwxr-xr-x 22 dru wheel 4096 Mar 24 07:07 .. -rwxr-xr-x 1 dru wheel 801 Feb 16 12:32 time.pl ls -l backup total 3 drwxr-xr-x 3 dru wheel 512 Mar 24 08:49 . drwxr-xr-x 8 dru wheel 512 Mar 24 08:49 .. -rwxr-xr-x 1 dru wheel 801 Mar 24 08:49 time.pl
You'll note that the last modified time for the file
time.pl has been
changed to the time I made the recursive copy, rather than the last
time this file was actually modified, which was back in February.
This may or may not be a big deal to you if you are only interested in backing up some files in your own home directory. However, this could certainly cause confusion if this was the backup solution for larger portions of your FreeBSD system.
There are other considerations when using
cp -r to backup files.
What if I wanted to backup files for several users? I would probably
do the backup as the superuser. Let's see what happens if I repeat that
copy, but this time as the superuser:
rm -r backup su Password: cp -r perlscripts backup ls -l backup total 3 drwxr-xr-x 3 root wheel 512 Mar 24 09:20 ./ drwxr-xr-x 8 dru wheel 512 Mar 24 09:20 ../ -rwxr-xr-x 1 root wheel 801 Mar 24 09:20 time.pl
You'll note that both the backup directory and the
time.pl file are owned by the user who did the copy, in this case root. This situation could have
been avoided if I had remembered to include the
p switch to preserve the
Just imagine the nightmare if I had backed up each user's
home directory as the superuser using
cp -r; I would have to readjust the
ownership and possibly the permissions of any file that needed to be restored, plus the original file modification times would still be unknown.
If that's still not a big deal to you, consider how I would backup my
entire home directory using
cp -r. I do NOT want to do it this way, even
though it seems logical enough:
cd cp -r . backup
If I do try to do this, my hard drive will churn for an eerily long period
of time before giving me an error message that includes several screens
worth of the word backup and something about the name being too long. This
is because the
cp command will go into an endless loop if your
destination happens to be in the same directory or a subdirectory of the
source you are backing up. It will copy
backup/backup/backup and so on until it runs out of space.
So how would I backup my entire home directory? This is where things start to involve a bit more work and I start to get the gnawing suspicion that there has to be an easier way to accomplish this. This will work:
mkdir backup cp -r .cshrc .history .mailrc .ssh2 .xinitrc articles file ip.c jpegs lynx_bookmarks.html pdfs perlscripts tricks unix backup/
but will quickly become time-consuming and inconvenient as the number of files in my home directory continues to grow. I could get a bit fancier by coming up with wildcard expressions that represent all of the files and directories in my home directory, but I would still be doing things the hard way.
This is where the concept of archiving and utilities that were designed to do archiving come into play. So what exactly is an archive? It is a file containing a collection of other files in a structure that preserves the contents, permissions, timestamp, owner, group, and pathnames of the original files so they can be reconstructed at a later time. In other words, archiving utilities can copy all of the files and subdirectories within a directory and then recreate that original directory structure without losing any permissions or modification times along the way.
This is actually even more interesting once you realize that there are
devices that don't even know what a filesystem is or how to read a
filesystem hierarchy. We are used to thinking of our files living in a
filesystem hierarchy. For example, my
time.pl file is a file that lives in the
perlscripts directory which is a subdirectory of my home directory (dru) which
is a subdirectory of the home directory which is a subdirectory of the /usr
Any device that can contain a filesystem and therefore understand a filesystem hierarchy is known as a block device. The hard drive that contains your FreeBSD operating system is an example of a block device.
However, there are devices that do not understand what a filesystem
hierarchy is. Consider how a tape device works. When you write data to
a tape, your characters are simply passed to the tape one after the other,
or sequentially. There is no filesystem, or any concept that the file
belongs within the perlscripts directory. Such devices are known as
character devices and are often called "raw."
Archiving utilities can backup to either a block or character device. The archive file itself contains all of the information required to recreate the original file hierarchy; that information is saved along with your data. This means you can backup your data to a character device such as a tape drive, and then later restore your data to a block device such as your hard drive.
There are several archiving utilities that come with your FreeBSD system.
I will be covering
see what the
whatis command has to say about each of these utilities:
whatis tar cpio pax dd dump tar(1) - tape archiver; manipulate tar archive files cpio(1) - copy files to and from archives pax(1) - read and write file archives and copy directory hierarchies dd(1) - convert and copy a file dump(8), rdump(8) - filesystem backup
pax are considered to be archivers. We'll
tar is easiest to use when you want to backup entire directory
structures. In contrast, the
cpio utility is the easiest command to use
when you want to pick and choose which files to backup. And the
pax command is a combination of both these commands with a bit of added
functionality thrown in.
dd utility is interesting -- it can actually convert files as it
backs them up. We'll see that this can be invaluable, say, when backing up
files from a PC to a SPARC. Finally, the
dump command is designed to
backup an entire filesystem, not just a directory structure.
I want to discuss a few more items, though, before we start using each of these commands. Most of these commands assume that you will be backing up to a SCSI tape drive but will let you change this default with a switch. Even if you don't have a tape drive, it is useful to understand the naming syntax your FreeBSD system uses for tape devices.
Like other Unix systems, FreeBSD stores information regarding devices in
/dev directory. Let's do a long listing of the first few files in
ls -l /dev | head total 62 drwxr-xr-x 3 root wheel 14336 Mar 17 19:31 . drwxr-xr-x 18 root wheel 512 Jan 31 19:17 .. -r-xr-xr-x 1 root wheel 43405 Sep 18 2001 MAKEDEV -r-xr-xr-x 1 root wheel 2064 Sep 18 2001 MAKEDEV.local crw-r----- 2 root operator 117, 0 Sep 22 2001 acd0a crw-r----- 2 root operator 117, 2 Sep 22 2001 acd0c crw-r----- 2 root operator 117, 8 Sep 22 2001 acd1a crw-r----- 2 root operator 117, 10 Sep 22 2001 acd1c crw-r----- 2 root operator 116, 0x00010002 Sep 22 2001 ad0
Notice the difference in the fifth field of that long listing. The first
few files indicate their size in bytes -- for example, the file
43405 bytes in size. However, the last five files have a "117," or "116," instead. Note that these files are also character devices; you can
tell this as their file mode is
c (just before their permissions).
Directories have a file mode of
d and regular files have a file mode of
The device files in the
/dev directory are really just pointers to a
driver contained in the kernel for the device that each device file represents.
This means that these files are really empty, they are just pointers. The
value in what is normally the size field of
ls -l represents a
"major_number,minor_number. " For example, the device file
acd1c has a major number of 117 and no minor number. The major number indicates which driver
should be used; the minor number gives any additional information about
the device to the driver.
MAKEDEV file in this directory is really a shell script used to make the device files. If you want to find out what a device file
refers to, read the comments at the beginning of this file. For example,
to see which devices refer to tape devices, I'll search this file for the
more /dev/MAKEDEV /tape
And I'll find that the following tape drives are supported on my FreeBSD system:
sa SCSI tape driver (formerly called st) wt QIC-02 or QIC-36 3M cartridge tape
There is also a third type that is supported:
wst ATAPI tape drive on IDE bus
Each of these has an associated man page which you can read if you have one of these tape devices.
If I look for these devices in the
/dev directory, I'll note that they
usually come with some additional letters:
ls /dev | grep wt nrwt0 nrwt0b nrwt0c nrwt0d rwt0 rwt0b rwt0c rwt0d
Most tape devices (but not all) will include the letter "r" indicating that they are a "raw" or character device. By default, after you backup to a tape device, it will rewind; meaning your backup will be overwritten if you do another backup to that tape. To prevent this default behavior, use the device that includes the letter "n" for no rewind.
Occasionally, a device will also include an "e," meaning that it will eject the tape once the backup is complete.
The last thing I want to mention in today's article is the difference between absolute and relative pathnames. Since an archiving utility will save the pathname of a file and use that pathname information when recreating the file, it is important to know the difference between the two types of pathnames.
If a pathname begins with a
/ it means it is an absolute pathname. This is usually considered to be a bad thing in a backup as you will only be
able to restore that file to the original directory it came from, meaning you
will lose any changes you've made to that file since you backed it up.
Even if you are in a different directory when you restore that file, it
will still restore that file to its original location.
If a pathname begins with
./ or no
/ at all it means it is a relative
pathname. This is usually considered to be a good thing in a backup as the
file can be restored anywhere. You simply
cd to the directory you want
to restore the file to, and the archiver will add the current directory to
the pathname as it restores the file.
In next week's article, we'll continue this series by demonstrating how to
Dru Lavigne is a network and systems administrator, IT instructor, author and international speaker. She has over a decade of experience administering and teaching Netware, Microsoft, Cisco, Checkpoint, SCO, Solaris, Linux, and BSD systems. A prolific author, she pens the popular FreeBSD Basics column for O'Reilly and is author of BSD Hacks and The Best of FreeBSD Basics.
Read more FreeBSD Basics columns.
Return to the BSD DevCenter.