The role of the network administrator is broad and varied. While most of our time is spent with day-to-day operational tasks such as assigning IP addresses and configuring hosts and applications to use an existing network, we'll occasionally be called upon to do some more interesting things.
One of the more interesting and rewarding activities of the network administrator involves the planning and design of a brand-new network and the selection and configuration of network applications to run across it. Network design inevitably involves compromise, trading off one characteristic against another. Common trade-offs are link bandwidth against cost, or latency against loss, or bandwidth against latency.
Consider the case of a remotely located site that you wish to network-connect to a central site. A satellite-based connection might offer high bandwidth at a reasonable cost but offer latency times of over a second or more, compared to a terrestrial link which might offer lower bandwidth but with latency of the order of hundreds of milliseconds instead for much the same price. A dedicated point-to-point link might offer a virtually loss-less connection but at a higher cost than a frame relay connection that exhibits some datagram loss.
network bandwidth - The volume of data a network can carry over a period of time. Usually expressed in terms of bits, kilobits or megabits per second for network links.
network latency - The total time taken to carry a unit of data from one point on a network to another point on the network. Sometimes called transit delay.
latency variation - The change in network latency from moment to moment. This often results from buffering caused by sudden bursts or lapses of data being carried by the network.
Choosing the best solution requires a detailed understanding of the end-user requirements and the network applications they will be using. Sometimes the best or only way of knowing what is the best network design is to actually try a few different ones and see which works best. Doing that is difficult and expensive unless you have a way of emulating the behavior of each of the options with your existing network. This is just one use of NISTNet.
Linux Network Administrator's Guide, 2nd Edition
NISTNet is a software package developed by Mark Carson of the North American National Institute of Standards and Technology (NIST) that allows network designers, application developers, and network researchers to pretty accurately emulate the performance conditions presented by a variety of TCP/IP networks and network paths.
NISTNet replaces the normal Linux IP forwarding mechanism with one that allows the network administrator to set and control the levels of each of a number of key network behaviors. These behaviors include datagram loss, datagram delay, delay variation, and the maximum available bandwidth in the forward and backward directions. Real networks display each of these characteristics. Consider the following examples:
NISTNet allows each of these conditions to be intentionally introduced into an otherwise healthy network connection to allow administrators to emulate larger, more complex networks and observe the resulting behavior of network protocols and applications.
NISTNet provides both command line and X11-based userspace clients to control the operation of the kernel code.
The NISTNet software package is public domain software and so is freely available for all to use and abuse as they please. The software provides a number of loadable kernel modules and corresponding userspace applications to control the behavior of the modules. Kernel patch files are supplied that provide replacement timer code that allows finer clock resolution, which is essential for accurate emulation.
The most recent version of the NISTNet software is version 2.0 alpha, revision 3. This version compiles against the most current 2.0 and 2.2 version kernels. The NISTNet source code is available from the NISTNet web site.
The software is fairly easy to compile. No special libraries are required.
There are only two small tricks. First, make sure that your kernel source is
either in the
/usr/src/linux directory or that there is a symlink
pointing to where you have your kernel source. There are scripts that automate
the application of the fast-timer patches that assume this to be the case.
Second, be sure to remember to inspect the
monitor/Imakefile file and
make sure the
OURXAWLIB define is pointing to the version of the
Athena Widgets library you wish to use; the default is probably OK.
To compile the software you should run the
Patch.Kernel script to
patch the kernel. Recompile the kernel with the patches applied using the
usual process. Be sure to select both of the new configuration options
Kernel hacking ---> [*] Fast timer [*] Packet routine hacking
When the kernel is compiled and installed, return to the top level of the NISTNet source and run:
make make install
This will compile both the text and X11 versions of the userspace programs as well as
the new kernel modules, and will install them in fairly sensible locations
on your machine. Debian users note: the kernel modules are installed in
/lib/modules/misc/, which is probably not what you want. You
can easily move them into the appropriate
misc/ subdirectory of the
kernel you've compiled. The installation step will also create some special
device files in the
/dev/ directory that provide the interface that
the userspace programs will use to talk to the kernel modules.
The way you'd usually use NISTNet is to install the software on a Linux-based router installed in your test environment. The router would have a number of network interfaces configured. For example, you might have two Ethernet interfaces configured, each supporting a different IP network. You'd place your test hosts on either side of the router and configure NISTNet to exhibit the characteristics you wish to model and run your test.
The primary module is
nistnet.o, which is the main body of the
network emulation code that augments the normal IP forwarding routines in the
kernel. The first thing you need to do is to load the module. You can use
either insmod or modprobe to do this:
When the module is loaded you will be able try out the client. You must have root permissions to run the client programs. While the text mode interface is convenient for scripting purposes, I find the X11 client much easier to use. The X11 client is called xnistnet.
When you first start the xnistnet client you will be presented with a large display looking something like that of figure 1.
Figure 1. NISTNet GUI client (click on image for full size view).
Each row of the display represents an emulation rule. In the left-hand panel, you configure the hosts or services that must match for that rule to apply when forwarding. You may specify host or network addresses, protocols such as TCP and UDP, and ports by name or by number. You must supply both a source and destination pattern, and rules are not bidirectional; that is, you must configure a rule for each direction. The rules for configuring a pattern are simple enough, but don't look it initially.
In the right-hand panel you configure the actual conditions that will be applied when that rule is matched, and in the right-most fields you are able to see some running statistics for the connections represented by the rule.
More comprehensive instructions and explanations of each of the fields are supplied with the package, so there is little point in reproducing them here. Instead, let's look at a simple example.
Let's imagine that our lab setup is built of three Linux machines. One, the
NISTNet router, has an Ethernet interface and a PPP interface to another Linux
machine with a null modem link at 33.6 kilobits/second. The third Linux machine
is on the Ethernet network and has the IP address of
that we'll use to send test traffic to the PPP-connected Linux machine, which
has the address
192.168.2.1. Our test environment is illustrated in
Figure 2. Example test environment.
We'll illustrate the capability of NISTNet using a simple ping test. In practice you'd probably be using something much more relevant to your application. In a stable state, no rules configured, our ping test looks like:
ping -c 10 -i 2 -s 1460 192.168.2.1PING 192.168.2.1 (192.168.2.1): 1460 data bytes 1468 bytes from 192.168.2.1: icmp_seq=0 ttl=254 time=1097.6 ms 1468 bytes from 192.168.2.1: icmp_seq=1 ttl=254 time=1064.4 ms 1468 bytes from 192.168.2.1: icmp_seq=2 ttl=254 time=1057.2 ms 1468 bytes from 192.168.2.1: icmp_seq=3 ttl=254 time=1021.9 ms 1468 bytes from 192.168.2.1: icmp_seq=4 ttl=254 time=1004.6 ms 1468 bytes from 192.168.2.1: icmp_seq=5 ttl=254 time=1004.8 ms 1468 bytes from 192.168.2.1: icmp_seq=6 ttl=254 time=1053.1 ms 1468 bytes from 192.168.2.1: icmp_seq=7 ttl=254 time=1056.1 ms 1468 bytes from 192.168.2.1: icmp_seq=8 ttl=254 time=1018.9 ms 1468 bytes from 192.168.2.1: icmp_seq=9 ttl=254 time=1062.7 ms --- 192.168.2.1 ping statistics --- 10 packets transmitted, 10 packets received, 0% packet loss round-trip min/avg/max = 1004.6/1044.1/1097.6 ms
Let's add a rule that will insert a 500 millisecond delay in datagrams in
the forward direction only. To do this we start the xnistnet client
and enter the address of our source machine,
192.168.1.1, into the
source field of a rule, and the destination address,
the destination field of the same rule. We then enter
500 into the
field labelled "
Delay (mS)" in the right-hand panel, and hit the
"Update" button to activate it. When we now look at our ping test we see:
ping -c 10 -i 2 -s 1460 192.168.2.1PING 192.168.2.1 (192.168.2.1): 1460 data bytes 1468 bytes from 192.168.2.1: icmp_seq=0 ttl=254 time=1566.8 ms 1468 bytes from 192.168.2.1: icmp_seq=1 ttl=254 time=1602.2 ms 1468 bytes from 192.168.2.1: icmp_seq=2 ttl=254 time=1552.5 ms 1468 bytes from 192.168.2.1: icmp_seq=3 ttl=254 time=1524.8 ms 1468 bytes from 192.168.2.1: icmp_seq=4 ttl=254 time=1515.1 ms 1468 bytes from 192.168.2.1: icmp_seq=5 ttl=254 time=1515.4 ms 1468 bytes from 192.168.2.1: icmp_seq=6 ttl=254 time=1517.0 ms 1468 bytes from 192.168.2.1: icmp_seq=7 ttl=254 time=1537.3 ms 1468 bytes from 192.168.2.1: icmp_seq=8 ttl=254 time=1517.8 ms 1468 bytes from 192.168.2.1: icmp_seq=9 ttl=254 time=1518.9 ms --- 192.168.2.1 ping statistics --- 10 packets transmitted, 10 packets received, 0% packet loss round-trip min/avg/max = 1515.1/1536.7/1602.2 ms
It's easy to see that that did precisely what was expected of it.
Let's try another test. Let's this time try having NISTNet emulate a circuit
with 25% datagram loss. To do this we edit our rule, entering
the delay field, entering
25 into the "
field and again hitting "Update". When we run our ping test it now looks like:
ping -c 10 -i 2 -s 1460 192.168.2.1PING 192.168.2.1 (192.168.2.1): 1460 data bytes 1468 bytes from 192.168.2.1: icmp_seq=0 ttl=254 time=1014.3 ms 1468 bytes from 192.168.2.1: icmp_seq=2 ttl=254 time=1011.1 ms 1468 bytes from 192.168.2.1: icmp_seq=3 ttl=254 time=1004.4 ms 1468 bytes from 192.168.2.1: icmp_seq=4 ttl=254 time=1027.4 ms 1468 bytes from 192.168.2.1: icmp_seq=6 ttl=254 time=1033.9 ms 1468 bytes from 192.168.2.1: icmp_seq=7 ttl=254 time=1056.5 ms 1468 bytes from 192.168.2.1: icmp_seq=9 ttl=254 time=1043.5 ms --- 192.168.2.1 ping statistics --- 10 packets transmitted, 7 packets received, 30% packet loss round-trip min/avg/max = 1004.4/1027.3/1056.5 ms
Ten datagrams is too short a test to expect very accurate results for this sort of test, but it's clear that again NISTNet has done what we've asked of it.
Finally, let's change our test such that rather than dropping 25% of datagrams,
we duplicate them. To do this we zero the drop field, and enter
into the "
Dup %" field instead, hitting the "Update" button
once more to activate the change. Our ping test now looks like:
ping -c 10 -i 2 -s 1460 192.168.2.1PING 192.168.2.1 (192.168.2.1): 1460 data bytes 1468 bytes from 192.168.2.1: icmp_seq=0 ttl=254 time=1097.0 ms 1468 bytes from 192.168.2.1: icmp_seq=0 ttl=254 time=1698.0 ms (DUP!) 1468 bytes from 192.168.2.1: icmp_seq=0 ttl=254 time=1893.2 ms (DUP!) 1468 bytes from 192.168.2.1: icmp_seq=1 ttl=254 time=1013.8 ms 1468 bytes from 192.168.2.1: icmp_seq=2 ttl=254 time=1137.5 ms 1468 bytes from 192.168.2.1: icmp_seq=3 ttl=254 time=1080.7 ms 1468 bytes from 192.168.2.1: icmp_seq=4 ttl=254 time=993.6 ms 1468 bytes from 192.168.2.1: icmp_seq=5 ttl=254 time=993.7 ms 1468 bytes from 192.168.2.1: icmp_seq=6 ttl=254 time=1219.9 ms 1468 bytes from 192.168.2.1: icmp_seq=6 ttl=254 time=1770.3 ms (DUP!) 1468 bytes from 192.168.2.1: icmp_seq=6 ttl=254 time=1828.0 ms (DUP!) 1468 bytes from 192.168.2.1: icmp_seq=7 ttl=254 time=1266.5 ms 1468 bytes from 192.168.2.1: icmp_seq=7 ttl=254 time=1514.3 ms (DUP!) 1468 bytes from 192.168.2.1: icmp_seq=8 ttl=254 time=1041.2 ms 1468 bytes from 192.168.2.1: icmp_seq=9 ttl=254 time=1065.5 ms --- 192.168.2.1 ping statistics --- 10 packets transmitted, 10 packets received, +5 duplicates, 0% packet loss round-trip min/avg/max = 993.6/1307.5/1893.2 ms
You can design tests that combine all of these together, and of course we've only looked at the simplest of capabilities. NISTNet has a whole suite of much more sophisticated things it can do with datagrams.
One final thing to look at are the statistics that are collected in real time. Start a ping test and scroll the right-hand panel across to the right-most fields. You will see running tallies of packet sizes, bytes transmitted, average bandwidth, and others. These are useful for keeping an eye on progress of the tests.
If you're in a situation where you need to quickly test how a network application or protocol will perform under realistically poor conditions, then NISTNet provides an excellent solution. Mark has gone to a lot of work to ensure that the emulation is as accurate and useful as possible, and I'm sure you'll agree that it truly is one of those tools that you know will come in handy one day; keep it around.
Terry Dawson is the author of a number of network-related HOWTO documents for the Linux Documentation Project, a co-author of the 2nd edition of O'Reilly's Linux Network Administrators Guide, and is an active participant in a number of other Linux projects.
Read more Linux Network Administration columns.
Discuss this article in the O'Reilly Network Linux Forum.
Return to the Linux DevCenter.
Copyright © 2009 O'Reilly Media, Inc.