Traffic shaping is the general term given to a broad range of techniques designed to enforce prioritization policies on the transmission of data over a network link. In this month's column, we'll look at some of the IP traffic shaping tools available for Linux and a simple example of how to use them.
Most of us will at some time or another have experienced the effects of network latency and queuing. A common experience is the sometimes frustrating delay before characters are echoed when ssh- or Telnet-connected to a remote host. Any of you who have been forced to use a low speed dial-up PPP or SLIP connection will have seen the effect that a file transfer has on interactive traffic over the connection. The file transfer easily consumes most of the link bandwidth, forcing the Telnet or ssh data to be queued, waiting for a free slot before being transmitted across the link.
This problem occurs because the datagrams containing the file transfer data are given equal priority on the link to the Telnet or ssh datagrams. No consideration is given to the type of data contained within the datagram when queuing it for transmission; queuing is performed on a "First In, First Out" (FIFO) basis, and the datagrams are scheduled for transmission on a "First Come, First Served" basis. When a new datagram arrives at the queue, it is added to the tail of the queue; when the link bandwidth becomes available, the datagram at the head of the queue is transmitted.
Traffic shaping allows us to implement a specific policy that alters the way in which data is queued for transmission. Datagrams associated with file transfers are generally quite large, often MTU sized, while datagrams associated with interactive sessions like ssh or Telnet are often quite small. All data takes time to transmit over a network connection; the larger the datagram, the longer it takes. As a result, if you have to wait for a large datagram associated with a file transfer to be transmitted before your Telnet keystroke can be transmitted, you will perceive considerable delay. Additionally, because file transfers don't need to wait for human input to continue, you can be sure that at any time you hit a key in your Telnet session, there are already not one, but a number of datagrams from the file transfer session sitting in the queue and further compounding the delay. Ultimately, it will take the same amount of time to transmit all of a set of datagrams across a network link, no matter what order they are transmitted in, so at some point you may decide that it is worth trading off a small amount of delay in completing the file transfer for more normal response times for your interactive sessions like ssh or Telnet by giving them higher priority.
Implicit here is a key principle: Traffic shaping affects only data to be transmitted across a link; it is mostly too late to shape data after it has been received. This has deployment implications that we'll look at later.
The shaper device
For some time a simple IP shaping mechanism has been supported by the Linux kernel. This is called the shaper device. The shaper was primarily designed to limit the amount of bandwidth that nominated data paths could consume. The shaper device is a virtual network device that is configured with two important parameters, a bandwidth limit and a physical network device to which it is attached. The sum total of traffic routed via the shaper device will be limited to meet the bandwidth cap by simply discarding any datagrams that would cause the cap to be exceeded.
The shaper device is included in your kernel if you select the
Traffic Shaper (EXPERIMENTAL) option when compiling it.
It may be built as either a loadable module or an inbuilt device. The shaper
uses a special configuration tool called
and is included in most Linux distributions.
To use the shaper, you first attach a physical network device to the master
shaper0. You then configure the shaper rate limit.
Finally you route the traffic you wish to be rate limited via the shaper
Let's assume we have a simple network that looks something like that depicted
in Figure 1. We have three Ethernet-based IP networks:
192.168.3.0/24. Internetworking these we have two routers, unimaginatively named
Figure 1. A simple network.
Further, imagine that we wish to shape all traffic routed from the
192.168.1 network to the
192.168.3 network to 5 Mbps while allowing all other traffic to be unshaped.
Remembering that shaping applies only in the forward direction, it should be
clear that it makes sense to configure our shaping on the
router. The network configuration on the
A router might look
# Configure our ethernet devices # ifconfig eth0 192.168.1.1 netmask 255.255.255.0 broadcast 192.168.1.255 up ifconfig eth1 192.168.2.1 netmask 255.255.255.0 broadcast 192.168.2.255 up # Associate the shaper device with our eth1 # device and apply the rate limit # # You must do this before bringing the shaper # device up. # shapecfg attach shaper0 eth1 # associate with eth0 shapecfg speed shaper0 5000000 # shape rate limit set to 5Mbps # Configure our shaper device # # The shaper device will usually be configured # with the same IP address as # that of the associated physical device. # ifconfig shaper0 192.168.2.1 netmask 255.255.255.0 up # Configure our routing such that traffic destined # for the 192.168.3 network is shaped and all other # traffic is unshaped. # route add default gw 192.168.2.2 dev eth1 route add 192.168.3.0 netmask 255.255.255.0 gw 192.168.2.2 dev shaper0
This configuration would cause all traffic routed via the
shaper0 network device of router
A to be rate limited to 5 Mbps. Any data received by the shaper device that would allow it to stay within the set bandwidth limit is sent to the associated physical device for transmission. Any data that would cause the limit to be exceeded is dropped (thrown away).
All traffic routed directly via the Ethernet device is unshaped. In our example any data routed via our default route (most likely destined for the
192.168.2 network!) will be unshaped.
If we wanted to bandwidth limit the data flowing in the reverse direction,
i.e., from the
192.168.3 network to the
192.168.1 network, we would use an almost identical configuration on the
B router (remember that shaping works in the forward direction only), substituting
appropriate IP addresses to reflect the data flowing from right to left on our
If you carefully design your routing, you can produce a variety of simple
and functional IP traffic shaping designs using the shaper device. Exploiting
ip policy routing tool, you could for example cause
shaping to be applied only to data sourced from chosen hosts. You can shape
anything you can route.
QoS Packet Scheduling, more sophisticated shaping
While the shaper device is useful in many circumstances, many real-world applications demand more flexible and sophisticated shaping techniques. A number of standards exist to implement "Quality of Service" (QoS) in an IP environment. The best known of these are the "Resource ReSerVation Protocol" (RSVP) described in RFC-2205, "Integrated Services" described in RFC-1633, and "Differentiated Services" described in RFC-2475. These each rely heavily on the ability of routers, especially at the edge of a network, to be able to shape the traffic flows they are carrying.
If you're interested in locating some of the related software, you can find it at:
Kernel support -- the
Linux supports a number of the most important QoS mechanisms in quite a flexible and powerful way. Packet classification methods supported include RSVP,
firewall marking, route-based, and Ugly 32-bit key. Some of the supported
scheduling disciplines are class-based queues (CBQ), token bucket flows (TBF),
3-band priority queues, random early drop (RED), stochastic fairness
queueing (SFQ), and Clark-Shenker-Zhang (CSZ). Kernel support for these is
provided in the
QoS and/or fair queueing kernel
configuration menu of the
Networking options menu. You should
select each of the datagram classifiers and schedulers that you will require.
They may be compiled as loadable modules, and this is probably a good idea.
Each of the classifiers and schedulers may be combined in a multitude of ways
to meet a wide variety of QoS and IP traffic requirements.
The primary tool for configuration of the Linux QoS features is the
/sbin/tc (Traffic Control) tool included in the
iproute package. To use the
tc tool, you must have
support for the
Kernel/User netlink socket in your kernel.
This is a very rich and complex topic. There is not enough space in this
column to adequately cover all you will need to know to deploy IP QoS in your
Linux-based networking environment. Fortunately, some good documentation is
available on-line that will help you. Saravanan Radhakrishnan has a published
document entitled the Linux-IP-QoS-HOWTO; this document provides a good description of the way that the various QoS mechanisms have been implemented in Linux and worked examples of how to use the
In a future column, I'll present a worked example of a QoS deployment. In the meantime, have fun exploring the features available and experiment!
Terry Dawson is the author of a number of network-related HOWTO documents for the Linux Documentation Project, a co-author of the 2nd edition of O'Reilly's Linux Network Administrators Guide, and is an active participant in a number of other Linux projects.
Read more Linux Network Administration columns.
Discuss this article in the O'Reilly Network Linux Forum.
Return to the Linux DevCenter.