oreilly.comSafari Books Online.Conferences.


Linux Network Administration

Traffic Shaping


Traffic shaping is the general term given to a broad range of techniques designed to enforce prioritization policies on the transmission of data over a network link. In this month's column, we'll look at some of the IP traffic shaping tools available for Linux and a simple example of how to use them.

Most of us will at some time or another have experienced the effects of network latency and queuing. A common experience is the sometimes frustrating delay before characters are echoed when ssh- or Telnet-connected to a remote host. Any of you who have been forced to use a low speed dial-up PPP or SLIP connection will have seen the effect that a file transfer has on interactive traffic over the connection. The file transfer easily consumes most of the link bandwidth, forcing the Telnet or ssh data to be queued, waiting for a free slot before being transmitted across the link.

This problem occurs because the datagrams containing the file transfer data are given equal priority on the link to the Telnet or ssh datagrams. No consideration is given to the type of data contained within the datagram when queuing it for transmission; queuing is performed on a "First In, First Out" (FIFO) basis, and the datagrams are scheduled for transmission on a "First Come, First Served" basis. When a new datagram arrives at the queue, it is added to the tail of the queue; when the link bandwidth becomes available, the datagram at the head of the queue is transmitted.

Traffic shaping allows us to implement a specific policy that alters the way in which data is queued for transmission. Datagrams associated with file transfers are generally quite large, often MTU sized, while datagrams associated with interactive sessions like ssh or Telnet are often quite small. All data takes time to transmit over a network connection; the larger the datagram, the longer it takes. As a result, if you have to wait for a large datagram associated with a file transfer to be transmitted before your Telnet keystroke can be transmitted, you will perceive considerable delay. Additionally, because file transfers don't need to wait for human input to continue, you can be sure that at any time you hit a key in your Telnet session, there are already not one, but a number of datagrams from the file transfer session sitting in the queue and further compounding the delay. Ultimately, it will take the same amount of time to transmit all of a set of datagrams across a network link, no matter what order they are transmitted in, so at some point you may decide that it is worth trading off a small amount of delay in completing the file transfer for more normal response times for your interactive sessions like ssh or Telnet by giving them higher priority.

Implicit here is a key principle: Traffic shaping affects only data to be transmitted across a link; it is mostly too late to shape data after it has been received. This has deployment implications that we'll look at later.

The shaper device

For some time a simple IP shaping mechanism has been supported by the Linux kernel. This is called the shaper device. The shaper was primarily designed to limit the amount of bandwidth that nominated data paths could consume. The shaper device is a virtual network device that is configured with two important parameters, a bandwidth limit and a physical network device to which it is attached. The sum total of traffic routed via the shaper device will be limited to meet the bandwidth cap by simply discarding any datagrams that would cause the cap to be exceeded.

The shaper device is included in your kernel if you select the Traffic Shaper (EXPERIMENTAL) option when compiling it. It may be built as either a loadable module or an inbuilt device. The shaper uses a special configuration tool called /usr/sbin/shapecfg and is included in most Linux distributions.

To use the shaper, you first attach a physical network device to the master shaper device: shaper0. You then configure the shaper rate limit. Finally you route the traffic you wish to be rate limited via the shaper network device.

Let's assume we have a simple network that looks something like that depicted in Figure 1. We have three Ethernet-based IP networks:,, and Internetworking these we have two routers, unimaginatively named A and B.

A simple network.

Figure 1. A simple network.

Further, imagine that we wish to shape all traffic routed from the 192.168.1 network to the 192.168.3 network to 5 Mbps while allowing all other traffic to be unshaped.

Remembering that shaping applies only in the forward direction, it should be clear that it makes sense to configure our shaping on the A router. The network configuration on the A router might look something like:

# Configure our ethernet devices
ifconfig eth0 netmask 
  broadcast up
ifconfig eth1 netmask 
  broadcast up

# Associate the shaper device with our eth1 
# device and apply the rate limit
# You must do this before bringing the shaper 
# device up.
shapecfg attach shaper0 eth1     
# associate with eth0
shapecfg speed shaper0 5000000   
# shape rate limit set to 5Mbps

# Configure our shaper device
# The shaper device will usually be configured 
# with the same IP address as
# that of the associated physical device.
ifconfig shaper0 netmask up

# Configure our routing such that traffic destined 
# for the 192.168.3 network is shaped and all other 
# traffic is unshaped.
route add default gw dev eth1
route add netmask 
  gw dev shaper0

This configuration would cause all traffic routed via the shaper0 network device of router A to be rate limited to 5 Mbps. Any data received by the shaper device that would allow it to stay within the set bandwidth limit is sent to the associated physical device for transmission. Any data that would cause the limit to be exceeded is dropped (thrown away).

All traffic routed directly via the Ethernet device is unshaped. In our example any data routed via our default route (most likely destined for the 192.168.2 network!) will be unshaped.

If we wanted to bandwidth limit the data flowing in the reverse direction, i.e., from the 192.168.3 network to the 192.168.1 network, we would use an almost identical configuration on the B router (remember that shaping works in the forward direction only), substituting appropriate IP addresses to reflect the data flowing from right to left on our diagram.

If you carefully design your routing, you can produce a variety of simple and functional IP traffic shaping designs using the shaper device. Exploiting the ip policy routing tool, you could for example cause shaping to be applied only to data sourced from chosen hosts. You can shape anything you can route.

QoS Packet Scheduling, more sophisticated shaping

While the shaper device is useful in many circumstances, many real-world applications demand more flexible and sophisticated shaping techniques. A number of standards exist to implement "Quality of Service" (QoS) in an IP environment. The best known of these are the "Resource ReSerVation Protocol" (RSVP) described in RFC-2205, "Integrated Services" described in RFC-1633, and "Differentiated Services" described in RFC-2475. These each rely heavily on the ability of routers, especially at the edge of a network, to be able to shape the traffic flows they are carrying.

Related software

If you're interested in locating some of the related software, you can find it at:

• iproute

• RSVP daemon

• Linux patch for RSVP daemon

• Sample applications supporting RSVP

• Kernel support -- the net/sched/ directory of your 2.2 (or later) kernel source tree.

Linux supports a number of the most important QoS mechanisms in quite a flexible and powerful way. Packet classification methods supported include RSVP, firewall marking, route-based, and Ugly 32-bit key. Some of the supported scheduling disciplines are class-based queues (CBQ), token bucket flows (TBF), 3-band priority queues, random early drop (RED), stochastic fairness queueing (SFQ), and Clark-Shenker-Zhang (CSZ). Kernel support for these is provided in the QoS and/or fair queueing kernel configuration menu of the Networking options menu. You should select each of the datagram classifiers and schedulers that you will require. They may be compiled as loadable modules, and this is probably a good idea. Each of the classifiers and schedulers may be combined in a multitude of ways to meet a wide variety of QoS and IP traffic requirements.

The primary tool for configuration of the Linux QoS features is the /sbin/tc (Traffic Control) tool included in the iproute package. To use the tc tool, you must have support for the Kernel/User netlink socket in your kernel.

This is a very rich and complex topic. There is not enough space in this column to adequately cover all you will need to know to deploy IP QoS in your Linux-based networking environment. Fortunately, some good documentation is available on-line that will help you. Saravanan Radhakrishnan has a published document entitled the Linux-IP-QoS-HOWTO; this document provides a good description of the way that the various QoS mechanisms have been implemented in Linux and worked examples of how to use the tc tool.

In a future column, I'll present a worked example of a QoS deployment. In the meantime, have fun exploring the features available and experiment!

Terry Dawson is the author of a number of network-related HOWTO documents for the Linux Documentation Project, a co-author of the 2nd edition of O'Reilly's Linux Network Administrators Guide, and is an active participant in a number of other Linux projects.

Read more Linux Network Administration columns.

Discuss this article in the O'Reilly Network Linux Forum.

Return to the Linux DevCenter.


Linux Online Certification

Linux/Unix System Administration Certificate Series
Linux/Unix System Administration Certificate Series — This course series targets both beginning and intermediate Linux/Unix users who want to acquire advanced system administration skills, and to back those skills up with a Certificate from the University of Illinois Office of Continuing Education.

Enroll today!

Linux Resources
  • Linux Online
  • The Linux FAQ
  • Linux Kernel Archives
  • Kernel Traffic

  • Sponsored by: