Traffic Engineering: Queuing, Traffic Shaping, and Policing
Pages: 1, 2
Queuing happens only when the interface is busy. As long as the interface is idle, packets will be transmitted without special treatment. Regular queues invariably employ the first in, first out (FIFO) principle: the packet that has been waiting the longest is transmitted first. When the queue is full, and additional packets come in, tail drops happen. More sophisticated queuing mechanisms usually employ several queues. Packets are classified by user-configurable means and then placed in the appropriate queue. Then, when the interface is ready to transmit, a queue from which the next packet will be transmitted is selected as per the queuing algorithm. Cisco routers support several queuing strategies: FIFO, WFQ, RED, priority, and custom queuing. Note that special queuing mechanisms have effect only when it's not possible immediately to transmit a packet over the output interface. If the interface is idle and there are no queued packets, the new packet is transmitted immediately.
First in, first out
FIFO queuing is the most basic queuing strategy: packets are transmitted in the same order they come in. This is the default for fast interfaces. FIFO queuing is enabled by removing all other queuing mechanisms:
! interface Serial0 no fair-queue !
Weighted fair queuing
WFQ tries to allocate bandwidth fairly to different conversations (typically TCP sessions) so high-bandwidth sessions don't get to monopolize the connection. WFQ is the default for lower-bandwidth interfaces. It can be enabled with:
! interface Serial0 fair-queue !
Random early detect
RED starts to drop packets as the output queue fills up, in order to trigger congestion-avoidance in TCP. The sessions with the most traffic are most likely to experience a dropped packet, so those are the ones that slow down the most. Weighted random early detect (WRED) takes the priority value in the IP header into account and starts dropping low-priority packets earlier than their higher-priority counterparts. Unlike WFQ, priority, and custom queuing, RED doesn't need much processing time and can be used on high-speed interfaces. It needs a transmit queue bigger than the default 40-packet queue to be able to start dropping packets early and avoid tail drops.
! interface Ethernet0 random-detect hold-queue 200 out !
TIP: In RFC 2309, the IETF recommends using RED for Internet routers.
This queuing strategy allows traffic to be classified as high, normal, medium, or low priority. If there is any high-priority traffic, it's transmitted first, then medium-priority traffic, and so on. This can slow down lower-priority traffic a lot or even completely block it if there is enough higher-priority traffic to fill the entire bandwidth capacity. Example 6-18 enables priority queuing and assigns a medium (higher than normal) priority to DNS traffic and a low priority to FTP.
Example 6-18: Enabling priority queuing
! interface Serial0 priority-group 1 ! priority-list 1 protocol ip medium udp domain priority-list 1 protocol ip low tcp ftp priority-list 1 protocol ip low tcp ftp-data !
Custom queuing has a large number of queues and transmits a configurable amount of data from a queue before proceeding to the next. This queuing strategy makes it possible to guarantee a minimum amount of bandwidth for certain traffic types, while at the same time making the bandwidth that is left unused available to other traffic types. Example 6-19 assigns 75% of the bandwidth to WWW traffic, 5% to the DNS, and 20% to all other traffic.
Example 6-19: Enabling custom queuing
! interface Serial0 custom-queue-list 1 ! queue-list 1 protocol ip 1 tcp www queue-list 1 protocol ip 2 udp domain queue-list 1 default 3 queue-list 1 queue 1 byte-count 7500 queue-list 1 queue 2 byte-count 500 queue-list 1 queue 3 byte-count 2000 !
If there is more WWW traffic than can fit in 75% of the interface bandwidth, and the non-WWW/non-DNS traffic requires only 5%, the unused 15% is reallocated to WWW traffic so that no bandwidth is wasted.
Traffic Shaping and Rate Limiting
With traffic shaping, all the traffic for an interface, or just that matching a certain access list, is counted. This happens regardless of whether the interface is idle or packets are queued for transmission. When the traffic reaches a user-configurable bandwidth threshold, additional packets are put in a queue and delayed, so bandwidth use is limited to the configured amount.
Rate limiting, sometimes referred to as traffic policing or CAR, is similar to traffic shaping, but instead of being delayed, the excess traffic is treated differently from regular traffic in a user-configurable way. A common way to handle the excess traffic is simply to drop it, but it's also possible to do other things, such as lowering the priority field in the IP header. Example 6-20 enables traffic shaping for one interface and rate limiting for another.
Example 6-20: Enabling traffic shaping and rate limiting
! interface Serial0 traffic-shape rate 128000 8000 8000 1000 ! interface Serial1 rate-limit output 128000 8000 8000 conform-action transmit exceed-action drop !
Both the traffic-shape rate and the rate-limit output commands take bandwidth limit as their next argument. The other figures are burst and buffer sizes. For most applications, having those isn't desirable (TCP performance is even a bit worse when there is room for bursts), so for traffic shaping, you can leave them out; for rate limiting, you can set them to the minimum of 8000.
Traffic shaping and rate limiting are often used to limit a customer's available bandwidth when a customer buys a certain amount of bandwidth that is lower than that of the interface that connects them. This isn't a good use of rate limiting, however, because it drops a lot of packets, which makes TCP think there is congestion. So it slows down, but after a while it tries to pick up the pace again, and then there is more packet loss, and so on. Traffic shaping, on the other hand, just slows the packets down so TCP adapts to the available bandwidth. Example 6-21 shows the FTP performance over a connection that is rate-limited to 128 Kbps.
Example 6-21: FTP over a 128-Kbps rate-limited connection
ftp> put testfile local: testfile remote: testfile 150 Opening BINARY mode data connection for 'testfile'. 100% |**********************************| 373 KB 00:00 ETA 226 Transfer complete. 382332 bytes sent in 35.61 seconds (10.48 KB/s)
The TCP performance is only 84 Kbps, about two thirds of the available bandwidth. Example 6-22 is the same transfer over the same connection, but now with traffic shaping to 128 Kbps in effect.
Example 6-22: FTP a 128-Kbps traffic-shaped connection
ftp> put testfile local: testfile remote: testfile 150 Opening BINARY mode data connection for 'testfile'. 100% |**********************************| 373 KB 00:00 ETA 226 Transfer complete. 382332 bytes sent in 24.73 seconds (15.10 KB/s)
The performance is now 121 Kbps, which is just a few percent under the maximum possible bandwidth, considering TCP, IP, and datalink overhead.
Apart from combating denial-of-service attacks, as discussed in Chapter 11, rate limiting has another potential use, because unlike traffic shaping and the different queuing mechanisms, it can also be applied to incoming traffic. When an ISP and a customer agree on a certain bandwidth use, the ISP can easily use traffic shaping to make sure the customer doesn't receive more incoming traffic than the agreed upon bandwidth with traffic shaping. But since it's impossible to traffic shape packets coming in on an interface, the customer is responsible for traffic shaping their outgoing traffic. To make sure they don't send out more traffic than agreed, the ISP can implement additional rate limiting for incoming traffic.
Iljitsch van Beijnum has been working with BGP in ISP and end-user networks since 1996.
2. W. Richard Stevens' book TCP/IP Illustrated, Volume 1: The Protocols (Addison-Wesley) has an excellent description of TCP internals.
Return to ONLamp.com.