How does Bandwidth Management work in ZXTM?

Often people ask how our Bandwidth Management feature works, so that they may understand more clearly how deploying it will affect their traffic.

ZXTM's Bandwidth Management feature is built on top of Linux's Quality of Service(QoS) infrastructure. In particular it interfaces with the Hierarchical Token Bucket(HTB) and Stochastic Fair Queueing(SFQ) Queueing Disciplines. Before I talk about ZXTM's part I'll explain a little bit more about the queueing disciplines I have mentioned.

Hierarchical Token Bucket(HTB): This is a very flexible scheme for applying limits to the traffic which passes through it. It works using the token bucket principle. Traffic leaves the bucket so long as it has tokens to do so. New tokens are granted at the rate of the configured maximum rate. This scheme also has the ability to share tokens between buckets to distribute spare capacity when it exists, however ZXTM does not use this facet. The powerful thing about HTB is that it accurately limits bandwidth while still allowing small bursts in traffic.

Stochastic Fair Queueing(SFQ): One problem that arises when shaping traffic is that a single abusive stream of traffic may take more than a fair share of queueing discipline's bandwidth. This happens because the stream is filling the queueing discipline's queue faster than other streams, resulting in more of its packets in the queue than the other streams. SFQ may be employed to combat this. By passing traffic through a stochastic fair queue you can guarantee fairness between traffic streams. This is achieved by putting all the packets for a stream in a queue of its own then taking packets from each queue in a round-robin fashion. It is unreasonable that every stream has its own queue, as this would lead to excessive memory use by the kernel, so there is by default a limit of 1024 queues into which the traffic is placed. A hash function which uses the source and destination IP address of a stream is used to determine which bucket is to be used for that stream. Also periodically the hash function is changed so that in the worst case where two streams are interfering with each other they are moved to separate buckets.

So what is ZXTM's role in all this? Well, every time you create a bandwidth class in the admin server this creates a corresponding HTB queueing discipline in the kernel. In addition it also creates a corresponding SFQ queueing discipline. These are then chained together so that traffic that is assigned to the SFQ then flows into the HTB. This way we can guarantee fairness between connections which have been assigned to a particular bandwidth class along with providing the configured limit.

ZXTM is very flexible when it comes to how it assigns traffic to a bandwidth class. Connections to clients may be limited using a Bandwidth class assigned to a Virtual Server or using TrafficScript via the response.setBandwidthClass() function. Similarly you can limit connections to backend nodes by applying a Bandwidth class to a Pool or using TrafficScript via the request.setBandwidthClass() function. When using TrafficScript to apply Bandwidth classes you may alter the class a connection is assigned to at any time and the effect should be immediate. This is particularly useful in protocols which consist of many requests and responses on the same connection, but that the different types of request or response have different bandwidth requirements.

One final thing to mention is that the configured limits of a Bandwidth class are honoured across a cluster. Should you have multiple active machines then the total traffic for all the connections assigned to a Bandwidth class over all the machines will still adhere to the configured limit of the class. So you don't have to worry about altering your limits depending on the number of active ZXTMs in your cluster.

Dec [Zeus Dev Team] 29 June 2006   1 comment