1CBQ(8)                               Linux                              CBQ(8)
2
3
4

NAME

6       CBQ - Class Based Queueing
7

SYNOPSIS

9       tc  qdisc  ...  dev  dev ( parent classid | root) [ handle major: ] cbq
10       avpkt bytes bandwidth rate [ cell bytes ] [ ewma log ] [ mpu bytes ]
11
12       tc class ... dev dev parent major:[minor] [ classid major:minor  ]  cbq
13       allot  bytes  [  bandwidth  rate ] [ rate rate ] prio priority [ weight
14       weight ] [ minburst packets ] [ maxburst packets ] [ ewma log ] [  cell
15       bytes ] avpkt bytes [ mpu bytes ] [ bounded isolated ] [ split handle &
16       defmap defmap ] [ estimator interval timeconstant ]
17
18

DESCRIPTION

20       Class Based Queueing  is  a  classful  qdisc  that  implements  a  rich
21       linksharing  hierarchy of classes. It contains shaping elements as well
22       as prioritizing capabilities. Shaping is performed using link idle time
23       calculations  based on the timing of dequeue events and underlying link
24       bandwidth.
25
26

SHAPING ALGORITHM

28       Shaping is done using link idle time calculations, and actions taken if
29       these calculations deviate from set limits.
30
31       When  shaping  a  10mbit/s connection to 1mbit/s, the link will be idle
32       90% of the time. If it isn't, it needs to be throttled so  that  it  IS
33       idle 90% of the time.
34
35       From  the kernel's perspective, this is hard to measure, so CBQ instead
36       derives the idle  time  from  the  number  of  microseconds  (in  fact,
37       jiffies)  that elapse between  requests from the device driver for more
38       data. Combined with the  knowledge of packet sizes,  this  is  used  to
39       approximate how full or empty the link is.
40
41       This is rather circumspect and doesn't always arrive at proper results.
42       For example, what is the actual link speed of an interface that is  not
43       really  able to transmit the full 100mbit/s of data, perhaps because of
44       a badly implemented driver? A  PCMCIA  network  card  will  also  never
45       achieve  100mbit/s  because of the way the bus is designed - again, how
46       do we calculate the idle time?
47
48       The physical link bandwidth may be ill defined in  case  of  not-quite-
49       real  network  devices  like PPP over Ethernet or PPTP over TCP/IP. The
50       effective bandwidth in that case is probably determined  by  the  effi‐
51       ciency of pipes to userspace - which not defined.
52
53       During operations, the effective idletime is measured using an exponen‐
54       tial weighted moving average (EWMA), which considers recent packets  to
55       be exponentially more important than past ones. The Unix loadaverage is
56       calculated in the same way.
57
58       The calculated idle time is subtracted from the EWMA measured one,  the
59       resulting  number  is  called 'avgidle'. A perfectly loaded link has an
60       avgidle of zero: packets arrive exactly at the calculated interval.
61
62       An overloaded link has a negative avgidle and if it gets too  negative,
63       CBQ throttles and is then 'overlimit'.
64
65       Conversely,  an  idle link might amass a huge avgidle, which would then
66       allow infinite bandwidths after a few  hours  of  silence.  To  prevent
67       this, avgidle is capped at maxidle.
68
69       If  overlimit, in theory, the CBQ could throttle itself for exactly the
70       amount of time that was calculated to pass between  packets,  and  then
71       pass  one  packet,  and  throttle  again.  Due to timer resolution con‐
72       straints, this may not be feasible, see the minburst parameter below.
73
74

CLASSIFICATION

76       Within the one CBQ instance many  classes  may  exist.  Each  of  these
77       classes contains another qdisc, by default tc-pfifo(8).
78
79       When enqueueing a packet, CBQ starts at the root and uses various meth‐
80       ods to determine which class should receive the data. If a  verdict  is
81       reached,  this  process is repeated for the recipient class which might
82       have further means of classifying traffic to its children, if any.
83
84       CBQ has the following methods available to classify  a  packet  to  any
85       child classes.
86
87       (i)    skb->priority  class  encoding.  Can be set from userspace by an
88              application with the SO_PRIORITY setsockopt.  The  skb->priority
89              class  encoding  only  applies  if  the  skb->priority  holds  a
90              major:minor handle of an existing class within  this qdisc.
91
92       (ii)   tc filters attached to the class.
93
94       (iii)  The defmap of a class, as set with the split  &  defmap  parame‐
95              ters.  The  defmap  may  contain  instructions for each possible
96              Linux packet priority.
97
98
99       Each class also has a level.  Leaf nodes, attached to the bottom of the
100       class hierarchy, have a level of 0.
101

CLASSIFICATION ALGORITHM

103       Classification  is a loop, which terminates when a leaf class is found.
104       At any point the loop may jump to the fallback algorithm.
105
106       The loop consists of the following steps:
107
108       (i)    If the packet is generated  locally  and  has  a  valid  classid
109              encoded within its skb->priority, choose it and terminate.
110
111
112       (ii)   Consult the tc filters, if any, attached to this child. If these
113              return a class which is not a leaf class, restart loop from  the
114              class returned.  If it is a leaf, choose it and terminate.
115
116       (iii)  If the tc filters did not return a class, but did return a clas‐
117              sid, try to find a class with that id within this qdisc.   Check
118              if  the  found class is of a lower level than the current class.
119              If so, and the returned class is not a leaf  node,  restart  the
120              loop at the found class. If it is a leaf node, terminate.  If we
121              found an upward reference to a higher level, enter the  fallback
122              algorithm.
123
124       (iv)   If  the tc filters did not return a class, nor a valid reference
125              to one, consider the minor number of the  reference  to  be  the
126              priority. Retrieve a class from the defmap of this class for the
127              priority. If this did not contain a class, consult the defmap of
128              this  class for the BEST_EFFORT class. If this is an upward ref‐
129              erence, or no BEST_EFFORT class was defined, enter the  fallback
130              algorithm.  If  a  valid  class  was found, and it is not a leaf
131              node, restart the loop at this class. If it is a leaf, choose it
132              and  terminate. If neither the priority distilled from the clas‐
133              sid, nor the BEST_EFFORT priority yielded  a  class,  enter  the
134              fallback algorithm.
135
136       The fallback algorithm resides outside of the loop and is as follows.
137
138       (i)    Consult  the  defmap  of the class at which the jump to fallback
139              occurred. If the defmap contains a class for the priority of the
140              class (which is related to the TOS field), choose this class and
141              terminate.
142
143       (ii)   Consult the map for a class for  the  BEST_EFFORT  priority.  If
144              found, choose it, and terminate.
145
146       (iii)  Choose  the  class  at which break out to the fallback algorithm
147              occurred. Terminate.
148
149       The packet is enqueued to the class which was chosen when either  algo‐
150       rithm  terminated. It is therefore possible for a packet to be enqueued
151       *not* at a leaf node, but in the middle of the hierarchy.
152
153
155       When dequeuing for sending to the network device, CBQ decides which  of
156       its  classes  will be allowed to send. It does so with a Weighted Round
157       Robin process in which each class with packets gets a chance to send in
158       turn.  The  WRR  process  starts by asking the highest priority classes
159       (lowest numerically - highest semantically) for packets, and will  con‐
160       tinue to do so until they have no more data to offer, in which case the
161       process repeats for lower priorities.
162
163       CERTAINTY ENDS HERE, ANK PLEASE HELP
164
165       Each class is not allowed to send at length  though  -  they  can  only
166       dequeue a configurable amount of data during each round.
167
168       If  a class is about to go overlimit, and it is not bounded it will try
169       to borrow avgidle from siblings that are not isolated.  This process is
170       repeated from the bottom upwards. If a class is unable to borrow enough
171       avgidle to send a packet, it is throttled and not asked  for  a  packet
172       for enough time for the avgidle to increase above zero.
173
174       I  REALLY  NEED HELP FIGURING THIS OUT. REST OF DOCUMENT IS PRETTY CER‐
175       TAIN AGAIN.
176
177

QDISC

179       The root qdisc of a CBQ class tree has the following parameters:
180
181
182       parent major:minor | root
183              This  mandatory  parameter  determines  the  place  of  the  CBQ
184              instance, either at the root of an interface or within an exist‐
185              ing class.
186
187       handle major:
188              Like all other qdiscs, the CBQ can be assigned a handle.  Should
189              consist only of a major number, followed by a colon. Optional.
190
191       avpkt bytes
192              For  calculations,  the average packet size must be known. It is
193              silently capped at a minimum of 2/3 of the interface MTU. Manda‐
194              tory.
195
196       bandwidth rate
197              To  determine the idle time, CBQ must know the bandwidth of your
198              underlying physical interface, or parent qdisc. This is a  vital
199              parameter, more about it later. Mandatory.
200
201       cell   The  cell  size determines he granularity of packet transmission
202              time calculations. Has a sensible default.
203
204       mpu    A zero sized packet may still take time to transmit. This  value
205              is  the  lower  cap  for packet transmission time calculations -
206              packets smaller than this value are still deemed  to  have  this
207              size. Defaults to zero.
208
209       ewma log
210              When  CBQ  needs  to  measure  the average idle time, it does so
211              using an Exponentially Weighted Moving Average which smooths out
212              measurements  into a moving average. The EWMA LOG determines how
213              much smoothing occurs. Defaults to 5. Lower values imply greater
214              sensitivity. Must be between 0 and 31.
215
216       A CBQ qdisc does not shape out of its own accord. It only needs to know
217       certain parameters about the underlying link. Actual shaping is done in
218       classes.
219
220

CLASSES

222       Classes have a host of parameters to configure their operation.
223
224
225       parent major:minor
226              Place  of  this class within the hierarchy. If attached directly
227              to a qdisc and not to  another  class,  minor  can  be  omitted.
228              Mandatory.
229
230       classid major:minor
231              Like  qdiscs,  classes  can  be  named. The major number must be
232              equal to the major number of the  qdisc  to  which  it  belongs.
233              Optional, but needed if this class is going to have children.
234
235       weight weight
236              When  dequeuing  to the interface, classes are tried for traffic
237              in a round-robin fashion. Classes with a higher configured qdisc
238              will  generally have more traffic to offer during each round, so
239              it makes sense to allow it to dequeue more traffic. All  weights
240              under  a  class  are  normalized,  so  only  the  ratios matter.
241              Defaults to the configured rate, unless  the  priority  of  this
242              class is maximal, in which case it is set to 1.
243
244       allot bytes
245              Allot  specifies  how many bytes a qdisc can dequeue during each
246              round of the process.  This  parameter  is  weighted  using  the
247              renormalized class weight described above.
248
249
250       priority priority
251              In  the  round-robin  process,  classes with the lowest priority
252              field are tried for packets first. Mandatory.
253
254
255       rate rate
256              Maximum rate this class and all its children combined  can  send
257              at. Mandatory.
258
259
260       bandwidth rate
261              This  is  different from the bandwidth specified when creating a
262              CBQ disc. Only used to determine maxidle and offtime, which  are
263              only  calculated when specifying maxburst or minburst. Mandatory
264              if specifying maxburst or minburst.
265
266
267       maxburst
268              This number of packets is used to calculate maxidle so that when
269              avgidle  is  at  maxidle,  this number of average packets can be
270              burst before avgidle drops to 0. Set it higher to be more toler‐
271              ant  of  bursts.  You  can't set maxidle directly, only via this
272              parameter.
273
274
275       minburst
276              As mentioned before, CBQ needs to throttle in case of overlimit.
277              The  ideal  solution is to do so for exactly the calculated idle
278              time, and pass 1 packet. However, Unix kernels generally have  a
279              hard  time  scheduling events shorter than 10ms, so it is better
280              to throttle for a longer period, and then pass minburst  packets
281              in one go, and then sleep minburst times longer.
282
283              The  time  to  wait is called the offtime. Higher values of min‐
284              burst lead to more accurate shaping in the  long  term,  but  to
285              bigger bursts at millisecond timescales.
286
287
288       minidle
289              If  avgidle is below 0, we are overlimits and need to wait until
290              avgidle will be big enough to send one packet. To prevent a sud‐
291              den  burst from shutting down the link for a prolonged period of
292              time, avgidle is reset to minidle if it gets too low.
293
294              Minidle is specified in negative microseconds, so 10 means  that
295              avgidle is capped at -10us.
296
297
298       bounded
299              Signifies  that  this  class  will not borrow bandwidth from its
300              siblings.
301
302       isolated
303              Means that this class will not borrow bandwidth to its siblings
304
305
306       split major:minor & defmap bitmap[/bitmap]
307              If consulting filters attached to a class did not  give  a  ver‐
308              dict,  CBQ  can  also  classify  based on the packet's priority.
309              There are 16 priorities available, numbered from 0 to 15.
310
311              The defmap  specifies  which  priorities  this  class  wants  to
312              receive, specified as a bitmap. The Least Significant Bit corre‐
313              sponds to priority zero. The split parameter tells CBQ at  which
314              class the decision must be made, which should be a (grand)parent
315              of the class you are adding.
316
317              As an example, 'tc class add ... classid 10:1 cbq .. split  10:0
318              defmap c0' configures class 10:0 to send packets with priorities
319              6 and 7 to 10:1.
320
321              The complimentary configuration would then be: 'tc class add ...
322              classid  10:2 cbq ... split 10:0 defmap 3f' Which would send all
323              packets 0, 1, 2, 3, 4 and 5 to 10:1.
324
325       estimator interval timeconstant
326              CBQ can measure how much bandwidth each class is using, which tc
327              filters  can use to classify packets with. In order to determine
328              the bandwidth it uses a very simple estimator that measures once
329              every  interval  microseconds  how much traffic has passed. This
330              again is a EWMA, for which the time constant can  be  specified,
331              also in microseconds. The time constant corresponds to the slug‐
332              gishness of the measurement or, conversely, to  the  sensitivity
333              of  the  average to short bursts. Higher values mean less sensi‐
334              tivity.
335
336
337
338

SOURCES

340       o      Sally Floyd and Van Jacobson, "Link-sharing and Resource Manage‐
341              ment  Models for Packet Networks", IEEE/ACM Transactions on Net‐
342              working, Vol.3, No.4, 1995
343
344
345       o      Sally Floyd, "Notes on CBQ and Guarantee Service", 1995
346
347
348       o      Sally Floyd, "Notes on  Class-Based  Queueing:  Setting  Parame‐
349              ters", 1996
350
351
352       o      Sally  Floyd and Michael Speer, "Experimental Results for Class-
353              Based Queueing", 1998, not published.
354
355
356
357

SEE ALSO

359       tc(8)
360
361

AUTHOR

363       Alexey N. Kuznetsov, <kuznet@ms2.inr.ac.ru>. This manpage maintained by
364       bert hubert <ahu@ds9a.nl>
365
366
367
368iproute2                        8 December 2001                         CBQ(8)
Impressum