User Tools

Site Tools


doc:howto:packet.scheduler:packet.scheduler

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
doc:howto:packet.scheduler:packet.scheduler [2012/09/12 15:01]
valentt
doc:howto:packet.scheduler:packet.scheduler [2014/09/01 10:41] (current)
hamy
Line 1: Line 1:
-====== ​The Linux Packet Scheduler ====== +====== ​Network ​Traffic Control ​====== 
- +Traffic Control is the umbrella term for packet prioritizing,​ traffic ​[[#shaping|shaping]], bandwidth limiting, AQM (Active Queue Management),​ etc. This HowTo will help you understand and set up traffic control ​on your routerIs it one strategy ​to address ​problems ​caused ​by [[wp>Network congestion]].
-===== An Introduction to Traffic Control ​/ Traffic Shaping / Traffic Prioritizing / QoS / Bandwidth Management ​===== +
-Traffic Control is the umbrella term for packet prioritizing,​ traffic shaping, bandwidth limiting, AQM (Active Queue Management),​ etc. This howto shall help you set up such on your GNU/Linux systemThe only OpenWrt specific thing about this, is that OpenWrt splits much stuff into small ''​[[doc:​techref:​opkg]]''​-packages to save on storage and memory. +
- +
-On other Linux distributions the tools presented here are all together in one software package called //​iproute2//​. The official (but out-of-date) description of iproute2 can be found at [[http://​lartc.org/​howto|LARTC (Linux Advanced Routing & Traffic Control)]]. +
- +
- +
-| {{:​meta:​icons:​tango:​dialog-information.png?​nolink}} | 1. You //can// control, i.e. prioritize and/or shape, ANY **upload traffic**, i.e. traffic being send from your router ​to the Internet. Doing so //will// solve problems ​with congestion or [[wp>​http://​en.wikipedia.org/​wiki/​Jitter#​Packet_jitter_in_computer_networks|Jitter]] or dealy of packets | +
-| {{:​meta:​icons:​tango:​dialog-information.png?​nolink}} | 2. By pure logic you cannot achieve the same, for your **download traffic**. However, you can POLICE the download traffic. By policing the download, you can still achieve noticeable improvements with incomming traffic as long as UDP traffic does not flood the DOWNload line  | +
-| {{:​meta:​icons:​tango:​48px-dialog-error-round.svg.png?​nolink}} | [[wp>​Transmission Control Protocol|TCP (Transmission Control Protocol)]] is a protocol that allows the adaptation of the sending rate to the actual charge state of the network. This is achieved ​by droping or delaying ACK-Packets.\\ ​[[wp>User Datagram Protocol|UDP (User Datagram Protocol)]] does not have ACK-packets and thus cannot adaptSo if UDP packets congest your DOWNLOAD bandwidth, there is NO scheduler in the world, that could help with that.  | +
- +
-Also wurde ein Protokoll geschaffen, das die Sende­rate an den aktuellen Lastzustand im Netz an­pas­sen kann. Das heutige Transmission Control Protocol (TCP) war geboren. +
- +
- +
- +
-| {{:​meta:​icons:​tango:​48px-outdated.svg.png?​nolink}} | In [[https://​dev.openwrt.org/​changeset/​31760/​trunk|R31760]] ''​kmod-sched''​ was splitted into ''​kmod-sched-core''​ and ''​kmod-sched''​ | +
- +
-==== Why Traffic Control? ==== +
-Let's have a look at the **upload** traffic: +
- +
-^ without tc ^ with tc  ^ +
-| bild1      | bild2    | +
- +
- +
-==== "​TCP-Turbo"​ ==== +
-When you prioritize ACK-Packets on the upload side, your download TCP-traffic //will// benefit in certain situations (a marketing term for this is "​TCP-Turbo"​):​ +
- +
-^ without tc ^ with tc  ^ +
-| {{:​doc:​howto:​packet.scheduler:​tcp-traffic_no_traffic_control.gif|without traffic control}} | {{:​doc:​howto:​packet.scheduler:​tcp-traffic_with_traffic_control.gif.gif|with traffic control}} | +
- +
- +
-**''​NOTE:''​** Download UDP traffic remains unchanged by this! +
- +
- +
-==== Traffic Policing ==== +
-Under ideal circumstances everyone could upload a traffic control configuration to his  [[wp>​Broadband Remote Access Server]]/ Access Concentrator and achieve the same for the download-direction as for the upload. But in reality nobody has something like this, and the queue on the BRAS is probably huge. To avoid this, avoid the accumulation of packets in the TX-Queue of the BRAS, you can enable policing on your OpenWrt-Router for the download. Its not much, but probably better then doing nothing! +
  
 +| {{:​meta:​icons:​tango:​dialog-information.png?​nolink}} | You //can// control, i.e. prioritize and/or shape, ANY **upload traffic**, i.e. traffic being sent from your router to the Internet. Doing so //will// solve problems that occur with congestion, i.e. [[wp>​Jitter#​Packet_jitter_in_computer_networks|jitter]] and delay. |
 +| {{:​meta:​icons:​tango:​dialog-information.png?​nolink}} | You do NOT have the same level of control over **download traffic**, i.e. traffic arriving at your router from the Internet. Here, you can only drop packets but not rearrange them.\\ The dropping of TCP packets, causes the sending site to reduce its transmission rate, the dropping of UDP packets however, will only help to keep the buffer empty. |
  
 ===== Preparations ===== ===== Preparations =====
-==== Prerequisites ==== +  ​Read about [[packet.scheduler.theory|Linux packet scheduling]]  
-  ​read about the [[packet.scheduler.theory|working principle]] of the Linux packet scheduler; this is really very simple +  - Learn about the''​[[man>​tc|tc command]]''​ 
-  - understand ​the available configuration parameters and their effect of the queueing algorithm you want to employ; this can get quite complicated +  - Determine ​the characteristics ​of the connection you are configuring the packet scheduler for: 
-  - learn the utilization of ''​[[man>​tc]]''​; again, very simple and straight forward +    * whether it is a full-duplex or a half-duplex line [[wp>duplex]] 
-  - muster ​the patience to adjust your configuration. For the packet scheduler to work effectively,​ your configuration parameters must be tailored to your actual available bandwidth and your usage scenario. A minimum viable script could be created in no more then half an hour, but most users will spend a couple ​of hours of tinkering and reading to understand ​the far-reaching possibilities and adapting the configuration to their needs and desires. +    * the actual ​available upload bandwidth! The Linux packet scheduler works on Layer 2, thus you should always work with the actual bandwidth for the Layer-2-Payload:​ 
-  - You have to know the parameters of the line your are configuring the packet scheduler for +    * ie: when you employ the Layer 1 protocol "​BASE100-TX":​ 
-    * whether it is a full-duplex or a half-duplex line (cf. [[wp>Duplex (telecommunications)]] +      * you have 100MBit/s of theoretically available bandwidth 
-    * the accurate (=true and precise) ​available upload bandwidth!!! +      * Physical factors such as interference, substandard cabling ​or faulty ​hardware can reduce that bandwidth. 
-    * <color red>**The Linux packet scheduler works on Layer 2, thus you should always work with the actual bandwidth for the Layer-2-Payload:​**</​color>​ +      * The Layer 2 protocol ​you use adds [[doc/​networking/​datagram.structures|protocol overhead]]. ​In the case of "​Ethernet"​, it adds about 2,5%, so a maximum of 97,5MBit/s remain for the Layer-2-payload. 
-    * e.g. when you employ the Layer 1 protocol "​BASE100-TX":​ +      * Download testing sites usually refer to Layer 4 bandwidthwhich includes ​Layer 3 and 4 protocol overhead
-      * you have 100MBit/s of theoretically available ​physical ​bandwidth +  Be prepared ​to adjust your configurationIt can take some time and experimentation to find what works best for your circumstances. 
-      * but due to interference ​or cabeling not adhering to specifications ​or faulty ​IC or whatever, you could have less then 100MBit/s of real/actual physical ​bandwidth! +    ​
-      * on top of the layer 1 protocol, you will utilize a Layer 2 protocol, this adds [[doc/​networking/​datagram.structures|protocol overhead]]. ​The Layer 2 protocol ​"​Ethernet"​ adds about 2,5%, so a maximum of 97,5MBit/s remain for the Layer-2-payload. +
-      * on top of the layer 2 protocol, you will utilze at least one Layer 3 protocollet's say IPv4. Due to protocol overhead, the Layer-3-Payload is smaller than the Layer-2-Payload. Ignore the Layer-3-Payload ​and always work with Layer2. If an application shows the used bandwidth or allows you to throttle the bandwidth it sends or receives data, it probably refers to the Layer-4-PayloadOr not. I have no idea. Just avoid mixing Layer-2-bandwidth with Layer-1 or Layer-3 bandwidth ;-) +
-    * e.g. when you buy VDSL 50000/​10000,​ what do the values refer to? 10MBit/s physical bandwidth or Layer-2 bandwidth? However, you still have to measure the available one ;-) +
-    * e.g. you employ IEEE 802.11n-hardware with 300MBit/s theoretical physical bandwidth. You have to measure the real physical bandwidth, which will most probably vary over time and then substract the protocol overhead and work with that value.+
  
 ==== Required Packages ==== ==== Required Packages ====
-  ***''​tc''​** (''​traffic control'', ​user space program to configure the Linux packet scheduler) +  ***''​tc''​** (''​traffic control'',​ program to configure the Linux packet scheduler) 
-    ***''​kmod-sched-core''​** (dependency of //tc//), package containing most used or advanced ​schedulers ​(aka queueing disciplines aka QDiscs) and classifiers +    ***''​kmod-sched-core''​** (dependency of //tc//), package containing most used or advanced (QDiscs) and classifiers 
-    ***''​kmod-sched''​** (optional), package containing other schedulers and classifiers (those not included in the previous)+    ***''​kmod-sched''​** (optional), package containing other schedulers and [[#​classifier|classifiers]] (those not included in the previous)
   ***''​iptables-mod-ipopt''​** optional! Contains some matches and TARGETs for iptables: ​ CLASSIFY, dscp/DSCP, ecn/ECN, length, mac, mark/MARK, statistic, tcpmms, tos/TOS, ttl/TTL, unclean   ***''​iptables-mod-ipopt''​** optional! Contains some matches and TARGETs for iptables: ​ CLASSIFY, dscp/DSCP, ecn/ECN, length, mac, mark/MARK, statistic, tcpmms, tos/TOS, ttl/TTL, unclean
-    ***''​kmod-ipt-ipopt''​** (user space module; dependency of corresponding user space module; +    ***''​kmod-ipt-ipopt''​** (module; dependency of corresponding user space module; 
-  ***''​iptables-mod-*''​** optional! (user space modules for iptables) +  ***''​iptables-mod-*''​** optional! (modules for iptables) 
-    ***''​kmod-ipt-*''​** (kernel ​space modules for iptables)+    ***''​kmod-ipt-*''​** (kernel modules for iptables)
   ***''​l7-protocols''​** optional! If you want to match Layer 7 content   ***''​l7-protocols''​** optional! If you want to match Layer 7 content
   ***''​l7-protocols-testing''​** optional! If you want to test. Check the projects own [[http://​l7-filter.clearfoundation.com/​docs/​start|Homepage]].   ***''​l7-protocols-testing''​** optional! If you want to test. Check the projects own [[http://​l7-filter.clearfoundation.com/​docs/​start|Homepage]].
  
  
-As long as your ISP does not give you access to the DSL-AC so you can install a simple TC-script, you will have to settle with policing the download:+As long as your ISP does not give you access to the DSL-AC so you can install a simple TC-script, you will have to settle with [[#policing|policing]] ​the download:
   * ''​kmod-ifb''​ and act_connmark   * ''​kmod-ifb''​ and act_connmark
  
Line 92: Line 54:
 </​code>​ </​code>​
  
-After thoroughly reading this wiki page, you are going to write a shell script, you are going to write a shell script, ​which will invoke ''​tc''​ a couple of times and configure the packet scheduler. Please also see the available [[#​examples]]. When everything works good enough, proceed with [[#Start on boot]] and [[#​Hotplug]].+After thoroughly reading this wiki page, you are going to write a shell script which will invoke ''​tc''​ a couple of times and configure the packet scheduler. Please also see the available [[#​examples]]. When everything works good enough, proceed with [[#Start on boot]] and [[#​Hotplug]]. 
  
  
-===== Explanation ===== 
-->​[[packet.scheduler.theory]] 
  
 ===== Configuration ===== ===== Configuration =====
-==== Part 1 – Hierarchy: Nesting of qdiscs & classes ==== +==== Hierarchy: Nesting of qdiscs & classes ==== 
-There are two types of scheduling algorithms (QDiscs), classfull ​and classless. If you choose to employ a classfull ​root QDisc, you will be able to tailor the configuration very closely to your needs, by constructing a hierarchy of "​nesting entities"​ and then further tune each branch of the tree separately.+There are two types of scheduling algorithms ​[[#​queueing.discipline.qdisc|(QDiscs)]] - [[#​classful.QDisc|classful]] ​and [[#classless.QDisc|classless]]. If you choose to employ a classful [[#​root.qdisc|root QDisc]], you will be able to tailor the configuration very closely to your needs, by constructing a hierarchy of "​nesting entities"​ and then further tune each branch of the tree separately.
  
- ''​Tc'' ​it the one and only user space program available to set up, maintain and inspect the configuration of the Linux packet scheduler. What ''​iptables'',​ ''​ip6tables''​ are for netfilter, ''​tc''​ is for the Linux packet scheduler. Generally only one change is made to the packet scheduler each time ''​tc''​ is executed. A small shell script containing multiple invocations of ''​tc'' ​are required to achieve a meaningful overall configuration:+ ''​tc'' ​is the only user space program available to set up, maintain and inspect the configuration of the Linux packet scheduler. What ''​iptables'',​ ''​ip6tables''​ are for netfilter, ''​tc''​ is for the Linux packet scheduler. Generally only one change is made to the packet scheduler each time ''​tc''​ is executed. A small shell script containing multiple invocations of ''​tc'' ​is required to achieve a meaningful overall configuration.
  
 ^         ​^^^^ ​ nesting ​ ^^  configuration ​ ^^ ^         ​^^^^ ​ nesting ​ ^^  configuration ​ ^^
Line 114: Line 75:
 | :::     | :::        | ''​replace''​ | ''​dev''​ eth0 | ''​parent''​ 1:1 | ''​classid''​ 1:20 |  ''​hfsc'' ​ | ls rate 250kbit ul rate 1000kbit ​ | | :::     | :::        | ''​replace''​ | ''​dev''​ eth0 | ''​parent''​ 1:1 | ''​classid''​ 1:20 |  ''​hfsc'' ​ | ls rate 250kbit ul rate 1000kbit ​ |
  
-**''​Question No1:''​** Why are there two types of nesting entities? What the fuck is the difference between a qdisc and a class??? Wouldn'​t it be enough to have qdiscs only and be done with it?\\ +==== Qdiscs ​(Packet Scheduling ​Algorithms) ==== 
-**''​Answer:''​** Bear in mind that we have the packet scheduler, which is a component like netfilter, and then we have specific scheduling algorithms working within the packet scheduler. These //​scheduling algorithms//​ are usually called queueing disciplines or QDiscs. And in case you decide to employ a classful QDisc (scheduling algorithm), you have two //nesting entities// with slightly different traits: qdiscs and classes. Don't confuse the nesting entity qdisc from "​qdiscs and classes"​ with QDisc aka Algorithm! ;-) To distinguish between the two types, I usually write qdisc and QDisc, but we should rather call the one scheduling algorithm and the other one qdisc. As long as you do not intend to use a classfull QDisc, you do not have to bother with the difference between qdiscs and classes!\\ +Once you decide ​what your entire configuration will look like, look up the //specific configuration of the QDisc algorithm// you intend to use. Each Qdisc aka Scheduling Algorithm gives you parameters to tune:
- +
-**''​Question No2:''​** But I want to understand the difference.\\ +
-**''​Answer:''​** Well, if you really insist, try reading [[packet.scheduler.theory#​classfull scheduling algorithm]]. This should make the necessity to distinguish between two types of nesting entities "​qdiscs"​ and "​classes"​ clear to you.\\ +
- +
-  * A chosen [[packet.scheduler.example1#​Implementation of classfull with 2 levels|implementation]] that matches your situation best. Alone, use neither. Use classless. +
- +
-==== Part 2 – Qdisc (Packet Scheduling ​Algorithm) ==== +
-Once you decide ​how your entire configuration will look like, you have to look up the //specific configuration of the QDisc algorithm// you intend to use. Each Qdisc aka Scheduling Algorithm gives you parameters to tune:+
  
 ^ Queueing Discipline ​ ^^  Classfull ​ ^ Description ​ ^  kmod-sched-core ​ ^  kmod-sched ^ ^ Queueing Discipline ​ ^^  Classfull ​ ^ Description ​ ^  kmod-sched-core ​ ^  kmod-sched ^
-| -> [[sch_atm]]    ​|  name                                 ​| ​ ??  | bla  |  |  | +| ->   ​sch_atm ​     |  name                                 ​| ​ ??  | bla  |  |  | 
-| -> [[sch_blackhole]]  ​|  Black hole queue                 ​| ​ ??  | bla  |  |  |+| ->   ​sch_blackhole| ​ Black hole queue                 ​| ​ ??  | bla  |  |  |
 | -> [[sch_cbq]] ​   |  Class-Based Queueing discipline ​     |  ☑  | very complex ​ |  |  | | -> [[sch_cbq]] ​   |  Class-Based Queueing discipline ​     |  ☑  | very complex ​ |  |  |
 | -> [[sch_choke]] ​ |  CHOKe scheduler ​                     |  ??  | bla  |  |  | | -> [[sch_choke]] ​ |  CHOKe scheduler ​                     |  ??  | bla  |  |  |
 | -> [[sch_codel]] ​ |  The Controlled-Delay Active Queue Management ​  ​| ​ ??  | available since [[https://​dev.openwrt.org/​changeset/​31756|r31756]] and [[https://​dev.openwrt.org/​changeset/​31757|r31757]],​ mainlined in Kernel 3.5  |  ☑  |  | | -> [[sch_codel]] ​ |  The Controlled-Delay Active Queue Management ​  ​| ​ ??  | available since [[https://​dev.openwrt.org/​changeset/​31756|r31756]] and [[https://​dev.openwrt.org/​changeset/​31757|r31757]],​ mainlined in Kernel 3.5  |  ☑  |  |
-| -> [[sch_drr]] ​   |  Deficit Round Robin scheduler ​       |  ??  | bla  ​|  |  | +| -> [[sch_drr]] ​   |  Deficit Round Robin scheduler ​       |  ??  | can handle packets of variable size without knowing their mean size.   |  |  | 
-| -> [[sch_dsmark]] |  Differentiated Services field marker ​ |  ??  | bla  |  |  ☑  |+| ->   ​sch_dsmark ​  ​|  Differentiated Services field marker ​ |  ??  | bla  |  |  ☑  |
 | -> [[sch_esfq]] ​  ​| ​ Enhanced Stochastic Fairness Queueing ​ |  ??  | removed in mainline kernel, but still available in OpenWrt ​ |  |  ☑  | | -> [[sch_esfq]] ​  ​| ​ Enhanced Stochastic Fairness Queueing ​ |  ??  | removed in mainline kernel, but still available in OpenWrt ​ |  |  ☑  |
 | -> [[sch_fifo]] ​  ​| ​ The simplest FIFO queue                |  ??  | bla  |  |  ☑  | | -> [[sch_fifo]] ​  ​| ​ The simplest FIFO queue                |  ??  | bla  |  |  ☑  |
Line 151: Line 104:
 | -> [[sch_sfb]] ​   |  Stochastic Fair Blue                 ​| ​ ??  | bla  |  |  | | -> [[sch_sfb]] ​   |  Stochastic Fair Blue                 ​| ​ ??  | bla  |  |  |
 | -> [[sch_sfq]] ​   |  Stochastic Fairness Queueing ​        ​| ​ ☒  | distibutes bandwidth for known tcp-connections fairly ​ |  |  ☑  | | -> [[sch_sfq]] ​   |  Stochastic Fairness Queueing ​        ​| ​ ☒  | distibutes bandwidth for known tcp-connections fairly ​ |  |  ☑  |
 +| -> [[sch_sfqred]] |  mixture of qfq and red               ​| ​ ?  |       | | |
 | -> [[sch_tbf]] ​   |  Token Bucket Filter ​                 |  ☒  | limit bandwidth ​ |  |  ☑  | | -> [[sch_tbf]] ​   |  Token Bucket Filter ​                 |  ☒  | limit bandwidth ​ |  |  ☑  |
 | -> [[sch_teql]] ​  ​| ​ True/​Trivial Link Equalizer ​         |  ??  | bla  |  |  ☑  | | -> [[sch_teql]] ​  ​| ​ True/​Trivial Link Equalizer ​         |  ??  | bla  |  |  ☑  |
Line 156: Line 110:
 **''​Note:''​** The PRIO QDisc does contain three classes, but since they cannot be configured further, PRIO is considered to be a classless QDisc. Its classes are sometimes called bands. **''​Note:''​** The PRIO QDisc does contain three classes, but since they cannot be configured further, PRIO is considered to be a classless QDisc. Its classes are sometimes called bands.
  
-==== Part 3 – Filters ​==== +==== Actions ​====
-This is where you configure which network packet belongs to which queue/​bucket. A rule used to allocate a group of IP packets to a certain classid consists of a number of classifiers (match) and one connected action (TARGET or VERDICT). In principle it works exactly like [[doc:​howto:​netfilter#​configuration|netfilter rules]], the only difference is that <color blue>​matches</​color>​ are called <color blue>​classifiers</​color>​ and the <color red>​TARGET</​color>​ are called <color red>​VERDICT</​color>​ in available documentation. However, since it is possible to do the filtering <​del>​entirely</​del>​ with netfilter (almost, doesn'​t forget l2 packets like arp), this does not really matter.+
  
-^ Filter ​ ^^  Description ​ ^  kmod-sched-core ​ ^  kmod-sched ^+^ Action ​ ^^  Description ​ ^  kmod-sched-core ​ ^  kmod-sched ^ 
 +| [[act_police]] ​ | Input police filter ​       |  |  | 
 +| [[act_nat]] ​    | Stateless NAT actions ​     | | | 
 +| [[act_mirred]] ​ | packet mirroring and redirect actions | | | 
 +| [[act_skbedit]] | | | | 
 + 
 + 
 +==== Filters ==== 
 +This is where you configure which network packet belongs to which queue/​bucket. A rule used to allocate a group of IP packets to a certain classid consists of a number of classifiers (match) and one connected action (TARGET or VERDICT).  
 + 
 +  * In principle it works exactly like [[doc:​howto:​netfilter#​configuration|netfilter rules]], the only difference is that <color blue>​matches</​color>​ are called <color blue>​classifiers</​color>​ and the <color red>​TARGET</​color>​ are called <color red>​VERDICT</​color>​ in available documentation. However, since it is possible to do the filtering <​del>​entirely</​del>​ with netfilter (almost, doesn'​t forget Layer 2 packets like arp), this does not really matter. 
 + 
 +^ Filter ​(Classifier) ​ ​^^ ​ Description ​ ^  kmod-sched-core ​ ^  kmod-sched ^
 | -> [[http://​git.kernel.org/?​p=linux/​kernel/​git/​torvalds/​linux.git;​a=commit;​h=e5dfb815181fcb186d6080ac3a091eadff2d98fe|cls_flow]] ​   | flow classifier ​ | bla  |  ☑  |  | | -> [[http://​git.kernel.org/?​p=linux/​kernel/​git/​torvalds/​linux.git;​a=commit;​h=e5dfb815181fcb186d6080ac3a091eadff2d98fe|cls_flow]] ​   | flow classifier ​ | bla  |  ☑  |  |
 | -> [[cls_fw]] ​   | firewall classifier ​ | bla  |  ☑  |  | | -> [[cls_fw]] ​   | firewall classifier ​ | bla  |  ☑  |  |
Line 166: Line 131:
 | -> [[cls_u32]] ​   | u32 classifier ​ | bla  |  ☑  |  | | -> [[cls_u32]] ​   | u32 classifier ​ | bla  |  ☑  |  |
 | -> [[cls_basic]] ​   | basic classifier ​ | bla  |  |  ☑  | | -> [[cls_basic]] ​   | basic classifier ​ | bla  |  |  ☑  |
 +| -> [[cls_cgroup]] | [[wp>​cgroups]] (Control Group) Classifier |   ​| ​  ​| ​
  
 === Filter with packet scheduler === === Filter with packet scheduler ===
-A filter is used by a classfull QDisc to determine in which bucket a packet will be enqueued. Whenever traffic arrives at a class with subclasses, it needs to be classified. Various methods may be employed to do so, one of these are the filters. All filters attached to the class are calleduntil one of them returns with a verdict. If no verdict ​was made, other criteria may be available. This differs per qdisc.+A filter is used by a classfull QDisc to determine in which bucket a packet will be enqueued. Whenever traffic arrives at a class with subclasses, it needs to be classified. Various methods may be used, one of which is filters. All filters attached to the class are called until one of them returns with a verdict. If no verdict ​is declared, other criteria may be considered. This behaviour varies between different QDiscs.
  
 ^                   ​^^^ ​ location ​ ^^ ^  match  ^^^  verdict/​target ​ ^ ^                   ​^^^ ​ location ​ ^^ ^  match  ^^^  verdict/​target ​ ^
Line 209: Line 175:
 **''​Note:''​** You may read on the internet that you can use target CLASSIFY only on POSTROUTING,​ but it's not true since at least 2006, you can also use it on FORWARD and OUTPUT. From kernel 2.6.39, you are no longer restricted to the mangle table, and can classify with arptables (on OUTPUT and FORWARD)(http://​comments.gmane.org/​gmane.comp.security.firewalls.netfilter.devel/​36340). **''​Note:''​** You may read on the internet that you can use target CLASSIFY only on POSTROUTING,​ but it's not true since at least 2006, you can also use it on FORWARD and OUTPUT. From kernel 2.6.39, you are no longer restricted to the mangle table, and can classify with arptables (on OUTPUT and FORWARD)(http://​comments.gmane.org/​gmane.comp.security.firewalls.netfilter.devel/​36340).
  
-===== Approach to an own configuration ===== +===== Approach to our own configuration ===== 
-The configuration of the packet scheduler has to be tailored to your situation. Bear in mind what you are actually doing here: you adjust ​the behavior of the packet scheduler ​working ​the egress ​buffer ​of one certain ''​physical software ​interface''​! Depending on the employed algorithm, we can: +The configuration of the packet scheduler has to be tailored to your situation. Bear in mind what you are actually doing - controlling ​the behavior of the packet scheduler ​managing ​the egress ​queue of a network ​interface.
-  - manipulate the order in which the packets, that currently are in the egress buffer are being sent (=re-order/​prioritize) +
-  - subdivide the buffer into sub-buffers,​ and then drop packets willfully, when they fill their sub-buffer (=traffic shaping)+
  
-This is helpful in certain situations+We can
-  ​* **traffic prioritization** helps with problems with [[wp>​jitter]] that occur when there are buffers and these get clogged +  ​- Manipulate the order of packets leaving the egress queue (re-order/​prioritize) which reduces ​[[wp>​jitter]] that occurs with congestion. 
-  ​* **traffic shaping** helps with dividing the available bandwidth between ​defined ​traffic types and/​or ​participants willfully+  ​- Divide the queue into sub-queues, and then drop packets when they are full (traffic shaping) which shares ​available bandwidth between traffic types and/​or ​users.
  
 So, let's check your situation and let's then configure your packet scheduler accordingly:​ So, let's check your situation and let's then configure your packet scheduler accordingly:​
-  - do you require traffic control? +  - Do you require traffic control? 
-    ​IF you generate more traffic then can go through the line, THEN simply stop this foolish behavior. Stop generating more traffic then your upload bandwidth. There is no point in generating more traffic than there is available upload bandwidthThis will only clog you egress buffer and cause serious problems with jitter. +    ​IF you generate more outgoing ​traffic than available upload bandwidth, STOP This will only clog your egress buffer and cause serious problems with jitter. 
-    - IF your aunt Margarette ​is responsible for the excess traffic, and you cannot ​influence ​this, THEN yes +    - IF your roommate ​is responsible for the excess traffic, and you cannot ​change ​this, THEN yes 
-    - IF you do not generate more traffic then can go through the line, but still have problems with Jitter, THEN you could profit ​from traffic prioritization+    - IF you do not generate more traffic then can go through the line, but have problems with jitter then you could benefit ​from traffic prioritization.
  
   - What can you configure?   - What can you configure?
Line 236: Line 200:
  
 ===== Examples ===== ===== Examples =====
-In my eyes google finds far to few complete examples for implementations. Let us try to keep them as modular and simple as possible: (use [[wp>​Vim]] for a better syntax highlighting) 
- 
   *[[packet.scheduler.example1]] PRIO; one user, simple prioritizing   *[[packet.scheduler.example1]] PRIO; one user, simple prioritizing
   *[[packet.scheduler.example2]] HTB;  plain simple bandwidth sharing without concern for delay   *[[packet.scheduler.example2]] HTB;  plain simple bandwidth sharing without concern for delay
Line 243: Line 205:
   *[[packet.scheduler.example4]] HFSC + FQ_CODEL + FLOW classifier; basic fair sharing behind triple play box    *[[packet.scheduler.example4]] HFSC + FQ_CODEL + FLOW classifier; basic fair sharing behind triple play box 
  
-**''​Note:''​** The above examples do not make any use of UCI or anthing else, that is OpenWrt-specific,​ so you can simply port them to any other Linux distributions and back.+**''​Note:''​** The above examples do not make any use of UCI or anthing else, that is not OpenWrt-specific,​ so you can simply port them to any other Linux distributions and back.
  
 ==== Check results ==== ==== Check results ====
Line 265: Line 227:
  
 ===== Testing ===== ===== Testing =====
-Once you managed to set up a working configuration,​ you need to test it. Thoroughly. Produce all kind of outgoing traffic ​and measure the bandwidth distribution. Then, measure the packet delay. An ideal set up would be full access and control over the three: //Source// -> line1 -> //Router// -> line2 -> //​Destination/​/. The whole effort is needed because the source(s) send more traffic then line2 can handle. To simulate a limited dsl-line over an Ethernet connection do... :!: TODO :!: But even without this set up, you can do measurements:​+Once you managed to set up a working configuration,​ you need to test it. Thoroughly! Failure to do so could produce unpredictable results ​and/or problems
  
-To measure and compare our results, the approach is always the same. We measure the latency of network packets in different situations:​ +Ideally ​you should set up your own mini-network that allows ​you to monitor and control ​the sourcedestination and traffic in between. ​
-  - on free line without any QoS; the technical feasible values; less is technically impossible +
-  - on congested line without QoS; you will have huge and also unsteady delays. (This is very reason you want to configure ​your egress buffer.) +
-  ​on free line with any QoS; you will add some delay, there is no way around this; the lesser the better +
-  - on congested line with QoS; the fruits of all your work: more delay compared to a free linebut not much; the lesser the better+
  
-Googleling is time consuming ​and sometimes fruitlessIf you got a good and working result //please// come back and **share you knowledge**!+Produce all kinds of outgoing traffic ​and measure the bandwidth distribution
  
-If you have a working statistic thingit will be of invaluable help with this.+Then measure and compare latency in different situations:​ 
 +  - with minimal trafficwithout QoS 
 +  - with heavy traffic and congestion, without QoS 
 +  - with minimal traffic, with QoS 
 +  - with heavy traffic and congestion, with QoS 
 + 
 + 
 +If you successful //please// **share your knowledge**!
  
 ===== Start on boot ===== ===== Start on boot =====
Line 303: Line 268:
  
 ===== Hotplug ===== ===== Hotplug =====
-For example the virtual network interface ''​pppoe-dsl''​ has a special behavior. ​If you reconnect ​your dsl-connection,​ the device ''​pppoe-dsl''​ will be closed down, thus it will "seize to exist"​. And so will its QDisc. So every time, it is been brought up again, the whole configuration will need to be set up again. A way to achieve that is described here.+If you disconnect ​your dsl-connection,​ the device ''​pppoe-dsl''​ will close and so will its QDisc. ​
  
-Make hotplug restart your script ​every time the interface the packet scheduler belongs to comes up again:+The following ​script ​will restart it whenever you reconnect:
  
 <code bash> <code bash>
 vi /​etc/​hotplug.d/​iface/​30-trafficc vi /​etc/​hotplug.d/​iface/​30-trafficc
-</​code>​+
  
 <file 30-trafficc>​ <file 30-trafficc>​
Line 318: Line 283:
 # interface is available as INTERFACE, the real device as DEVICE. # interface is available as INTERFACE, the real device as DEVICE.
  
-[ "​$ACTION"​ = ifup -a "​$INTERFACE"​ = "​dsl"​ ] && /​etc/​init.d/​trafficc enabled && /​etc/​tc_hfsc.sh+[ "​$ACTION"​ = ifup -a "​$INTERFACE"​ = "​dsl"​ ]  
 +&& /​etc/​init.d/​trafficc enabled && /​etc/​tc_hfsc.sh
 </​file>​ </​file>​
 +</​code>​
  
 ===== Statistical Data ===== ===== Statistical Data =====
Line 332: Line 299:
  
  
-===== Troubleshooting ​===== +===== Tips =====
-  * The packet scheduler comes after netfilter and it is possible to block traffic, if you configure it badly. To avoid this, the default behavior of the packet scheduler should be to have some bandwidth available for all traffic that has no specific configuration! In contrary to the packet filter, where the default behavior should be DROP.+
  
-  * A common mistake can be forgetting to classify ARP packets (even if you match all packets ​in iptables, ​you won't match ARP packet, as iptables is layer 3 and ARP is layer 2)+  * Leave some bandwidth available ​in your packet scheduler configuration for unspecified traffic. This will help avoid blocking important low-volume traffic ​you may have forgetten to consider.
  
-  * If you're adding ​you qdisc on a "​virtual"​ dev (vlan (eth0.1)bridge (br-wan)) ​you may have dropped ​packet, ​it's because for these type of device the default txqueuelen ​is and the qdisc inherit this value. Simply set an higher value: "​ifconfig br-wan txqueuelen X" (the value of x depends on the speed of your link, you can try with 1 or txqueuelen of the real dev (eth0)).+  * Don't forget to classify ARP packets! (even if you match all packets in iptables, you won't match ARP packet, ​as iptables ​is layer 3 and ARP is layer 2)
  
-===== Notes ===== +  * If you use any virtual interfaces, don't forget to configure the queue size using the txqueuelan traffic. The default is 0.  
-  * http://​lartc.org/ + 
-  * http://​lartc.org/​howto/ ​ GOTO => 9. Queueing Disciplines for Bandwidth Management +===== Terminology ​===== 
-  * http://​lartc.org/​manpages/ +**''​Queueing Discipline (QDisc)''​** An algorithm that manages the queue of a device, either incoming (ingress) or outgoing (egress). Also referred to as a packet scheduler. 
-  * http://​www.bufferbloat.net/​projects/​bloat/​wiki/​Linux_Tips +  * ''​**Root QDisc**''​ Not an actual queuing discipline (QDisc), but rather the location where traffic control structures can be attached to an interface for egress (outbound traffic). It can contain any of the queuing disciplines (qdiscs) with potential classes and class structures.  
-  * Source Code: [[http://​lxr.free-electrons.com/​source/​net/​sched/​]]+  * ''​**Ingress QDisc**''​ The location where ingress (incoming traffic) filters can be attached. For practical purposes, the ingress qdisc is merely a convenient object onto which to attach a policer to limit the amount of traffic accepted on a network interface. 
 +  * ''​**Classless QDisc**''​ A QDisc with no configurable internal subdivisions. 
 +  * ''​**Classful QDisc**''​ Contains multiple classes. Some of these classes contain a further QDisc, which may again be classful, but need not be.  
 +  * ''​**Work-Conserving**''​ A work-conserving QDisc never delays packets. It does NOT "​shape"​ packets. 
 +  * ''​**Non-Work-Conserving**''​ A non-work-conserving QDiscs may delay packets and "​shape"​ them. This means that they sometimes refuse to pass a packet, even though they have one available. 
 +  * ''​**Tail drop Queue**''​ see [[wp>​Tail drop]] 
 + 
 +**''​Classes''​** Classes are sub-QDiscs which allow the user to configure QoS in more detail. Classes can contain additional classes. Classes do not have a queue, do not contain any network packets and cannot contain filters. 
 + 
 +  * ''​**Leaf Class**''​ End class without any child classes. Always contains a QDisc! In case one is not configure, the default pfifo_fast is used. Leaf classes give unused bandwidth back to their parent class. 
 +  * ''​**Inner Class**''​ Classes which contain leaf-classes. 
 +  * ''​**Parent Class**''​ Parent class can dynamically pass bandwidth to leaf-classes 
 +  * ''​**Child Class**''​ Class that has another class or a QDisc as parent and contains classes. 
 + 
 +**''​Classifier''​** Determines which class to send a packet. 
 + 
 +**''​Filter''​** Classification can be performed using filters. A filter contains a number of conditions which if matched, make the filter match. 
 + 
 +**''​Scheduling''​** A QDisc may, with the help of a classifier, decide that some packets need to leave earlier than others. This process is called scheduling. 
 + 
 +**''​Shaping''​** [[wp>​Traffic_shaping|Traffic Shaping]] is the process of delaying packets to limit egress traffic to a maximum rate or smooth bursts. 
 + 
 +**''​Policing''​** [[wp>​Traffic_policing| Traffic Policing]] is the practice of dropping, marking or ignoring ingress packets that don't comply with user-defined criteria. 
 + 
 +**''​Filters''​** Filters are used by classful QDiscs to determine which class a packet will be queued to. 
 + 
 +**''​TCP Turbo''​** Yet another [[wp>​Buzzword bingo]] term; means the prioritization of TCP ACK-packets on the upload-side 
 + 
 +===== Reference ===== 
 +  * [[http://​lartc.org|Linux Advanced Routing & Traffic Control]] 
 +  * [[http://​lartc.org/​howto/​lartc.qdisc.html|Queueing Disciplines for Bandwidth Management]] 
 +  * [[http://​lartc.org/​manpages|tc man pages]] 
 +  * [[http://​www.bufferbloat.net/​projects/​bloat/​wiki/​Linux_Tips|Linux Tips]] 
 +  * [[http://​lxr.free-electrons.com/​source/​net/​sched/​|Source code]]
   * [[http://​linuxgazette.net/​135/​pfeiffer.html|TCP and Linux' Pluggable Congestion Control Algorithms]]   * [[http://​linuxgazette.net/​135/​pfeiffer.html|TCP and Linux' Pluggable Congestion Control Algorithms]]
-  * [[http://​www.mail-archive.com/​lartc@mailman.ds9a.nl/​msg17009.html|packet ​scheduler and VLANs]]+  * [[http://​www.mail-archive.com/​lartc@mailman.ds9a.nl/​msg17009.html|Packet ​scheduler and VLANs]] 
 + 
 +===== Tags ===== 
 +{{tag>​QoS}}
doc/howto/packet.scheduler/packet.scheduler.1347454884.txt.bz2 · Last modified: 2012/09/12 15:01 by valentt