1. Computer Network An Open Source Approach Chapter 7: Internet QoS Ying-Dar Lin, Ren-Hung Hwang, Fred Baker 1 Chapter 7: Internet QoS

2. Content 7.1 General Issues  Signal Protocol  QoS Routing  Admission Control  Packet Classification  Policing  Scheduling 7.2 QoS Architectures  Integrated Service (IntServ)  Differential Service (DiffServ)  Comparison 7.3 Algorithms for QoS Components  Admission Control  Flow Identification  Token Bucket  Packet Scheduling  Packet Discarding 7.4 Summary 2

3. Chapter 7: Internet QoS Problem Statement There is no QoS in Internet currently (Only Best Effort is supported) ‚However, the Internet architecture (Ethernet/IP) is popular ƒAnd QoS is required for many applications (Bounded delay and fixed sending rate requirement) „So.. What is the QoS Internet Architecture? …The first kind of model is Integrated Services. (It is intuition, but is difficult to implement and scale) †The Second kind of model is Differentiated Services. (A simple solution for implementation and deployment) 3

4. Chapter 7: Internet QoS 7.1 General Issues  Signal Protocol  QoS Routing  Admission Control  Packet Classification  Policing  Scheduling 4

5. Chapter 7: Internet QoS Requirements for the QoS Network Admission Control Classification Marking Policing Shaping Signaling Protocol QoS Routing Scheduling Control Plane Data Plane Taipei: 90 mins -> <- HsinChu: 30 mins QoS Routing Admission Control packet 5

6. An Operating Example of Admission Control Chapter 7: Internet QoS 6

7. The General Model of a Queuing Discipline Dequeue Queuing Discipline Black Pipe Enqueue arrival packets departure packets Single FIFO Queue FIFO Queue SC 7

8. Chapter 7: Internet QoS Open Source Implementation 7.1: Traffic Control Elements in Linux Input Device IP Forwarding Upper Layers Process Output Queuing Output Device Traffic Control Output Device Without QoS FilterClass Queuing Discipline FilterClass Policing Queuing Discipline 8

9. Chapter 7: Internet QoS Open Source Implementation 7.1: Traffic Control Elements in Linux Functions invoked when enqueue and dequeue dev_queue_xmit qdisc_enqueue qdisc_wakeup qdisc_restart net_bh qdisc_run_queues qdisc_dequeue hard_start_xmit Timer 9

10. Chapter 7: Internet QoS  Integrated Service (IntServ)  Service Type: Guaranteed, Control load, Best effort  The Trip of a Resource Reservation Request  Request Handling by IntServ Routers  Request Enforcement in IntServ Routers  Differential Service (DiffServ)  DS field, per hop forward behavior,  Ingress, interior, exgress router  Comparison 7.2 QoS Architectures 10

11. Chapter 7: Internet QoS Concept of IntServ Network A “ flow “ is the basic management unit Supporting accurate quality control. Resource reservation is dynamic. All traffic control functions are embedded in one router. All routers in the network must be IntServ aware. 11

12. Chapter 7: Internet QoS Service Types Provided by IntServ Service Types GuaranteedControl LoadBest Effort Provide QoS Guaranteed BW E2E * Delay Bound Emulate a lightly loaded network for AP None RFC RFC 2212RFC 2211None *) “E2E” implies End-to-End Common Traffic Description Parameters: Peak rate p Average rate r Burst size b 12

13. Chapter 7: Internet QoS The RSVP Process from the Viewpoint of an Application Application Server A B C Accept? IntServ Domain N-> Reject the request Y-> Forward the request Reservation request QoS-Aware Router 13

14. Chapter 7: Internet QoS The Data Plane in an IntServ Router. Flow Identifier Fq 1 Fq n scheduler Fq 2 Best Effort Q Src IPDest. IP Dest PortSrc Port Protocol ID Flow Queue Policer Data Plane of IntServ Router 14

15. Chapter 7: Internet QoS Basic Elements Tree of DiffServ Differentiated Services Service Level Agreement (SLA) Traffic Control Service Link- Layers Signaling Traffic Engineering Policy Traffic Conditioning Agreement (TCA) service provisioning service configuration classification policer marker scheduler Expedited Forwarding Assured Forwarding Best-effort Pt-to-Pt MPLS ATM RSVP LDAP COPS 15

16. Chapter 7: Internet QoS Concept of DiffServ Network 1. If you want to get QoS, you should sign the contracts with SP first 2. Support the aggregated traffic resource allocation only 3. Provide a better behavior and provision (no accurate bandwidth reservation ) Objective: Define a simple and coarse methods of providing differentiated classes of service for Internet traffic. Edge Router Core Router DiffServ Domain Ingress Router Egress Router Core Routers Police, Mark, Shape, Drop Packets Forward Packet 16

17. Chapter 7: Internet QoS Differentiated Services (DS) Field Redefine the existing IP TOS field to DS field Some level of backward compatibility to IP TOS Precedence DTR00 IP TOS DS DSCP 00 12345678 No use 2 6 = 64 behaviors 12 AF PHBs 1 EF PHB 1 Best Effort PHB 8 Class Selector PHBs 12 AF PHBs 1 EF PHB 1 Best Effort PHB 8 Class Selector PHBs 17

18. Chapter 7: Internet QoS Allocated Space of Code Points 18

19. Chapter 7: Internet QoS Per-Hop Behavior Group PHB GroupAF (Assured Forwarding) EF (Expedited Forwarding) Best-Effort Features Olymic service (an example) 4 delay priority classes, each with 3 drop precedence subclasses (quantitative service) Premium/Virtual Leased Line Service (quantitative service) none Recommended DSCP in DS- field AF1AF2AF3AF4 101110000000 Low 010000011000100000101000 Middle 010010011010100010101010 High 010100011100100100101100 Traffic Control Static SLA Policing, classification, marking, RIO/WRED scheduling Dynamic SLA Policing, classification, marking, Priority/WFQ scheduling FIFO scheduling Non- conforming Traffic Re-mark as Best-EffortDropForward 19

20. Chapter 7: Internet QoS The Ingress stage of a packet in the Edge Router Packet Classifier Meter DSCP Marker Shaper Dropper Traffic Conditioning DS Domain DSCP Classifier Class Scheduler Traffic Forwarding 20

21. Chapter 7: Internet QoS The Interior Stage of a Packet in the Core Router DS Domain DSCP Classifier Class Scheduler Traffic Forwarding Routing Database Control Plane Data Plane 21

22. Chapter 7: Internet QoS Differences between DiffServ and IntServ Compared ItemsDiffServIntServ Manageable UnitClassFlow Router’s CapabilityEdge and CoreAll in One Defined in the Standard Forwarding behavior Service Type Guarantee RequiredProvisioningReservation Work RegionDomainEnd-to-end 22

23. 7.3 Algorithms for QoS Components  Admission Control  Flow Identification  Token Bucket  Packet Scheduling  Packet Discarding 23 Chapter 7: Internet QoS

24. Objectives and Issues of Admission Control Objectives Provide a mechanism to admit users’ resource reserved request according to 1) Current Resource Usage 2) The User’s Requirement Major Issues 1) How to get the information about link resource usage efficiently ? 2) How to know the residual resource is enough to satisfy a user’s request under OVERBOOKING? 24

25. Chapter 7: Internet QoS Basic Approaches of Admission Control Detail Version: - Average/Peak Rate Combinatory - Additive Effective Bandwidths - Engineering the loss Curve - Maximum-Variance-Based Approaches Drawbacks  Sometimes, it is difficult to give a a ccurate and tight traffic model.  It may cause low utilization of network in order to meet the worst-case requirements. Statistical-Based Apply to the Specific traffic Model 25

26. Chapter 7: Internet QoS Basic Approaches of Admission Control Measurement-Based - Exponential averaging over consecutive measurement estimation new =(1-w) X estimation old + w X measurement new - Time Window Estimated rate=max[C1,C2,C3….Cn] Apply to the non-specific traffic Model Drawbacks  It c an not be used to provide tight guarantee constraints  Measurement is expensive. 26

27. Chapter 7: Internet QoS Objectives and Issues of Flow Identification Objectives Identify a packet to decide whether it belongs to one reserved flow or nothing. Major Issues  Search Speed  Low Storage requirement  Fast Update  Scalability  Number Fields  Flow Numbers Source IP, Port Dest IP, Port Protocol ID Identified Fields in a IP pkt 104 bits Direct Memory Lookup Binary Search Tree SPACE VS. SPEED 27

28. Chapter 7: Internet QoS rsvp_session Open Source Implementation 7.3: Flow Identification rsvp_classify of CLS_RSVP.h rsvp_session dstaddr protocol id tunnelid rsvp_filter RF rsvp_session RF h2 RF srcaddr tcf_result tcf_police info. RF skb h1=hash_dst() h1 h2=hash_src() X return value rsvp_filter src tcf_result rsvp_filtertcp_police 28

29. Chapter 7: Internet QoS rsvp_head: First-LevelHash hash_dst() src_dst() rsvp_session: Second-LevelHash hash bucket hash function rsvp_session list rsvp_filter list total 256 (dst,protocol id, tunnelid) lists rsvp_filter list 16 (src,src port) list + 1 wildcard src lists Apkt. arrives Open Source Implementation 7.3: Flow Identification The Double-level Hash Structure in CLS_RSVP.C 29

30. Chapter 7: Internet QoS Open Source Implementation 7.3: Flow Identification rsvp_classify & rsvp_change of CLS_RSVP.h hash_dst() sequential search in the rsvp_session list hash_src () sequential search in the rsvp_filter list matchnomatch N N The flowchart of function rsvp_classify Has rsvp_filter assigned ? adjust classid Has rsvp_session existed ? modify create insert a rsvp_session insert a rsvp_filter Y N Y The flowchart of function rsvp_change 30

31. Chapter 7: Internet QoS Traffic Description Model Traffic Regulator & Traffic Policing Objective: Conforming the incoming traffic for each flow to it’s traffic specification described as resource reserved. Token Bucket Peak Rate p Token Stream with average rate r Bucket depth b Flow Queue Rate Permitted Packets Incoming Packets drop? 31

32. Chapter 7: Internet QoS Example for Token Bucket p=2 r=1 h=10 Flow Queue (a) t=10s p=2 r=1 h=5 Flow Queue (b) t=15s 10 9 p=2 r=1 h=0 Flow Queue (c) t=19s 9 10 5 5 p=2 r=1 h=0 Flow Queue (d) t=24s 9 10 5 9 5 32

33. Chapter 7: Internet QoS Open Source Implementation 7.4: Token Bucket The enqueue function of TBF get a packet from sk_buff skb->len > q->max_size qdisc_enqueue(skb, q->qdisc) Y N drop full 33

34. Chapter 7: Internet QoS Open Source Implementation 7.4: Token Bucket The dequeue function of TBF is the peak rate set? Get one packet from skb queue Calculate the accumulated tokens toks after the last query toks=min(t_c-now,buffer) Estimate the admitted tx time ptoks given the pkt is sent out ptoks = ptokens + toks – (len(pkt)/P) both ptoks and toks >=0 Reinsert the pkt into the head of the skb queue Estimate the admitted tx time given the pkt is sent out toks = tokens + toks – (len(pkt)/R) Admitted the packet to be txed and update the value of tbf_sched Y N Y N 34

35. Chapter 7: Internet QoS Objectives and Issues of Packet Scheduling Objective Provide a mechanism to divide the bandwidth and guarantee that each flow can get the allocated resource Issues  Isolation of Flows  Low End-to-End Delays  Utilization  Fairness  Simplicity of Implementation  Scalability  Work-conserving or Non Work-conserving 35

36. Chapter 7: Internet QoS Basic Type of Scheduling: Round Robin (Frame Based) Deficit Round Robin: A Practical Scheduler It is simple, but not very fair within a small time interval 100 200 300 400 200 400200 300 Flow Queue 1 Flow Queue N S credit 200 36

37. Chapter 7: Internet QoS An operating case of DRR: Round 1 and 2 100 0 0 200 400 200 300 Flow Queue 1 Flow Queue N S credit F2 F3 F4 0 0 0 0 200 300 Flow Queue 1 Flow Queue N S credit 200 F1 F2 F3 400 200 37

38. Chapter 7: Internet QoS 200 F3 100 200 100 0 200 300 Flow Queue 1 Flow Queue N S credit 0 0 0 0 200 Flow Queue 1 Flow Queue N S credit 200 F1 F2 F3 300 200 An operating case of DRR: Round 3 and 4 38

39. Chapter 7: Internet QoS Advanced Concept of Scheduling: Smallest Selection (Sorted-Based) G eneralized P rocessor S haring (GPS)  An ideal fair queuing based on the fluid model.  It provides perfect fairness in bandwidth allocation  For a traffic source LB(b,r), it can guarantee - a delay bound of b/r - the received BW >=  However, it’s impossible to be implemented!! A1A2A3 B1B2 Flow A Flow B Fluid Model Packetized Model C1C2C3 Flow C A1B1A2B2 C3 C1C2A3 39

40. Chapter 7: Internet QoS  Try to emulate a GPS by calculating the departure time of a packet (finish time) and give a virtual timestamp for each packets.  Finish time is a number representing the order of sending out the packets. ( not is a real departure time )  If there are packets in the flow queue i, the finish time of k-th packet is  If the flow queue i is empty, the finish time of k-th packet is  Virtual Time Implementation t j is the time where the j th event occurs. Packet-by-Packet GPS (PGPS) 40

41. Chapter 7: Internet QoS Arrival-Departure Curve 41

42. Chapter 7: Internet QoS S S 1, 2002, 2003, 2004, 200 6, 200 9, 400 10, 200 7, 300 Flow Queue 1 (  1 =0.1) 5, 200 8, 200 13, 20012, 200 2000400060008000 2000 2666 3500 1000 666 1000500 12, 200 8, 2005, 20013,2001, 2006, 200 9, 400 10, 200 7, 300 2, 200 F4 F3F2 F4 F1 F2 F3 F2 F1 Output queue 3, 2004, 200 F1 11, 200 3332 11,200 id packet length VFT id packet length Flow Queue 2 (  1 =0.2) Flow Queue 4 (  1 =0.4) Flow Queue 3 (  1 =0.3) Flow Queue 1 (  1 =0.1) Flow Queue 2 (  1 =0.2) Flow Queue 4 (  1 =0.4) Flow Queue 3 (  1 =0.3) An operating case of WFQ 42

43. Chapter 7: Internet QoS Round Robin and Sorted-Based will be similar as …. Assume the credits got by all flows in one round is decrease to infinitesimal with a fixed ratio. Assume it spends no time to decide whether there are packets allowed to be send out in one flow. Then, the order of packet transmitting will be similar. In fact, Sorted-Based may be regarded as a ingenious implementation of Round Robin 43

44. Chapter 7: Internet QoS csz_classify(): get the flow id csz_update(): update VST check the len. of flow queue drop the pkt full calculated new VFT based on the last VST Is the flow active? calculated new VFT based on the VST csz_insert_finish () : Wake up the flow csz_insert_start () : Wake up the flow skb_queue_tail() N Y Open Source Implementation 7.5: Packet Scheduling The enqueue function of sch_csz.c 44

45. Chapter 7: Internet QoS get the csz_flow where the headed packet with the smallest VFT skb_dequeue() : get the headed packet of the flow recalculate the min VFT in the flow if the flow is non empty csz_insert_start() Return the packet for sending out Y N Open Source Implementation 7.5: Packet Scheduling The VFT calculation function of sch_csz.c 45

46. Chapter 7: Internet QoS Set delay to the time escaped from last arrival packet If any active flow exists Get the minimum VFT, F, from the headed flow of the list f Assume all flows are active and calculate the current VST VST<F All residual flows are active and we get the correct F. The first flow in the list f is inactive Calculate VST at the time that the flow became inactive and adjust delay. Open Source Implementation 7.5: Packet Scheduling The dequeue function of sch_csz.c Y N 46

47. Chapter 7: Internet QoS Packet Discard(Buffer Management) (a) Tail Drop (The Natural Method ) : Drop packet as queue is full queue Drop (b) Early Drop : To early drop packets before queue is full queue Drop Drop with P Queue 47

48. Chapter 7: Internet QoS Longest Tail Drop Logic Memory Pool AAAA BBB CCC B C C AAAA BBB CCC B C C queue but has a P to be dropped Rerspective Space 48

49. Chapter 7: Internet QoS Random Early Detection (RED) 1. To Drop or Mark all packets as Qlength avg >max_threshold 2. To Queue all packets as Qlength avg <min_threshold 3. To Drop packets with p as min_threshold < Qlength avg <max_threshold where p = max p * ( Qlength avg – min_threshold) / (max_threshold-min_threshold) Rules To Maintain a variable Qlength avg computed by Qlength Qlength avg = (1-W)Qlength avg + W * Qlength 49

50. Chapter 7: Internet QoS An operating case of RED qth_max=4 Drop Drop with P Queue 1 qth_min=1 Drop with P Queue 2 3 45 1 2 3 Drop 5 (a) qave = 0 and then Pb=0 (b) qave = 2; then Pb=0.1*(2-1)/(4-1)=0.033 67 Drop with P Queue 2 4 8 Drop 5 11 (c) qave = 6; then Pb=0.1*(4-1)/(4-1)=0.1 1213 67 10 9 qth_max=4 qth_min=1 qth_max=4 qth_min=1 4 Let qth_min=1, qth_max=4, max_P=0.1 and w=1. Such a w implies that qave is always equal to the present queue length. 50

51. 7.4 Summary 6 key QoS components  Control-plane: signaling protocol, QoS routing, admission control  Data-plane: classifier, policer, scheduler Per-flow IntServ vs. per-class DiffServ QoS architectures: failed to be deployed QoS components: in our daily usage Chapter 7: Internet QoS 51