Copyright © 2004 Juniper Networks, Inc. 1 SEEREN2 Summer School Heraklion, Sept 25 th Routing Issues: QoS/CoS Jean-Marc Uzé Liaison Research & Education, EMEA

2 Copyright © 2004 Juniper Networks, Inc. Agenda: QoS/CoS Workshop  Module 1: Overview of QoS/CoS  Module 2: JUNOS CoS implementation (J/M/T-Series)  Module 3: Introduction to JUNOS CLI  Module 4: GEANT2 QoS services Implementation The content of this module is courtesy of Dante (

3 Copyright © 2004 Juniper Networks, Inc. Module 4: GEANT2 QoS services implementation  GÉANT Network and Services  Premium IP  Less than Best Effort  Queuing on GÉANT and status  Router Configuration  Premium IP Management

4 Copyright © 2004 Juniper Networks, Inc.   10 Gb/s IP/MPLS backbone with Juniper T640s, M160s, M40s   4 x 10 Gb/s to North America   Dark fiber and WDM optical technology   Connecting 34 European Countries and 30 National R&E Networks   European connectivity to over 3000 R&E institutions   Advanced Services:   IPv6   Premium IP   Multicast v4 + v6   Best Effort   Less Than Best Effort   Layer 2 VPN GEANT2 / Dante

5 Copyright © 2004 Juniper Networks, Inc. Global Connectivity

6 Copyright © 2004 Juniper Networks, Inc. IP QoS Services on GÉANT  Premium IP  upper-bounded one-way delay  upper-bounded IPDV  negligible packet loss  guaranteed capacity  Less than Best Effort  class of traffic using the un-utilised Best Effort and higher classes of service bandwidth 

7 Copyright © 2004 Juniper Networks, Inc. AGENDA  GÉANT Network and Services  Premium IP  Less than Best Effort  Queuing on GÉANT and status  Router Configuration  Premium IP Management

8 Copyright © 2004 Juniper Networks, Inc. Premium IP Model  End-to-end service across multiple management domains  using diffserv, ATM CBR or over-provisioning(!)  packet tagged DSCP 46 (EF )  destination aware service  packet with other DSCP are left untouched (packets from other service)  Premium IP bandwidth limited to 10% of the link capacity  can cope with 20% in case of circuit failure

9 Copyright © 2004 Juniper Networks, Inc. GEANT and IP Premium Service Source:

10 Copyright © 2004 Juniper Networks, Inc. Premium IP on GEANT  Protection of authorised Premium IP traffic  under normal circumstances, the Premium IP traffic of a circuit is limited to 10% of the circuit capacity  20% in case of another circuit failure  bullet-proof all the GÉANT accesses against unauthorised Premium IP traffic (tagged DSCP 46) on all the ingress interfaces  if DSCP 46 packet arrives on GÉANT and part of an unauthorised flow: classify the packet into the Best Effort queue and remark it as Best Effort (DSCP 0)  if DSCP 46 packet arrives on GÉANT and is part of an authorised flow: check against policer according capacity requested in the SLA (in-profile accepted, out-of-profile dropped)

11 Copyright © 2004 Juniper Networks, Inc. Premium IP on GEANT  Protection of authorised Premium IP traffic [cont]  per next AS rate-limitation (implemented by Juniper for GÉANT)  can also do source-destination IP addresses when NREN don’t do it. (NREN = National Research & Education Network, a Dante customer network directly connected to GEANT)  Trust the Premium IP traffic received from a GÉANT backbone interface.  Traffic checked at the GÉANT ingresses.  Configure queuing mechanism on the backbone and access interfaces.  strict-high priority is configured to the Premium IP queue.  Don’t forget that the amount of Premium traffic expected in the Premium IP queue is 10% of the link capacity (service over- provisioned by a factor 9); this is assured by ingress policing.  90% for the BE and 5% for the network control (and 5% for LBE)

12 Copyright © 2004 Juniper Networks, Inc. Test result end-to-end IP Premium

13 Copyright © 2004 Juniper Networks, Inc. AGENDA  GÉANT Network and Services  Premium IP  Less than Best Effort  Queuing on GÉANT and status  Router Configuration  Premium IP Management

14 Copyright © 2004 Juniper Networks, Inc. Less than Best Effort  Class of traffic using the un-utilised Best effort and higher classes of service bandwidth  in case of competition for resources, the LBE traffic will de discarded before any Best-Effort or higher classes of traffic.  use the DSCP 8 (001000) - same as Internet2 scavenger service.  Congestion on an interface due to LBE  should be transparent to the BE or higher classes of services  no BE or higher classes of services packet loss

15 Copyright © 2004 Juniper Networks, Inc. Less than Best Effort  No end-to-end guarantees  no metric needed to quantitatively describe the service  Can be supported on one interface  anywhere else, the LBE tagging should be passed transparently.  Application scenarios  mirroring, test traffic, some GRID data transfers, network backups, protection of research traffic from student dormitory one.

16 Copyright © 2004 Juniper Networks, Inc. LBE Queuing Technique  For algorithm with bandwidth shared assignment, as Weighted Wound Robin and Weighted Fair Queuing, a very small bandwidth share is allocated to the LBE queue.  Typically between 0% and 5%

17 Copyright © 2004 Juniper Networks, Inc. LBE: Measurement with congestion  One-way delay  Increase of LBE maximum one-way delay of 1.5ms  Increase of BE maximum one-way delay of 400µs

18 Copyright © 2004 Juniper Networks, Inc.   Normal Traffic   Normal Traffic + Less Than Best Effort 2.0 Gbit/s   Normal Traffic + Radio Astronomy Data 500 Mbit/s   Normal Traffic + Radio Astronomy Data + Less Than Best Effort 2.0 Gbit/s LBE live test: ER2002 Demo - VLBI - dataGRID

19 Copyright © 2004 Juniper Networks, Inc. AGENDA  GÉANT Network and Services  Premium IP  Less than Best Effort  Queuing on GÉANT and status  Router Configuration  Premium IP Management

20 Copyright © 2004 Juniper Networks, Inc. Queuing Technique  WRR - Juniper M-series  Weight  Assure the queue to be given a minimum amount of bandwidth proportional to the weight.  Priority  queue with high priority are served before the low priority  allow the BE (and other high priority queues) to be served first until empty before serving the LBE one.  WRED  is used to limit the queuing delay in case of congestion  use to protect one class of traffic over the other within a queue.

21 Copyright © 2004 Juniper Networks, Inc. DSCP/ToS Values used by GEANT ServiceDSCP valueToS valueJuniper aliasToS (hex)DSCP-ToS binary Premium IP 46184efB xx LBE 832cs xx DWS 32128cs xx Network control cs6C xx Network control cs7E xx   The DSCP/ToS values used in GÉANT to classify the traffic of the different QoS classes are shown in the table below. In addition to the three service classes offered to transiting traffic there is a DWS (IP commodity service) and a Network Control class, which are traffic classes used internally to the GÉANT network.

22 Copyright © 2004 Juniper Networks, Inc. Juniper Networks and CoS Services GEANT with IP Premium + LBE Service  Junos CoS features include policing, (strict) priority queuing, weighted round robin (WRR), precedence/DSCP field rewrite, and random early drop RED.  On a Juniper M-series Router each port has 4 Queues Weighted Round Robin Percentages can be set for each Queue  New generation Q-PICs offers multiples queues per logical interfaces (Ethernet VLAN, ATM PVC, etc.) WRR Source: and

23 Copyright © 2004 Juniper Networks, Inc. QoS Configuration on GEANT  The configuration has completed on most of the GEANT routers allowing Premium IP, BE and LBE to co-exist.  The routers where the three services have been enable are represented as green on the following map.  The routers coloured yellow are Juniper routers where “old” FPCs have been re-used from TEN-155 (1999) These old FPC’s that do not allow for the full functionality of QoS.  As such BE is not ideally protected by LBE and the bandwidth is effectively shared. Premium IP only is supported.

24 Copyright © 2004 Juniper Networks, Inc. Current QoS Configuration on GEANT

25 Copyright © 2004 Juniper Networks, Inc. AGENDA  GÉANT Network and Services  Premium IP  Less than Best Effort  Queuing on GÉANT and status  Router Configuration  Premium IP Management

26 Copyright © 2004 Juniper Networks, Inc. Router Configuration  Each router in the GÉANT network contains certain QoS building blocks in order to configure Per Hop Behaviors (PHB). The configuration shown here is taken from a Juniper M160 router with JUNOS 5.7 and with E-FPC (enhanced FPCs) and SDH interfaces.  Classifiers, schedulers and rewrite rules can be associated to each interface. In GÉANT two types of interface configurations are used for QoS  a backbone interface  an access interface (i.e. the interface where the traffic from an NREN is entering GÉANT)

27 Copyright © 2004 Juniper Networks, Inc. DSCP and ToS Values  Type of Service field illustration  Illustration of DSCP Type of Service field configuration ServiceDSCP valueToS valueJuniper aliasToS (hex)DSCP-ToS binary Premium IP46184efB xx LBE832cs xx DWS32128cs xx Network control cs6C xx Network control cs7E xx

28 Copyright © 2004 Juniper Networks, Inc. Router Interfaces  Backbone interface so-7/0/0 { scheduler-map MAP-BASIC; unit 0 { classifiers { dscp backbone-classifier; } rewrite-rules { dscp basic-rewrite-rules; } } }  Access interface so-0/2/3 { scheduler-map MAP-BASIC; unit 0 { classifiers { dscp access-classifier; } rewrite-rules { dscp basic-rewrite-rules; } } } In addition, the access interface may contain filters in order to classify and police Premium IP traffic. The following configurations apply to all (access and backbone) interfaces.

29 Copyright © 2004 Juniper Networks, Inc. Drop Profiles  Drop profile define the parameters used by the Random Early Detection (RED) mechanism that MAY be used in a queue dws-drop-profile { fill-level 35 drop-probability 10; fill-level 40 drop-probability 100; } be-drop-profile { fill-level 15 drop-probability 30; fill-level 19 drop-probability 50; fill-level 24 drop-probability 70; fill-level 30 drop-probability 100; } less-than-be-drop-profile { fill-level 25 drop-probability 30; fill-level 30 drop-probability 50; fill-level 40 drop-probability 70; fill-level 50 drop-probability 100; }

30 Copyright © 2004 Juniper Networks, Inc. Queues and Schedulers 1. Associate a name with each queue  Note: a queue is sometimes also called a forwarding class forwarding-classes { queue 0 best-effort; queue 1 expedited-forwarding; queue 2 less-than-best-effort; queue 3 network-control; } The Premium IP traffic is classified into the expedited-forwarding queue. The naming of the queues is performed once and applies to all interfaces of the router.

31 Copyright © 2004 Juniper Networks, Inc. Queues and Schedulers 1. Define scheduler configurations  scheduler weight, queue size and priority as set at the GÉANT router for each queue: sch-best-effort { transmit-rate percent 90; buffer-size percent 50; priority low; } sch-expedited-forwarding { buffer-size percent 15; priority strict-high; } sch-less-than-best-effort { transmit-rate percent 5; buffer-size percent 30; priority low; drop-profile-map loss-priority low protocol any drop-profile less-than-be-drop-profile; drop-profile-map loss-priority high protocol any drop-profile be-drop-profile; } sch-network-ctrl { transmit-rate percent 5; buffer-size percent 5; priority high; }

32 Copyright © 2004 Juniper Networks, Inc. Queues and Schedulers 3. Associate a scheduler with a queue (FC)  The main advantage of the scheduler-map is that it can be applied to more than one interface. MAP-BASIC { forwarding-class best-effort scheduler sch-best-effort; forwarding-class expedited-forwarding scheduler sch-expedited-forwarding; forwarding-class less-than-best-effort scheduler sch-less-than-best-effort; forwarding-class network-control scheduler sch-network-ctrl; }

33 Copyright © 2004 Juniper Networks, Inc. Classification  The classifier is a functional block located at the input interface that sets three internal bits for each IP packet  Two bits that select the output-queue. There are four output queues, also called forwarding-classes, per port.  One bit to indicate the loss-priority, the packets classified in an output-queue can have two different values of loss-priority (low or high).  Note that it is also possible to classify packets by means of an input filter. Backbone Classifier dscp backbone-classifier { import default; forwarding-class best-effort { loss-priority low code-points [ af11 af12 af13 ]; loss-priority high code-points cs4; } forwarding-class less-than-best-effort { loss-priority low code-points cs1; } } Access Classifier dscp access-classifier { import default; forwarding-class best-effort { loss-priority low code-points [ af11 af12 af13 ]; } forwarding-class less-than-best-effort { loss-priority low code-points cs1; loss-priority high code-points [ ef cs4 ]; }

34 Copyright © 2004 Juniper Networks, Inc. Marking  The marking of packets with a DSCP value is the last QoS action performed before the transmission of the packet (hence after firewall filter evaluation) dscp basic-rewrite-rules { forwarding-class best-effort { loss-priority high code-point cs4; } forwarding-class expedited-forwarding { loss-priority low code-point ef; } forwarding-class network-control { loss-priority low code-point nc1; loss-priority high code-point nc2; } forwarding-class less-than-best-effort { loss-priority high code-point be; loss-priority low code-point cs1; } } Service Incoming DSCP value New DSCP value Authorised Premium IP4646/drop Un-authorised Premium IP 460/5 DWS320 LBE8 8 Network Control48/5648 Best Effortother valuesUnchanged

35 Copyright © 2004 Juniper Networks, Inc. AGENDA  GÉANT Network and Services  Premium IP  Less than Best Effort  Queuing on GÉANT and status  Router Configuration  Premium IP Management

36 Copyright © 2004 Juniper Networks, Inc. Useful Tool  Feature of the NANOG traceroute to discover the DSCP changes along the path (Simon Leinen from Switch:-)

37 Copyright © 2004 Juniper Networks, Inc. Need for Automation  Service management was done manually  Service set up, maintenance and termination was done by phone calls and s  Considerable manual effort required  Complexity in keeping track of:  Path information  Current and future reservations  Premium IP utilisation levels  Changes in network topology  Multi-party communication

38 Copyright © 2004 Juniper Networks, Inc. System Architecture  Java web-based architecture  Using Apache 1.3 web server, Tomcat servlet container & MySQL Database

39 Copyright © 2004 Juniper Networks, Inc. Premium IP Reservation Tool Features (1)  Authentication & Authorisation  Path Finder  Find shortest path between two end points  Dynamic based upon configured IS-IS cost  Utilisation Monitoring  Check Premium IP reservation levels on each intermediate link along the path  Take into account all active reservation during the given time period  Display the available Premium IP capacity

40 Copyright © 2004 Juniper Networks, Inc. Premium IP Reservation Tool Features (2)  Reservation Management  View, Request, Modify, Cancel  Based upon available Premium IP capacity  Contact management  IP Address management  Automated notification  to User, DANTE - Premium IP team, NOC  upon Reservation Request, Modification, Cancellation  Router Configuration update (script)  Others: IS-IS cost, Reports, Archival, System Administration...

41 Copyright © 2004 Juniper Networks, Inc.

42 Copyright © 2004 Juniper Networks, Inc. Thank you Jean-Marc Uzé Liaison Research & Education, EMEA Mobile: Place Ronde, Paris-La-Defense, France