1 Some QoS Deployment Issues Shumon Huque University of Pennsylvania MAGPI GigaPoP April 15th NSF/ITR Scalable QoS Workshop

2 University of Pennsylvania network Large research university in Philadelphia, PA 22,000 students, 4,000 faculty, 10,000 staff 48,000 registered IP addresses 200 switched subnets Central routing between them and out to Internet and Internet2

3 MAGPI GigaPoP An Internet2 GigaPoP Value added services Commodity Internet transit Facilitator of regional edu/research initiatives Subscribers UPENN, Lehigh U, Princeton U, PA county school units, J&J Pharmaceuticals

4 MAGPI GigaPoP (cont) External Connectivity Internet2  OC-12c POS to Abilene Commodity Internet  UUNET: OC-3  Cogent: Gigabit Ethernet  Yipes: Gigabit Ethernet (rate limited)  DCANet: Fast Ethernet

5 Who wants QoS? University researchers QoS researchers in CS department Research applications needing strict guarantees on latency, b/w, jitter etc Networking staff (Univ and gigaPoP) Manage exploding b/w needs Enable new classes of applications  Eg. VoIP, video conferencing, streaming Run non-mission critical traffic at lower priority  Eg. File sharing apps, dorm traffic, bulk transfers

6 What types of QoS? DiffServ in routing core and gigaPoP Layer2 priority (802.1p) in the switched portions of the campus network Mapping L3 QoS to/from L2 QoS Signalling and admission control? RSVP intra-domain? Aggregate reservations & map to Diffserv traffic class at edge? Bandwidth Broker signalling?

7 DiffServ Types of forwarding behavior we are most interested in: EF (Expedited Forwarding) BE (Best Effort - default PHB) LBE/Scavenger (eg. QBSS) ABE - low delay form

8 Interdomain Internet QoS Not very optimistic Some ISPs are starting to offer services Multiplicity of providers Need for them to run interoperable QoS implementations Mechanisms to ask for QoS reservations across administrative domains Peering/SLA issues

9 Interdomain Internet2 QoS More optimistic Typically one or a few QoS enabled I2 backbone networks (eg. Abilene) Agreed upon QoS architecture Common set of operational practices and procedures Some provisioning procedures in place Existing demand from researchers

10 Deployment Challenges I2 backbone is an R&E network, but.. Universities are using it to transport production traffic between them And not just traffic associated with meritorious research applications (one of the original ideas) GigaPoP is a production network providing access to I2 and Commodity Internet

11 Deployment Challenges (2) So, we need to be very careful about changes we introduce to the network to facilitate QoS Don’t jeopardize existing production traffic

12 Router support for QoS Not mature or well tested Often the features are in experimental code trains, unsuitable for deployment in a production network Marking, re-marking, policing, traffic shaping, appropriate queue scheduling disciplines etc Insufficient #queues to support large scale service differentiation Often software implementations of required queueing disciplines instead of hardware Obviously this situation will improve in the future

13 Router code support (cont) Example: Juniper routers 4.x release: Can police DS BA’s but not much more 5.x release: More queue scheduling disciplines Per queue traffic shaping DSCP marking and re-marking DSCP based prioritization and forwarding  Eg. Assigning EF BA to a high priority queue Mapping of 802.1p to Layer-3 QoS

14 Parallel Network Infrastucture Deploy parallel network infrastructure Place QoS enabled routers on this Researchers are happy, but.. Cost prohibitive

15 QoS policy issues Where does marking occur? Endstations First hop routers or switches Edge routers Who’s allowed to mark? How to validate? Complexity of deploying policies Additional controls and checks to enforce the policies  Policy servers: COPS, bandwidth brokers etc

16 Inter-domain signalling No suitable mechanisms today for end2end inter-domain signalling of QoS reservations, call admission control Manual/static provisioning Bandwidth brokers/SIBBS work ongoing

17 What we do today To facilitate researchers doing wide-area QoS experimentation: Conscious effort not to impede Provide research labs with an uncongested path though campus/gigapop to QoS enabled Abilene network Make sure intervening routers don’t mark or re- mark DSCP code points

18 Abilene QoS testbed

19 Bandwidth Management Alternatives University has experienced rapidly increasing bandwidth requirements Overprovision the campus network Buy more commodity Internet bandwidth through the gigaPoP Employ rate limiting where appropriate Employ lightweight QoS, eg. LBE/Scavenger

20 Endstation problems Network apps often unable to use available bandwidth because of problems on end- stations: Poorly designed applications, application protocols Insufficiently sized socket buffers Inefficient, insufficiently tuned network stacks Duplex mismatch MTU mismatch Having QoS in the network does not address this class of performance problems

21 Conclusion We’re interested in QoS Too early to deploy end2end reservation based QoS in many production networks Intra-domain QoS a near-term possibility Both reservation based and lightweight VoIP, degrading non-mission critical traffic End2End Inter-domain QoS difficult Co-ordination, SLAs, inter-domain signalling