Jon Turner Extreme Networking Achieving Nonstop Network Operation Under Extreme Operating Conditions DARPA PI Meeting, July 23-26, 2002

2 - Jonathan Turner - July 24, 2002 Project Overview Motivation »data networks have become mission-critical resource »networks often subject to extreme traffic conditions »need to design networks for worst-case conditions »technology advances making extreme defenses practical Extreme network services »Lightweight Flow Setup (LFS) »Network Access Service (NAS) »Reserved Tree Service (RTS) Key router technology components »Super-Scalable Packet Scheduling (SPS) »Dynamic Queues with Auto-aggregation (DQA) »Scalable Distributed Queueing (SDQ)

3 - Jonathan Turner - July 24, 2002 Switch Fabric IPPOPP FPX SPC TI IPPOPP FPX SPC TI IPPOPP FPX SPC TI IPPOPP FPX SPC TI IPPOPP FPX SPC TI IPPOPP FPX SPC TI Control Processor Prototype Extreme Router Field Programmable Port Ext. Network Interface Device Reprogrammable Application Device SDRAM 128 MB SRAM 4 MB Field Programmable Port Extenders Smart Port Card Sys. FPGA 64 MB Pentium Cache North Bridge APIC ATM Switch Core Transmisson InterfacesEmbedded Processors

4 - Jonathan Turner - July 24, 2002 Resource Reservation in Internet? Bandwidth reservation can provide dramatically better performance for some applications. Obstacles to resource reservation in Internet. »distaste for signaling protocols »perceived complexity of IntServ+RSVP »requires end-to-end deployment »little motivation for service providers How to get resource reservation in Internet? »keep it simple focus on top priorities - one-way unicast flows avoid complex signaling - leverage hardware routing mechanisms »make it useful when only partially deployed »provide motivation for ISPs to deploy it

5 - Jonathan Turner - July 24, Mb/s available 5 Mb/s available 2 Mb/s available 5 Mb/s available 20 Mb/s available Basic LFS Operation One way, unicast setup with partial reservation. »complete reservations locally when bandwidth released Optional ack returned by far-end access router. Reservation may terminate explicitly or time out. May alter reserved bandwidth but no re-routing. A B Reserve 8 Mb/s to B Select best next hop Reserve bandwidth Select path and attempt to reserve Select path and reserve Complete reservation

6 - Jonathan Turner - July 24, 2002 Soft Reservations Basic LFS provides firm reservations. »user guaranteed bandwidth until releases Can extend to provide soft reservations as well. »soft reservation can be adjusted by the network as traffic changes »can be intermixed with firm reservations to provide a firm minimum, plus more bandwidth as available Uses of soft reservation. »apps. that need guaranteed minimum and can sometimes use more, but can adjust use to what’s available »more rapidly responding congestion control for traditional best-effort traffic

7 - Jonathan Turner - July 24, 2002 Basic IP Option for LFS lengthop.Rrate flags Arate code IP header (fixed part) trace IP payload Code identifies LFS option. Operations »request firm reservation »request soft reservation »release state Flags »sender status request »sender network status request »public network status request »intra-domain status request »congested path Rrate: requested rate. Arate: allocated rate. Trace used by each domain to track usage. Allocated rate stored at “last hop” router for status generation F.P. rates with 4 bit mantissa, 4 bit exponent. »specify rates from 64 Kb/s to 4 Gb/s, 6% “granularity”

8 - Jonathan Turner - July 24, 2002 Use of Trace Field Network providers need to monitor LFS usage for network management and accounting purposes. »trace field used by ingress router of each domain to mark LFS packets with domain-specific identification »egress router of each domain maintains record of each LFS flow, including copy of trace field »end-to-end records created through off-line accounting resolution mechanisms domain Udomain Vdomain W X Y Z B A XYZ acct. record [A,B,..] thru X acct. record [A,B,..] thru Z acct. record [A,B,..] thru Y

9 - Jonathan Turner - July 24, 2002 Status Reporting Basic LFS option supports sender status and trace field for accounting. Network providers likely to want more. »sender net status allows LFS service verification »public net status allows “end-to-end” status check »intra-domain status for verifying local status »each “extra” status report requires insertion of requestor’s IP address, increasing LFS option length ISP V rcvr. LAN sender LANISP U sender status sender net status public net status intra-domain status

10 - Jonathan Turner - July 24, 2002 Partial Deployment Receivers need not be LFS-aware. »web site may use LFS to reserve bandwidth for streaming media - users benefit, even without LFS-aware hosts Issues with non-contiguous LFS domains. »route changes may create “orphan reservations” »no simple way to determine status reporter No support for non-contiguous LFS domains. »LFS router forwarding to a non-LFS router (or host) strips LFS option and implements status reporting status report includes IP address of reporting router, letting sender know how far the reservation went Public IP carrier can accept LFS option from client networks (LAN) even if client net is not LFS-aware. Clients may use tunnel to access LFS service.

11 - Jonathan Turner - July 24, 2002 Regulating LFS Use - Net Access Svc Permitting unconstrained access to LFS creates big security vulnerability. Limit use to authorized users. Limit number of reservations and amount of reserved bandwidth by authorized users. »access router keeps record and enforces limits »complication - user may use LFS from multiple locations maintain records in distributed set of servers - each server keeps records for some fraction of the users - use hashing to select Access router needs means to identify user. »host IP address insufficient (DHCP, NAT) »encryption-based authentication (IPSEC) Combine access control with usage accounting. What special issues arise with multiple domains?

12 - Jonathan Turner - July 24, 2002 LFS Video Demo Configuration Wavelet-coded video with and without LFS. »competing datagram traffic »with no reservation, lost packets cause poor video quality »with reservation, high quality preserved video source cross traffic sources cross traffic sinks video sink 100 Mb/s links

13 - Jonathan Turner - July 24, 2002 Video Demo - No Reservation video flow - no reservation datagram cross traffic flow 1 datagram cross traffic flow 2 video source cross traffic sources all sinks

14 - Jonathan Turner - July 24, 2002 Video Demo - With Reservation video flow - with reservation datagram cross traffic flow 1 datagram cross traffic flow 2 video sink cross traffic sinks

15 - Jonathan Turner - July 24, 2002 Competing LFS Flows no reservations flow 1 - no reservation flow 2 - reservation added flow 3 - no reservation reservation for flow 2 sources sinks sink 2 sink 1

16 - Jonathan Turner - July 24, 2002 Partial Reservation flow 1 - partial reservation made sink 1 flow 2 source 1 sink 3

17 - Jonathan Turner - July 24, 2002 Completing Partial Reservation flow 1 - completes partial reservation sink 1 sink 3 flow 2 - drops reservation

18 - Jonathan Turner - July 24, 2002 Addition of Flow 3 Reservation sink 3 flow 3 - adds reservation sink 2

19 - Jonathan Turner - July 24, 2002 Performance of LFS at Single Link m = number of flows link can carry exponential session times for flows, infinite queue OC-48 link can carry 200 flows of 12 Mb/s very few flows experience any delay Pareto distributed session times make little difference

20 - Jonathan Turner - July 24, 2002 Sensitivity to Load and Hop Count at 90% load, less than 1 flow in 100 delayed more than 12% of session time delay probability scales linearly with number of hops

21 - Jonathan Turner - July 24, 2002 Overload Performance with infinite buffer, no sessions get small delays (10%) with no buffer most sessions still succeed buffer reduces rejection fraction at low loads

22 - Jonathan Turner - July 24, 2002 Summary LFS provides simple reservations for QoS. »no complex signaling, wire speed setup »limited deployment can be broadly beneficial »support for usage monitoring & accounting gives network providers a motivation to deploy service Network access service for regulating usage. »preliminary specification has been developed »uses IPSEC for host/user authentication Performance analysis, simulation study underway. Routing issues. »evaluate QoS routing with multiple-choice forwarding »link state distribution for inter-domain routing »inter-domain routing policies