Internet2 Update R/D and Infrastructure Guy Almes Internet2 Project NANOG Meeting Dearborn — 9 June 1998

Outline of the Talk  Technical Working Groups  The Challenge of Delay-Bandwidth Products  Abilene Project Update

Applications and Engineering Applications Engineering MotivateEnables

Comments on Apps and Plumbing  Advanced applications transform high-speed plumbing into value  Advanced plumbing enables advanced applications  Profligate use of bandwidth, per se, does not make an application ‘advanced’  Megalomaniac plumbing, per se, does not make the plumbing ‘advanced’

Technical Working Groups  IPv6  Measurement  Multicast  Network Management  Network Storage  Quality of Service  Routing  Security  Topology

IPv6  Chair: Dale Finkelson, Univ Nebraska  Membership: Total 12; 9.edu, 3.com, 1.gov  Focus: Explore the rôle that IPv6 might play in the Internet2 project Work with those interested in IPv6 to build IPv6 testbeds across the Internet2 structure, including vBNS and Abilene

Measurement  Chair: David Wasley, Univ California  Focus: Places to measure:  at campuses, at gigaPoPs, within interconnect(s) Things to measure  traffic utilization  performance: delay and packet loss  traffic characterization

One example measurement technology  IETF IPPM WG defining one-way delay  Take all delay to be due to: Propagation Transmission Queuing  Variation in delay suggests congestion

Multicast  Chair: vacant [Dave Meyer, Univ Oregon still serving] Nearing completion of naming a successor  Membership: Total 3; 3.edu  Focus: Make native IP multicast scalable and operationally effective

Network Management  Chair: Mark Johnson, MCNC  Membership: Total 4; 3.edu, 1.com  Focus: Common trouble ticket system How can all our interconnects and gigaPoPs and universities appear to be a seamless whole?

Network Storage  Chair: Micah Beck, Univ Tennessee  Membership: Total 13; 9.edu, 4.com  Focus: Distributed Storage Infrastructure for Internet2 Replication Physical proximity Transparency

Quality of Service  Chair: Ben Teitelbaum, Internet2 staff  Membership: Total 36; 17.edu, 19.com  Focus: Multi-network IP based QoS Relevant to advanced applications Interoperability: carriers and kit Scalable Administratable and Measurable Hosts, campus/gigaPoP/Interconnect routers/switches

Quality of Service Sketch Does the approach support advanced applications? Are there implementations that work? Only one? If cloud ‘A’ and cloud ‘B’ both implement QoS, does the combined A+B catenation implement QoS? AB

QoS, continued  Results to date: Requirements document Series of technical recommendations  First Internet2 Joint Applications/ Engineering QoS Workshop Santa Clara, California May 21-22, 1998 Hosted by Bay Networks

Routing  Chair: Steve Corbato, Univ Washington  Membership: Total 48; 32.edu, 16.com  Focus: Internal and External routing Critical issues  gigaPoP internal routing design  Explicit routing requirement (the “fish problem”)  Met at UCSD in January (21 attendees)  gigaPoP external routing recommendations  Subscribers (Internet2 campuses)  National interconnects (vBNS, Abilene, and NGI networks)

Security  Chair: Peter Berger, Carniege Mellon Univ  Membership: Total 13; 13.edu  Focus: Authentication Application to QoS Application to Digital Libraries

Topology  Chair: Paul Love, Internet2 staff  Membership: Total 16; 13.edu, 2.com, 1.gov  Focus: Topology of Internet2 Internal Internet2 Connections Internet2 with other Advanced Research Networks

Summary  Internet2’s WGs focus on project’s needs  Complement IETF WGs  Membership by invitation - welcome participation by Internet2 corporate members

Large Delay-Bandwidth Products  As the product of delay and bandwidth grows: The number of unacknowledged packets grows It becomes more difficult to sustain a steady stream of data from end to end  Several consequences: Need for direct physical paths Tradeoff between buffering and variation in delay

A pessimistic result from Mathis et al.  Mathis, Semke, Mahdavi, and Ott, "The Macroscopic Behavior of the TCP Congestion Avoidance Algorithm", Computer Communication Review, July   BW  C * packet-size / (delay *  packet- loss)

Consider the implications for the international high-performance Internet  BW  packet-size  BW  1 / delay  BW  1 /  packet-loss

Example: Delay BW  C / delay delay due to distance original raw bandwidth

Example: Delay with fatter pipe BW  C / delay delay due to distance more raw bandwidth

Example: Packet Loss  similar phenomenon, but …  to double bandwidth, you must  cut packet loss by four

Abilene Update  UCAID Project  Addresses infrastructure needs of Internet2

Goals and Objectives  Provide high-quality, widely available Interconnect among participating gigaPoPs/universities  Connect to Internet2 members via the vBNS and to other key research/ education sites via Internet2/NGI- class federal and non-US nets

Goals and Objectives, continued  Support QoS architecture as it evolves  Support other advanced functionality as it evolves  Maximize Robustness  Minimize Latency  Provide Capacity to Avoid Congestion

Evolution of Abilene with Time  Phase 1: use of operational Qwest Sonet  Phase 2: use of separate wavelengths  Phase 3: use of separate fibers  Allows capacity to grow with Internet2 needs

Key Attributes  IP over Sonet  Benefit from Qwest OC-48 Sonet capacity and collocation sites  Benefit from Nortel OC-192 Sonet kit and Lucent fiber  Benefit from Cisco GSR routers

Architecture: Core  About 11 (up to 30) core nodes Each located at a Qwest PoP Each with a Cisco router Rack also contains measurements/ management computers  Interior lines connect core nodes OC-12 and (eventually) OC-48 Sonet IP-over-Sonet interfaces

sttl eugn scrm anhm phnx albq hstn nwortlhs atln rlgh wash phil nycm bstnsyrc clev pitb dtrt milwchcg mpls kscydnvr slkc ipls lsvl nsvl rcmd Subset of Route Map of Interest to Abilene tpka alby elpa

Attitude toward interior lines  Robustness: mesh plus Sonet  Latency: direct physical paths  Capacity: avoid congestion

Architecture: Access  Access node at many Qwest PoPs Qwest Sonet switches needed equipment  Access lines connect from core node to gigaPoP Local part: gigaPoP to access node Long distance part: access node to core node IP-over-Sonet or IP-over-ATM possible OC-3 and OC-12 typical

One cost-sharing implication  Long-distance part of access line is considered part of the ‘backbone’  Thus, number/location of core nodes does not affect costs borne by gigaPoP

One robustness implication  Each access line is Sonet  Long-distance part (at least) will be configured from protected Sonet ring  Thus, single access line can tolerate a break in the long-distance part of the access line

OK, so where’s the map?  Self-selection is key  Each gigaPoP will determine where, when, at what speed it connects  Detailed topology will be based on engineering considerations