1. Internet2 Intel Partnership Planning Meeting November 19,2001

2. 2 What Is Internet2? A project of the university community working with our corporate colleagues and government to close the gap between the potential and reality of the Internet

3. 3 Internet2 Universities 188 Universities as of November 2001

4. 4 Internet2 Mission Develop and deploy advanced network applications and technologies, accelerating the creation of tomorrow’s Internet.

5. 5 Internet2 Goals Enable a new generation of applications Re-create leading edge R&E network capability Transfer technology and experience to the global production Internet

6. 6 Today’s Internet Doesn’t Provide reliable end-to-end performance Encourage cooperation on new capabilities Allow testing of new technologies Support development of revolutionary applications

7. 7 Why University Leadership? The Internet came from the academic community Stanford -- the Internet protocols NSFNet -- the scaled-up Internet CERN -- the WWW protocols University of Illinois -- the Web browser Universities’ research and education mission require an advanced Internet and have demonstrated they can develop it

8. 8 Technology Transfer Conduits Collaborating on advanced applications Deploying pre-commercial infrastructure and protocols Establishing expertise and human capital Large-scale proof of concept

9. 9 Research and Development Commercialization Partnerships Privatization Internet Development Spiral Today’s Internet Internet2 NSFNetARPANet NYSERNet SURANet MichNet ANS/Core PSI UUNet InternetMCI AOL GigaBit Testbeds MBone NGI Intelligent Networks Source: Ivan Moura Campos

10. 10 Internet2 Activities and Focus Areas Advanced Network Infrastructure Middleware Engineering Advanced Applications End-to-End Performance Advanced Network Management Partnerships

11. 11 Internet2 Activities and Focus Areas Advanced Network Infrastructure

12. 12 Internet2 Network of the Future Current state of Abilene Evolution of optical networking Next phase of Abilene

13. 13 Abilene background & milestones Abilene is a UCAID project in partnership with Qwest Communications Nortel Networks Cisco Systems Indiana University ITECs in North Carolina and Ohio Timeline Apr 1998: Project announced at White House Jan 1999: Production status for network Oct 1999: IP version of HDTV (215 Mbps) over Abilene Apr 2001: First state education network added Jun 2001: Participation reaches all 50 states & D.C. Nov 2001: Raw HDTV/IP (1.5 Gbps) over Abilene

14. 14 Abilene focus Enabling innovative applications and services not possible over the commercial Internet Advanced service efforts Multicast IPv6 QoS Measurement Security –DDoS detection efforts (Arbor Networks & Asta Networks)

15. 15 Abilene status – November, 2001 IP-over-SONET (OC-48c) backbone 54 direct connections 3 OC-48c (2.5 Gbps) connections 22 will connect via at least OC-12c (622 Mbps) by year end 200+ primary participants All 50 states, District of Columbia, & now Puerto Rico 15 regional GigaPoPs support ~70% of participants 37 sponsored participants 15 state education networks (SEGPs) Collaboration of sponsoring member universities and Abilene connectors

16. 16

17. 17

18. 18 International peering Transoceanic R&E bandwidths growing! Key international exchange points facilitated by Internet2 membership and the U.S. scientific community STARTAP  STAR LIGHT – Chicago Pacific Wave – Seattle AMPATH – Miami New York City – EP under development CUDI - CENIC and Univ. of Texas at El Paso International transit service

19. 19 Measurement and DDoS Traffic characterization (Ohio ITEC) Network utilization by SEGPs and Abilene ITN Abilene Scavenger Service policing GigaPoP pair hotspot identification Passive measurement Planned for Indianapolis router backbone links Collaboration with SDSC Distributed Denial of Service detection Strong IU Global NOC interest Asta Networks (UCSD/U of Washington roots) Arbor Networks (U of Michigan/Merit roots) Data privacy and anonymity policy

20. 20 Network of the Future: Context for the next backbone Computational science as an emerging interdisciplinary field Bandwidth and distributed sensing capability as the next critical parameters –Complement CPU, memory & storage Increasingly distributed data collection and storage NSF Distributed Terascale Facility solicitation Emergence of optical technologies Dense Wave Division Multiplexing (DWDM) Important distinction: optical transport vs. switching Much new transcontinental conduit and fiber in place; a lot of business plans abandoned… Glut of fiber & conduit – but not bandwidth

21. 21 Current state of optical networking Dense Wave Division Multiplexing (DWDM) Current systems can support >160 10-Gbps ’s (1.6 Tbps!) Optical growth can overwhelm Moore’s Law (routers) Costs scale dramatically with distance Three possible scenarios for the future Enhanced IP transport (higher BW and circuit multiplicity) Fine-grained traffic engineering –p2p links between campuses, HPC centers, & Gigapops Physical manifestation of switched circuits (a la ATM SVCs) Evolution of optical switching will be critical Leading international efforts in R&E exploration The Netherlands, Canada, STAR LIGHT (Chicago)

22. 22 National optical networking options 1 - Incremental wavelengths Provision 10-Gbps ’s from provider(s) in the same way that SONET circuits are done for Abilene now Exploit smaller incremental cost of additional ’s 2 - Dim Fiber Acquisition of fiber IRU and subsequent O&M agreement for inter-PoP services (amps, regenerators, DWDMs?) National footprint of 1-2 fiber pairs –IRU would cost $10-20M Most likely awaits the availability of lower-cost optical transmission equipment

23. 23 Future of Abilene Original UCAID/Qwest MoU amended on October 1, 2001 Extension of Qwest’s original commitment to Abilene for another 5 years – 10/01/2006 Originally expired March, 2003 Upgrade of Abilene backbone to optical transport capability - ’s x4 increase in the core backbone bandwidth –OC-48c SONET (2.5 Gbps) to 10-Gbps DWDM Capability for flexible provisioning of ’s to support future point-to-point experimentation & other projects

24. 24 Key aspects of the next backbone IPv6 Running natively concurrently with IPv4 Replicate multicast deployment strategy Motivations –Resolving IPv4 address exhaustion issues –Preservation of the original End-to-End Architecture –International collaboration –Router and host capabilities Close collaboration with Internet2 IPv6 Working Group Network resiliency MPLS/TE fast reroute or IP-based IGP fast convergence Opportunity for new measurement capabilities Support of End-to-End Performance Initiative

25. 25 Next generation network deployment October, 2001: Detailed technical design starts February, 2002: PoP upgrades start deployment in three phases April, 2002 – Phase 1 October, 2002 – Phase 2 April, 2003 – Phase 3 October 2003 - Completion of 10-Gbps upgrade

26. 26 Network design overview Overall next generation topology is expected to be very similar to current design Previous iterations to router locations –Washington DC, Chicago, Sunnyvale, Houston Some differences expected due to Qwest DWDM deployment Expect same number of backbone routers

27. 27 Optical fanout Next generation architecture: Regional & state based optical networking projects are critical Three-level hierarchy: backbone, GigaPoPs, campuses CENIC ONI, I-WIRE, SURA Crossroads, Indiana, Ohio Pacific/Northwest Gigapop and PREN are relevant players in the Northwest Collaboration with the Quilt Regional Optical Networking project Carrier DWDM access is now not nearly as widespread as with SONET circa 1998

28. 28 The Quilt A UCAID project support regional advanced networking initiatives 15 charter GigaPoPs EDUCAUSE and SURA Quilt GigaPoPs support over 70% of Abilene participants Initial projects Commodity Internet Services Regional Optical Networking Measurement Led by Wendy Huntoon (Pittsburgh SC)

29. 29 Conclusions Abilene future UCAID’s partnership with Qwest extended through 2006 Backbone to be upgraded to 10-Gbps in three phases starting spring 2002 Capability for flexible provisioning in support of future experimentation in optical networking Overall approach to the new technical design and business plan is for an incremental, non-disruptive transition Follow-on network most likely will be developed around national dark fiber facility and will utilize next generation optical transport technology

30. 30 For more information Web: www.internet2.edu/abilenewww.internet2.edu/abilene E-mail: abilene@internet2.eduabilene@internet2.edu

31. 31 Engineering Emphases Internet2 Activities and Focus Areas

32. 32 Engineering: Advanced Functionality Multicast IPv6 QoS

33. 33 Internet2 Multicast Kevin Almeroth, Univ California Santa Barbara, chair Increasingly pervasive high-quality deployment of native IP multicast throughout the Internet2 infrastructure Keeping an eye on SSM Implications of SSM on scalability, manageability Adapting applications to make use of SSM Clarifying the application story Internet2's multicast infrastructure is a valuable sand box in which to test the value of new multicast applications

34. 34

35. 35 Internet2 IPv6 Dale Finkelson, Univ Nebraska, Michael Lambert, PSC, co-chairs Build the Internet2 IPv6 infrastructure Currently, based on v4-over-v6 tunnels Planned as first-class service with the coming 10 Gb/s upgrade of Abilene Educate campus network engineers to support IPv6 Explore the Motivation for IPv6 within the Internet2 community Make IPv6 'real' within the university community (and to our students)

36. 36

37. 37 Internet2 QoS Ben Teitelbaum, Internet2 staff, chair QBone Premium Service Hard priority service for selected streams Very hard due to need for policing/administration Scavenger Service Voluntary less-than-best-effort service Enables unconscionable bulk data transfers without threatening performance of best-efforts traffic Other 'non-elevated' services E.g., delay- vs loss-sensitive best effort service Interoperability without policing / administration

38. 38 Internet2 Measurements Matt Zekauskas, Internet2 Staff, chair Define architecture: Usage Active Measurements of Performance Passive Measurements Uniform Access to Results Contributing to Measurement Infrastructure for the End-to- end Performance Initiative

39. 39 Active Measurements within Abilene Surveyors with: Active delay/loss measurements Ad hoc throughput tests

40. 40 Application to Performance Debugging

41. 41 Application to Performance Debugging

42. 42 Divide and Conquer Systematically identify/isolate the network segment at fault Can we make this systematic and (eventually) automated?

43. 43 Internet2 Activities and Focus Areas End-to-End Performance

44. 44 Why the End-to-End Performance Initiative? Even with high bandwidth network links, the Internet2 community often does not see expected performance.

45. 45 The Wizard Gap

46. 46 The E2Epi Mission To enable the researchers, faculty, students and staff who use high performance networks to obtain optimal performance from the current infrastructure on a consistent basis. Raw Connectivity Applications Performance

47. 47 True End-to-End Experience User perception Application Operating system Host IP stack Host network card Local Area Network Campus backbone network Campus link to regional network/GigaPoP GigaPoP link to Internet2 national backbones International connections EYEBALL APPLICATION STACK JACK NETWORK...

48. 48 The Problem Applications Developer System Administrator LAN Administrator Campus Networking Gigapop BackboneCampus Networking LAN Administrator System Administrator Applications Developer How do you solve a problem along a path? Hey, this is not working right! The computer Is working OK Talk to the other guysEverything is AOK No other complaints The network is lightly loaded All the lights are green We don’t see anything wrong Looks fine Others are getting in ok Not our problem

49. 49 First Steps Workshop in Ann Arbor on 9 January, 2001 40+ participants Each participant provided a short paper on “What does E2EPerformance Mean?” Planned agenda was not used in order to respond to more pressing issues from participants. Design team formed to create an overall vision paper.

50. 50 Areas of the Initiative Applications Host/OS Tuning Measurement Infrastructure Performance Improvement Environment (PIE) Operations and Human Communications Performance Evaluation and Review Framework (PERF)

51. 51 Applications Work with specific application communities to help solve their performance problems. High Energy Physics Medical Sciences – Visible Human Project Use a few key, general purpose applications for performance testing. FTP Video Conferencing

52. 52 Host/OS Tuning Web100 has a leading role Provide Best Practices for getting the most from your computer. Locate or build tools for Host/OS performance diagnostics. Work with OS vendors on tuning capabilities Work with computer vendors on Internet2 Performance Packages.

53. 53 Measurement Infrastructure Bring together current measurement efforts and projects in the community. Establish an End-to-End Measurement Infrastructure from the intersection of these works. Create diagnostic tools to determine the health of the network and locate performance problems.

54. 54 Standard Operational Info Applications Developer System Administrator LAN Administrator Campus Networking Gigapop BackboneCampus Networking LAN Administrator System Administrator Applications Developer Information from All Parts of the Network Ops Info

55. 55 Standard Operational Info Applications Developer System Administrator LAN Administrator Campus Networking Gigapop BackboneCampus Networking LAN Administrator System Administrator Applications Developer Know the Health of the Network Ops Info End-to-End Analyzer

56. 56 Standard Operational Info Applications Developer System Administrator LAN Administrator Campus Networking Gigapop BackboneCampus Networking LAN Administrator System Administrator Applications Developer Applications Adapt to the Network Ops Info End-to-End Analyzer Apps Tuning

57. 57 Performance Improvement Environment (PIE) Develop a dynamic environment where collaboration and information sharing will happen. Identify, collect and disseminate appropriate information for end-to-end related information. Include success stories,measurement statistics, reference materials, measurement tool documentation. Include pointers to materials already developed by other communities.

58. 58 Operations and Human Communications Establish communications among common interest groups System administrators LAN administrators Campus NOCs GigaPoP Application support staff Establish communications between groups for operations and problem resolution.

59. 59 Groups of Common Interest Applications Developer System Administrator Campus Networking Gigapop BackboneCampus Networking LAN Administrator System Administrator Applications Developer Provide a means of communications Let them share experiences. I don’t know how to solve this problem! I do! LAN Administrator

60. 60 Find a Solution? Applications Developer System Administrator LAN Administrator Campus Networking Gigapop BackboneCampus Networking LAN Administrator System Administrator Applications Developer A System to Check a Specific Problem Hmm, Time to Check the PIE and Talk to others Performance OK here Throughput OKEverything is Still AOK Not a bottleneck At this point The network is still lightly loaded All the lights are green We don’t see anything wrong Yup. Duplex Does not Agree! It is slow for others too! Ah ha, an Ethernet Duplex problem!

61. 61 Can You Go Direct to the Problem? How can you tell where is the problem? Need a tool to tell you: Where the problem is. The type of problem Who to contact to get it fixed Terry Gray, University of Washington “We Need a Finger Pointing Tool”

62. 62 Gray Finger Pointing Tool Applications Developer System Administrator LAN Administrator Campus Networking Gigapop BackboneCampus Networking LAN Administrator System Administrator Applications Developer Locate the Problem Gray Finger Pointing Tool Terry Gray University of Washington Finger Pointing Tool Time!

63. 63 Gray Finger Pointing Tool Applications Developer System Administrator LAN Administrator Campus Networking Gigapop BackboneCampus Networking LAN Administrator System Administrator Applications Developer Finger Pointing Tool Time You ARE the Weakest Link! OK, I’ll fix it

64. 64 Performance Evaluation and Review Framework (PERF) Establish a framework for resolving performance problems Finger Pointing Tool Provide known solutions by using the PIE Tap community knowledge by facilitating group communications Coordinate a team of experts to solve hard problems

65. 65 The Hard Problems Applications Developer System Administrator LAN Administrator Campus Networking Gigapop BackboneCampus Networking LAN Administrator System Administrator Applications Developer What if no one has the answer? Hey, Its is not working again! I don’t know what is wrong No clue hereI’m stumpedThis is strange It looks normal here Doh! We can’t figure it out I am Cluefully Challenged Others are getting in ok I don’t know Need a Tiger Team of Experts to Research the Problem

66. 66 Anticipated Partners Faculty and discipline communities Campuses GigaPoPs International partners Research projects in performance Internet2 corporate members Federal labs and agencies

67. 67 Calls For Participation Identify core applications and services Seek stories and best practices Current Call for Experiences Seek participants in the various work areas Internet2 E2Epi Measurement Workshop Tempe, AZ, 27-28 January 2002 Campus Participation in E2Epi

68. 68 Internet2 Organization Role Staffing Cheryl Munn-Fremon, Initiative Director Russ Hobby, Technical Architect George Brett, Information Architect Lisa Wilberding, Communications Coordinator Terri Saarinen, Program Assistant

69. 69 For More Information E2Epi Info-E2Epi@internet2.edu http://www.internet2.edu/e2epi

70. 70 Internet2 Activities and Focus Areas Advanced Network Management

71. 71 Advanced Network Management Layer 2 QueryProtocol Steven Wallace Mark Meiss Indiana University Advanced Network Management Laboratory

72. 72 Presentation Overview Why is there a need for layer 2 visualization Why use a proxy agent? Why host the proxy in the router? How does this work?

73. 73 The Need for Layer 2 Visualization Many end-to-end performance problems are caused by defects in the end- system’s broadcast domain Frequently caused by duplex mismatches Topology of broadcast domains typically not known Hop by hop analysis requires you to know the hops

74. 74 Why Use a Proxy Agent To Implement this Protocol? Frequently network engineers from “other” organizations assist in network troubleshooting Rather than open SNMP access to some other organization, develop a service and related protocol to allow a remote engineer to safely determine the broadcast domain topology and health

75. 75 Why Host the Proxy in the Router? The router is in the path of the probe (which is in the form of a traceroute probe), and will automatically route the packet to the general purpose CPU due to the expired TTL The router “knows” something about the broadcast domain and is a good point of instrumentation.

76. 76 How Does Layer 2 Query Protocol Work? Assumes broadcast domain made up of SNMP managed switches that support the standard bridge MIB Switches discovered via an IP broadcast SNMP query ARP information retrieved for all of the switch SNMP agents by querying the router Subsets of the Ethernet forwarding tables are retrieved from the switches SNMP agents Topology is calculated

77. 77 The Protocol Initial request looks like a traceroute probe with the addition of a special signature and three arguments: client’s IP address, port number, and initial query sequence number Router-based agent establishes a TCP connection to the client on the given port and sends the initial query sequence number Client and agent exchange commands over the TCP connection. Responses from the router are encoded in XML

78. 78 A Working Client Implemented in Java

79. 79 More Internet2 Information On the Web www.internet2.edu Email info@internet2.edu

80. 80 Internet2 Contact Information Guy Almes: almes@internet2.edualmes@internet2.edu Jill Arnold: jillar@internet2.edujillar@internet2.edu Steve Corbato: corbato@internet2.educorbato@internet2.edu Ted Hanss: ted@internet2.eduted@internet2.edu Russ Hobby: rdhobby@internet2.edurdhobby@internet2.edu Doug Van Houweling: dvh@internet2.edudvh@internet2.edu Steve Wallace: ssw@indiana.edussw@indiana.edu

81. www.internet2.edu