Internet2 Spring Member Meeting Tutorial: Internet2 101 Advanced Services and End-to-End Performance Russ Hobby This is a general overview presentation about Internet2. Internet2 is a consortium, led by US universities, which is recreating the partnership among academia, industry and government that fostered today’s Internet in its infancy. 21 February 2019

Outline Original Internet2 Intents and Reasons What we have done What we have discovered Changes in direction based on what we have learned 2/21/2019

Original Reasons for Advanced Services in Internet2 Commercial Internet could not deliver what researchers wanted Eliminate the network as a limitation for applications Re-create the sharing environment to test and develop new capabilities Transfer new technologies to the larger Internet 2/21/2019

Provide Reliable Performance Across the Network Implement Quality of Service (QoS) capabilities from Source to Destination Create a coordinated bandwidth reservation system Let applications determine the level they need Allow resource control and accounting Provide “Bandwidth Overkill” until QoS is available 2/21/2019

Support efficient “one-to-many” and “few-to-few” applications Internet2 video broadcast Support effective video conferencing Allow shared use of common data The solution: Multicast 2/21/2019

Create an National Network for Technology Development 2/21/2019

Test New Protocols and Capabilities New Network Equipment New Link Technologies Complex Routing Measurement Techniques IPv6 2/21/2019

What We Have Tried Creating a National Network for Development Implementing wide-spread Multicast Trials of QoS across the Network Tests and Training of IPv6 2/21/2019

We learned from all the experiences! What We Have Learned Some ideas have worked Some ideas are more complex than expected Some had to go back to the drawing board Some had a change in direction We learned from all the experiences! 2/21/2019

High Capacity Development Networks They Enable New Applications 2/21/2019

Gigapops have helped to complete the backbones in local and regional areas. 2/21/2019

High Capacity Development Networks Enable New Applications Many good Demonstrations Real Time Application can now work Transfers of Large Datasets Provide the Test Bed for New Technologies Packet over Sonet New Measurement Techniques Allows sharing of Ideas and Results 2/21/2019

Multicast Robust Multicast Deployment Has Been Done 2/21/2019

Why Isn’t Multicast Used More? For few to few applications it is still easier to use Application Hubs Applications Hubs provide better control One to many applications could benefit from Multicast but we have not yet reached “many” for application users. There is still the control issue. How do you know who is connected? 2/21/2019

QoS Abilene Premium Service failed to get off the ground It is a complex system that needs many parts before it can work. Some key part were not implemented for high speed links. Premium Service requires all parts of the end-to-end path to participate and coordinate. This is against the current Internet operating model. Allocation models for who gets Premium Service need a lot more work. Trials are going on with Scavenger Service Low Priority Best Effort Service Only helps conjection. Jitter. Less words 2/21/2019

IPv6 An Operational IPv6 National Backbone V6 works and is not hard An Operational IPv6 National Backbone Established Hands-on Training Workshops 2/21/2019

What Did Not Work QoS was too hard to implement at this time Even with High Bandwidth available, researchers often did not see the expected performance. WHY? The End-to-End Performance Initiative (E2Epi) was create to figure this out. 2/21/2019

The Wizard Gap 2/21/2019

Hey, this is not working right! A Problem Hey, this is not working right! Others are getting in ok Not our problem Applications Developer Applications Developer LAN Administrator LAN Administrator Talk to the other guys System Administrator Everything is AOK System Administrator Campus Networking Campus Networking The computer Is working OK No other complaints Looks fine Gigapop Gigapop All the lights are green How do you solve a problem along a path? Backbone We don’t see anything wrong The network is lightly loaded 2/21/2019

True End-to-End Experience User perception Application Operating system Host IP stack Host network card Local Area Network Campus backbone network Campus link to regional network/GigaPoP GigaPoP link to Internet2 national backbones International connections EYEBALL APPLICATION STACK JACK NETWORK . . . 2/21/2019

Good Work Already Going On Many projects and research is underway by others investigating performance on portions of the end-to-end path. E2Epi seeks to bring these efforts together to look at the end-to-end path as a complete system. 2/21/2019

Packet Loss The Performance Killer On high-speed, long-distance connections even one lost packet can greatly affect performance. For TCP the maximum transmission rate is: MSS 0.7 Rate = * As the equation shows the rate is limited by the packet size and the round-trip-time. The longer the distance the lower the rate possible. But to get the most out of that rate, the packet loss needs to be as small as possible. Packet loss can occur an any of a number of locations along the end-to-end path and not just in the network. RTT P Matt Mathis MSS = Maximum Segment Size RTT = Round Trip Time P = Packet Loss 2/21/2019

Top Three Problems - #1 Ethernet Duplex Mismatch Duplex negotiations failed One side full duplex, one side half duplex Works fine under light load Heavy load, full duplex dominates The top three problems according to a paper by Claudia DeLuna, JPL 2/21/2019

Top Three Problems - #2 Computer Tuning Buffers not set big enough, can not fill pipe Software options not set correctly Hardware limitations 2/21/2019

Top Three Problems - #3 Bad Media – Copper or Fiber Usually between wall and computer Kinked copper wire Dirty optic fibers Causes random errors and packet loss 2/21/2019

Partners Campuses Faculty and discipline communities GigaPoPs International partners NSF-sponsored engineering efforts NLANR, www.nlanr.net Web100 Project, www.web100.org Internet2 corporate members Federal labs and agencies 2/21/2019

E2Epi as a Coordination Point E2Epi has provided a rallying point for the many efforts going on in measurement and performance improvement. These groups are trying to develop a common reference framework. 2/21/2019

A View of Measurement System Components Data Collection Hardware Platform Test Programs Measurement Attributes Data Queue Data Repository Data Gathering Database Data Access Exchange Format Exchange Format Problem Reporting Contact Database Report Format Contact Method Analysis Data Retrieval Path Capabilities Problem Detection 2/21/2019

Internet2 Performance Package Application Beacons piPES E2Epi Projects Internet2 Performance Package Application Beacons piPES 2/21/2019

Internet2 Performance Package Project Establish computer performance testing procedures and minimum capabilities Gather test reports from the Internet2 community Allow computer vendors to certify Internet2 Performance Packages for sale to the community 2/21/2019

Application Beacons Provide a method to check applications performance Facilitate divide and conquer methods for locating problems 2/21/2019

Let’s Bounce it off a Beacon! Application Beacons Let’s Bounce it off a Beacon! OK you test your end Applications Developer Applications Developer LAN Administrator LAN Administrator System Administrator System Administrator Campus Networking Campus Networking Beacon Applications beacons can be used by end-users to create a “divide and conquer” technique to figure out where a performance problem lies. Gigapop Gigapop How do you solve a problem along a path? Backbone 2/21/2019

Application Beacons My side tests OK Try from your end Beacon Applications Developer Applications Developer LAN Administrator LAN Administrator System Administrator System Administrator Campus Networking Campus Networking Beacon First one side test to the application beacon that is somewhere along the path and determines if the expected performance can be achieved. Gigapop Gigapop Backbone 2/21/2019

Application Beacons Beacon Oh, My end has a lot of packets lost Applications Developer Applications Developer LAN Administrator LAN Administrator System Administrator System Administrator Campus Networking Campus Networking Beacon The other end also tests to the beacon to check performance. In this case it was determined that the problem is on the this half of the path. If other beacons are available along the path the location of the problem can be more closely determined. Gigapop Gigapop Backbone 2/21/2019

E2E piPES Project: End-to-End Performance Initiative Performance Environment System (E2E piPES) Goal: To allow end-users and network operators to determine performance capabilities, locate problems, and contact the right person to get a problem resolved. Approach: Collaborative project combining the best work of many organizations. Introduced at Spring Member Meeting. Have made good progress on architectural design. Have made good progress on outreach to other organizations and initiating collaboration. 2/21/2019

The Measurement System The New Abilene will have measurement devices as part of its structure Abilene PMP PMP PMP PMP = Performance Measurement Point (at each Abilene Node) 2/21/2019

The Measurement System Extend the System to Campuses Campus X Abilene PMP PMP PMP Campus PAS is a specified PC based computer ($2000 estimated) with a software load provided by Internet2 Campus Y PMP PMP PMP at each Campus Border 2/21/2019

Performance Reporting Perform regular tests to provide web based Performance Reports across Abilene and for Campus-Abilene paths Campus X Abilene Capability SNVA/DNVR – 10 Gbps DNVR/IPLS – 10 Gbps IPLS/NYCM – 10 Gbps Abilene PMP PMP PMP At first test results will be reported on web pages. An end-user suspecting performance problems can check the reported performance from their campus to Abilene, across Abilene, and from Abilene to the destination campus. If there are no reported problems on any of these segments, the the end-user can suspect the campus networks on the ends, or the computer systems themselves. Campus Y Campus-Abilene Capability Campus W – 155 Mbps Campus X – 612 Mbps Campus Y – 1000 Mbps Campus Z – 2 Mbps PMP PMP 2/21/2019

Reaching the End-User Provide a Testing Tool for the End-Users Computer to test across campus and beyond Campus X Abilene PMP PMP PMP Host A Host B The tool will use test results stored in a data repository as much as possible to reduce the overall amount of active testing going on. The tools will probably be a JAVA Applet to make it as portable as possible. Campus Y PMP PMP The tool reports performance of the path to a specified destination by using active tests (like to the campus border) or using stored test results from regularly scheduled tests. It also performs tests on the host system to check host tuning parameters 2/21/2019

Adding Path Information Add PMP along paths to provide additional detail Campus X Abilene PMP Gigapop S Regional Network M PMP PMP PMP Host A PMP Host B PMP E2Epi is looking for the Quilt to lead in the deployment at the Gigapops and Regional Networks. Ideally the system would have measurement information for each administrative domain along each path. For larger networks, multiple Path Measurement Points may be desireable. Campus Y PMP PMP PMP Regional Network J Gigapop T Regular tests are performed between adjacent PMP and stored in a Data Repository. This information is used by the End-User tool to reduce the number or active tests needed. 2/21/2019

segment in the path appears to have a problem Who You Gonna Call? E2E piPES will provide contact information when a segment in the path appears to have a problem Campus X Abilene PMP Gigapop S Regional Network M PMP PMP PMP Host A PMP Host B PMP E2Epi need input on how to make this work. There are operational and policy issues here. piPES Test Analysis Looks like a problem In Gigapop T. Pass these test result to: Joe@gigapopT.net Campus Y PMP PMP PMP Regional Network J Gigapop T The test results obtained by the End-User will be passed on to the contact person 2/21/2019

Access, Authentication, Authorization Roles for Access Standard End-User? NOC Staff? Network Engineer? Shibboleth for Implementation Each campus decides who in each role Others trust campus designation 2/21/2019

ARENA Allows users to understand network speeds and route options Database and web access to network link information Manually updated by network operators http://arena.internet2.edu 2/21/2019

DNS LINFO Idea: Create DNS LINFO records for interfaces that indicate link speed, max MTU, and media type Use: Modified Traceroute that will look up the LINFO and provide information on each link in the path 2/21/2019

Campus Network Infrastructure Guide Write a Guide to help campuses in building Internet2 level networks Draw upon the experiences of others Provide advise on various technologies 2/21/2019

E2Epi Web Site Case Studies Lists of Tools Other Projects working on Performance A Place to Communicate E2Epi-Interest Email list 2/21/2019

Network Magazine Article Strategies & Issues: Measuring End-to-End Internet Performance Discusses major cause of performance degradation. http://www.networkmagazine.com/article/NMG20030403S0006 2/21/2019

Future Works More use of passive measurements Analysis techniques for measurement data Make the Internet work as a system 2/21/2019

Peer-to-Peer and Grid Systems Peer-to-Peer (P2P) and Grid Systems are global computers that use the network as the computer backplane End-to-End Performance can greatly affect their usability. P2P is a developing area that will affect and be affected by E2Epi 2/21/2019

Network Security Initial Internet Premise: Security belongs on the end-system Development of security capabilities has been slow. Devices have been put into the network to try to help Firewalls, NAT boxes, Filters These often impede Applications. Network devices themselves need to be protected 2/21/2019

2/21/2019