E2E piPEs / AMI / OWAMP Status Eric Boyd and Jeff Boote April 10, 2003 This is a general overview presentation about Internet2. Internet2 is a consortium, led by US universities, which is recreating the partnership among academia, industry and government that fostered today’s Internet in its infancy.

Agenda Motivation piPEs / AMI Overview piPEs / AMI Deployment Plan OWAMP Release Demo BOF, feel free to ask questions, make comments. 5/5/2019

Context for E2E Performance Even with high bandwidth network links, the Internet2 community often does not see expected performance. 5/5/2019

The Wizard Gap 5/5/2019

Hey, this is not working right! A Problem Hey, this is not working right! Others are getting in ok Not our problem Applications Developer Applications Developer LAN Administrator LAN Administrator Talk to the other guys System Administrator Everything is AOK System Administrator Campus Networking Campus Networking The computer Is working OK No other complaints Looks fine Gigapop Gigapop All the lights are green How do you solve a problem along a path? Backbone We don’t see anything wrong The network is lightly loaded 5/5/2019

Good Work Already Going On Many projects and research is underway by others investigating performance on portions of the end-to-end path. E2Epi seeks to bring these efforts together to look at the end-to-end path as a complete system. 5/5/2019

Can You Go Direct to the Problem? How can you tell where is the problem? Need a tool to tell you: Where the problem is. The type of problem Who to contact to get it fixed Terry Gray, University of Washington “We Need a Finger Pointing Tool” 5/5/2019

Gray Finger Pointing Tool Finger Pointing Tool Time! Applications Developer Applications Developer LAN Administrator LAN Administrator System Administrator System Administrator Campus Networking Campus Networking Gigapop Gigapop Locate the Problem Gray Finger Pointing Tool Terry Gray University of Washington Backbone 5/5/2019

Gray Finger Pointing Tool Finger Pointing Tool Time OK, I’ll fix it Applications Developer Applications Developer LAN Administrator LAN Administrator You ARE the Weakest Link! System Administrator System Administrator Campus Networking Campus Networking Gigapop Gigapop Backbone 5/5/2019

Agenda Motivation piPEs / AMI Overview piPEs / AMI Deployment Plan OWAMP Release Demo 5/5/2019

BNI AMI Project: BNI Abilene Measurement Infrastructure Goal: Instrument Next Generation Abilene network with extensive performance measurement capabilities Approach: Collaborative project among BNI, Engineering, and E2E. 5/5/2019

E2E piPES Project: End-to-End Performance Initiative Performance Environment System (E2E piPES) Goal: To allow end-users and network operators to determine performance capabilities, locate problems, and contact the right person to get a problem resolved. Approach: Collaborative project combining the best work of many organizations. Introduced at Spring Member Meeting. Have made good progress on architectural design. Have made good progress on outreach to other organizations and initiating collaboration. 5/5/2019

The Measurement System Extend the System to Campuses Campus X Abilene PMP PMP PMP Campus PAS is a specified PC based computer ($2000 estimated) with a software load provided by Internet2 Campus Y PMP PMP PMP at each Campus Border 5/5/2019

The Measurement System The New Abilene will have measurement devices as part of its structure Abilene PMP PMP PMP PMP = Performance Measurement Point (at each Abilene Node) 5/5/2019

Performance Reporting Perform regular tests to provide web based Performance Reports across Abilene and for Campus-Abilene paths Campus X Abilene Capability SNVA/DNVR – 10 Gbps DNVR/IPLS – 10 Gbps IPLS/NYCM – 10 Gbps Abilene PMP PMP PMP At first test results will be reported on web pages. An end-user suspecting performance problems can check the reported performance from their campus to Abilene, across Abilene, and from Abilene to the destination campus. If there are no reported problems on any of these segments, the the end-user can suspect the campus networks on the ends, or the computer systems themselves. Campus Y Campus-Abilene Capability Campus W – 155 Mbps Campus X – 612 Mbps Campus Y – 1000 Mbps Campus Z – 2 Mbps PMP PMP 5/5/2019

Reaching the End-User Provide a Testing Tool for the End-Users Computer to test across campus and beyond Campus X Abilene PMP PMP PMP Host A Host B The tool will use test results stored in a data repository as much as possible to reduce the overall amount of active testing going on. The tools will probably be a JAVA Applet to make it as portable as possible. Campus Y PMP PMP The tool reports performance of the path to a specified destination by using active tests (like to the campus border) or using stored test results from regularly scheduled tests. It also performs tests on the host system to check host tuning parameters 5/5/2019

Adding Path Information Add PMP along paths to provide additional detail Campus X Abilene PMP Gigapop S Regional Network M PMP PMP PMP Host A PMP Host B PMP E2Epi is looking for the Quilt to lead in the deployment at the Gigapops and Regional Networks. Ideally the system would have measurement information for each administrative domain along each path. For larger networks, multiple Path Measurement Points may be desireable. Campus Y PMP PMP PMP Regional Network J Gigapop T Regular tests are performed between adjacent PMP and stored in a Data Repository. This information is used by the End-User tool to reduce the number or active tests needed. 5/5/2019

segment in the path appears to have a problem Who You Gonna Call? E2E piPES will provide contact information when a segment in the path appears to have a problem Campus X Abilene PMP Gigapop S Regional Network M PMP PMP PMP Host A PMP Host B PMP E2Epi need input on how to make this work. There are operational and policy issues here. piPES Test Analysis Looks like a problem In Gigapop T. Pass these test result to: Joe@gigapopT.net Campus Y PMP PMP PMP Regional Network J Gigapop T The test results obtained by the End-User will be passed on to the contact person 5/5/2019

E2E piPES Architecture 5/5/2019

Testing / Analysis Engine Problem: Encode Matt Z.’s Brain Open Questions: What are measurement results are “acceptable” for a given application (family)? What tools generate those results? How do you handle incomplete data? How do you rank multiple result-generators? What is the iterative decision tree to understand E2E problem? 5/5/2019

Measurement Schema Measurement Types? e.g. BW, Latency, Loss, Jitter Measurement Units? e.g. Seconds vs. Microseconds Map Tools to Measurements? e.g. OWAMP -> 1-way Latency Measurement Metadata? Database Table Design? 5/5/2019

Access, Authentication, Authorization Roles for Access Standard End-User? Near Neighbor (Test Buddy)? NOC Staff / Network Engineer? Shibboleth for Implementation Each campus decides who in each role Others trust campus designation 5/5/2019

Other R&D Issues Design system to scale (eliminate centralization)? How to “discover” PMPs? How to insert On-Demand tests into Regularly-Scheduled test set Balance centralization and distributed database requirements 5/5/2019

Agenda Motivation piPEs / AMI Overview piPEs / AMI Deployment Plan OWAMP Release Demo 5/5/2019

E2E piPES Architecture v1.0 5/5/2019

piPEs / AMI Rollout 5/5/2019

piPEs / AMI Rollout 5/5/2019

piPEs / AMI Rollout 5/5/2019

piPEs / AMI Rollout 5/5/2019

piPEs / AMI Rollout 5/5/2019

piPEs / AMI Development & Testing Environment 5/5/2019

piPEs / AMI Internal Resources Eric Boyd – piPEs development, project coordination Jeff Boote – OWAMP, piPEs development Prasad Calyam – piPEs development Chris Heermann – AMI Matt Zekauskas - AMI Susan Evett - Documentation Russ Hobby – Campus Deployment George Brett – Schema, “Grid Service” 5/5/2019

piPEs External Resources Yee-Ting Li – UCL (50%) Paul Mealor – UCL (up to 100%) PhD Student – UCL (contingent on grant) Warren Matthews – SLAC Jim Ferguson et al – NLANR / DAST Many early adopter volunteers 5/5/2019

Agenda Motivation piPEs / AMI Overview piPEs / AMI Deployment Plan OWAMP Release Demo 5/5/2019

Latency Tests Problem: Existing platforms are not interoperable (SURVEYOR,RIPE,…) Solution: Standards 5/5/2019

One-Way-Active-Measurement-Protocol OWAMP One-Way-Active-Measurement-Protocol Possible due to growing availability of good time sources Wide deployment of “open” servers would allow measurement of one-way delay to become as commonplace as measurement of RTT using ICMP tools such as ping. Current Draft: draft-ietf-ippm-owdp-05.txt Shalunov,Teitelbaum,Zekauskas Sample implementation under development Beta Release: http://owamp.internet2.edu/ 5/5/2019

OWAMP Implementation Basically: NTP system call interface Multiple processes for recv/send loops Written as an API to allow one-off implementations 5/5/2019

OWAMP Error Estimates Rate = 1 packets/sec Min/Median = 59/64 usec 5/5/2019

OWAMP Error Estimates Rate = 10 packets/sec Min/Median = 57/63 usec 5/5/2019

OWAMP Error Estimates Rate = 100 packets/sec Min/Median = 54/73 usec 5/5/2019

OWAMP Error Estimates Rate = 1000 packets/sec Min/Median = 53/176 usec 5/5/2019

OWAMP Errors Preliminary Findings: Min error estimates look to be in the 55-60 usec range. Serialization Delay: ~5usec x 2 Get Timestamp: ~15usec x 2 Additional error is: Time from userland “send” to 1st byte hits the wire Time from kernel has packet to userland “recv” returns Potentially recv process data processing before calling “recv” 5/5/2019

OWAMP implementation status Sample implementation http://owamp.internet2.edu/ Beta Release: No “policy” No authentication/encryption FreeBSD only 5/5/2019

Abilene OWAMP deployment 2 overlapping full meshes (IPv4 & IPv6) 11 measurement nodes = 220 ongoing tests UDP singletons Rate: 10 packets/second* Packetsize: (32 byte payload)* Results are continuously streamed back to “Measurement Portal” for long-term archive and data dissemination (Near real-time) * These are the values we are currently we are planning on using, but this is not set in stone. 5/5/2019

Agenda Motivation piPEs / AMI Overview piPEs / AMI Deployment Plan OWAMP Release Demo 5/5/2019

piPEs / AMI / OWAMP Demo http://owamp.internet2.edu when available 5/5/2019

More Internet2 Information On the Web www.internet2.edu e2epi.internet2.edu Email Info-e2epi@internet2.edu For more information about Internet2, please see these web sites, or contact info@internet2.edu directly by email. 5/5/2019

Questions? 5/5/2019

www.internet2.edu