1. E2E piPEs / AMI / OWAMP Status
Eric Boyd and Jeff Boote
April 10, 2003
This is a general overview presentation about Internet2. Internet2 is a consortium, led by US universities, which is recreating the partnership among academia, industry and government that fostered today’s Internet in its infancy.

2. Agenda Motivation piPEs / AMI Overview piPEs / AMI Deployment Plan
OWAMP Release
Demo
BOF, feel free to ask questions, make comments.
5/5/2019

3. Context for E2E Performance
Even with high bandwidth network links, the Internet2 community often does not see expected performance.
5/5/2019

4. The Wizard Gap
5/5/2019

5. Hey, this is not working right!
A Problem
Hey, this is not working right!
Others are
getting in ok
Not our problem
Applications
Developer
Applications
Developer
LAN
Administrator
LAN
Administrator
Talk to the other guys
System
Administrator
Everything is
AOK
System
Administrator
Campus
Networking
Campus
Networking
The computer
Is working OK
No other
complaints
Looks fine
Gigapop
Gigapop
All the lights
are green
How do you solve
a problem along a path?
Backbone
We don’t see
anything wrong
The network is lightly loaded
5/5/2019

6. Good Work Already Going On
Many projects and research is underway by others investigating performance on portions of the end-to-end path.
E2Epi seeks to bring these efforts together to look at the end-to-end path as a complete system.
5/5/2019

7. Can You Go Direct to the Problem?
How can you tell where is the problem?
Need a tool to tell you:
Where the problem is.
The type of problem
Who to contact to get it fixed
Terry Gray, University of Washington
“We Need a Finger Pointing Tool”
5/5/2019

8. Gray Finger Pointing Tool
Finger Pointing Tool Time!
Applications
Developer
Applications
Developer
LAN
Administrator
LAN
Administrator
System
Administrator
System
Administrator
Campus
Networking
Campus
Networking
Gigapop
Gigapop
Locate the Problem
Gray Finger Pointing Tool
Terry Gray
University of Washington
Backbone
5/5/2019

9. Gray Finger Pointing Tool
Finger Pointing Tool Time
OK, I’ll fix it
Applications
Developer
Applications
Developer
LAN
Administrator
LAN
Administrator
You ARE the Weakest Link!
System
Administrator
System
Administrator
Campus
Networking
Campus
Networking
Gigapop
Gigapop
Backbone
5/5/2019

10. Agenda Motivation piPEs / AMI Overview piPEs / AMI Deployment Plan
OWAMP Release
Demo
5/5/2019

11. BNI AMI Project: BNI Abilene Measurement Infrastructure
Goal: Instrument Next Generation Abilene network with extensive performance measurement capabilities
Approach: Collaborative project among BNI, Engineering, and E2E.
5/5/2019

12. E2E piPES
Project: End-to-End Performance Initiative Performance Environment System (E2E piPES)
Goal: To allow end-users and network operators to determine performance capabilities, locate problems, and contact the right person to get a problem resolved.
Approach: Collaborative project combining the best work of many organizations.
Introduced at Spring Member Meeting.
Have made good progress on architectural design.
Have made good progress on outreach to other organizations and initiating collaboration.
5/5/2019

13. The Measurement System
Extend the System to Campuses
Campus X
Abilene
PMP
PMP
PMP
Campus PAS is a specified PC based computer ($2000 estimated) with a software load provided by Internet2
Campus Y
PMP
PMP
PMP at each Campus Border
5/5/2019

14. The Measurement System
The New Abilene will have measurement devices as part of its structure
Abilene
PMP
PMP
PMP
PMP = Performance Measurement Point (at each Abilene Node)
5/5/2019

15. Performance Reporting
Perform regular tests to provide web based Performance
Reports across Abilene and for Campus-Abilene paths
Campus X
Abilene Capability
SNVA/DNVR – 10 Gbps
DNVR/IPLS – 10 Gbps
IPLS/NYCM – 10 Gbps
Abilene
PMP
PMP
PMP
At first test results will be reported on web pages. An end-user suspecting performance problems can check the reported performance from their campus to Abilene, across Abilene, and from Abilene to the destination campus. If there are no reported problems on any of these segments, the the end-user can suspect the campus networks on the ends, or the computer systems themselves.
Campus Y
Campus-Abilene Capability
Campus W – 155 Mbps
Campus X – 612 Mbps
Campus Y – 1000 Mbps
Campus Z – 2 Mbps
PMP
PMP
5/5/2019

16. Reaching the End-User
Provide a Testing Tool for the End-Users Computer
to test across campus and beyond
Campus X
Abilene
PMP
PMP
PMP
Host A
Host B
The tool will use test results stored in a data repository as much as possible to reduce the overall amount of active testing going on. The tools will probably be a JAVA Applet to make
it as portable as possible.
Campus Y
PMP
PMP
The tool reports performance of the path to a specified destination by using active tests
(like to the campus border) or using stored test results from regularly scheduled tests.
It also performs tests on the host system to check host tuning parameters
5/5/2019

17. Adding Path Information
Add PMP along paths to provide additional detail
Campus X
Abilene
PMP
Gigapop S
Regional
Network M
PMP
PMP
PMP
Host A
PMP
Host B
PMP
E2Epi is looking for the Quilt to lead in the deployment at the Gigapops and Regional Networks. Ideally the system would have measurement information for each administrative domain along each path. For larger networks, multiple Path Measurement Points may be desireable.
Campus Y
PMP
PMP
PMP
Regional
Network J
Gigapop T
Regular tests are performed between adjacent PMP and stored in a Data Repository.
This information is used by the End-User tool to reduce the number or active tests needed.
5/5/2019

18. segment in the path appears to have a problem
Who You Gonna Call?
E2E piPES will provide contact information when a
segment in the path appears to have a problem
Campus X
Abilene
PMP
Gigapop S
Regional
Network M
PMP
PMP
PMP
Host A
PMP
Host B
PMP
E2Epi need input on how to make this work. There are operational and policy issues here.
piPES Test Analysis
Looks like a problem
In Gigapop T. Pass
these test result to:
Campus Y
PMP
PMP
PMP
Regional
Network J
Gigapop T
The test results obtained by the End-User will be passed on to the contact person
5/5/2019

19. E2E piPES Architecture
5/5/2019

20. Testing / Analysis Engine
Problem: Encode Matt Z.’s Brain
Open Questions:
What are measurement results are “acceptable” for a given application (family)?
What tools generate those results?
How do you handle incomplete data?
How do you rank multiple result-generators?
What is the iterative decision tree to understand E2E problem?
5/5/2019

21. Measurement Schema Measurement Types? e.g. BW, Latency, Loss, Jitter
Measurement Units?
e.g. Seconds vs. Microseconds
Map Tools to Measurements?
e.g. OWAMP -> 1-way Latency
Measurement Metadata?
Database Table Design?
5/5/2019

22. Access, Authentication, Authorization
Roles for Access
Standard End-User?
Near Neighbor (Test Buddy)?
NOC Staff / Network Engineer?
Shibboleth for Implementation
Each campus decides who in each role
Others trust campus designation
5/5/2019

23. Other R&D Issues Design system to scale (eliminate centralization)?
How to “discover” PMPs?
How to insert On-Demand tests into Regularly-Scheduled test set
Balance centralization and distributed database requirements
5/5/2019

24. Agenda Motivation piPEs / AMI Overview piPEs / AMI Deployment Plan
OWAMP Release
Demo
5/5/2019

25. E2E piPES Architecture v1.0
5/5/2019

26. piPEs / AMI Rollout
5/5/2019

27. piPEs / AMI Rollout
5/5/2019

28. piPEs / AMI Rollout
5/5/2019

29. piPEs / AMI Rollout
5/5/2019

30. piPEs / AMI Rollout
5/5/2019

31. piPEs / AMI Development & Testing Environment
5/5/2019

32. piPEs / AMI Internal Resources
Eric Boyd – piPEs development, project coordination
Jeff Boote – OWAMP, piPEs development
Prasad Calyam – piPEs development
Chris Heermann – AMI
Matt Zekauskas - AMI
Susan Evett - Documentation
Russ Hobby – Campus Deployment
George Brett – Schema, “Grid Service”
5/5/2019

33. piPEs External Resources
Yee-Ting Li – UCL (50%)
Paul Mealor – UCL (up to 100%)
PhD Student – UCL (contingent on grant)
Warren Matthews – SLAC
Jim Ferguson et al – NLANR / DAST
Many early adopter volunteers
5/5/2019

34. Agenda Motivation piPEs / AMI Overview piPEs / AMI Deployment Plan
OWAMP Release
Demo
5/5/2019

35. Latency Tests
Problem: Existing platforms are not interoperable (SURVEYOR,RIPE,…)
Solution: Standards
5/5/2019

36. One-Way-Active-Measurement-Protocol
OWAMP
One-Way-Active-Measurement-Protocol
Possible due to growing availability of good time sources
Wide deployment of “open” servers would allow measurement of one-way delay to become as commonplace as measurement of RTT using ICMP tools such as ping.
Current Draft: draft-ietf-ippm-owdp-05.txt
Shalunov,Teitelbaum,Zekauskas
Sample implementation under development
Beta Release:
5/5/2019

37. OWAMP Implementation Basically: NTP system call interface
Multiple processes for recv/send loops
Written as an API to allow one-off implementations
5/5/2019

38. OWAMP Error Estimates Rate = 1 packets/sec Min/Median = 59/64 usec
5/5/2019

39. OWAMP Error Estimates Rate = 10 packets/sec Min/Median = 57/63 usec
5/5/2019

40. OWAMP Error Estimates Rate = 100 packets/sec Min/Median = 54/73 usec
5/5/2019

41. OWAMP Error Estimates Rate = 1000 packets/sec Min/Median = 53/176 usec
5/5/2019

42. OWAMP Errors Preliminary Findings:
Min error estimates look to be in the usec range.
Serialization Delay: ~5usec x 2
Get Timestamp: ~15usec x 2
Additional error is:
Time from userland “send” to 1st byte hits the wire
Time from kernel has packet to userland “recv” returns
Potentially recv process data processing before calling “recv”
5/5/2019

43. OWAMP implementation status
Sample implementation
Beta Release:
No “policy”
No authentication/encryption
FreeBSD only
5/5/2019

44. Abilene OWAMP deployment
2 overlapping full meshes (IPv4 & IPv6)
11 measurement nodes = 220 ongoing tests
UDP singletons
Rate: 10 packets/second*
Packetsize: (32 byte payload)*
Results are continuously streamed back to “Measurement Portal” for long-term archive and data dissemination (Near real-time)
* These are the values we are currently we are planning on using, but this is not set in stone.
5/5/2019

45. Agenda Motivation piPEs / AMI Overview piPEs / AMI Deployment Plan
OWAMP Release
Demo
5/5/2019

46. piPEs / AMI / OWAMP Demo http://owamp.internet2.edu when available
5/5/2019

47. More Internet2 Information
On the Web
e2epi.internet2.edu
For more information about Internet2, please see these web sites, or contact directly by .
5/5/2019

48. Questions?
5/5/2019