1. Internet2 Spring Member Meeting
Tutorial: Internet2 101
Advanced Services and End-to-End Performance
Russ Hobby
This is a general overview presentation about Internet2. Internet2 is a consortium, led by US universities, which is recreating the partnership among academia, industry and government that fostered today’s Internet in its infancy.
21 February 2019

2. Outline Original Internet2 Intents and Reasons What we have done
What we have discovered
Changes in direction based on what we have learned
2/21/2019

3. Original Reasons for Advanced Services in Internet2
Commercial Internet could not deliver what researchers wanted
Eliminate the network as a limitation for applications
Re-create the sharing environment to test and develop new capabilities
Transfer new technologies to the larger Internet
2/21/2019

4. Provide Reliable Performance Across the Network
Implement Quality of Service (QoS) capabilities from Source to Destination
Create a coordinated bandwidth reservation system
Let applications determine the level they need
Allow resource control and accounting
Provide “Bandwidth Overkill” until QoS is available
2/21/2019

5. Support efficient “one-to-many” and “few-to-few” applications
Internet2 video broadcast
Support effective video conferencing
Allow shared use of common data
The solution: Multicast
2/21/2019

6. Create an National Network for Technology Development
2/21/2019

7. Test New Protocols and Capabilities
New Network Equipment
New Link Technologies
Complex Routing
Measurement Techniques
IPv6
2/21/2019

8. What We Have Tried Creating a National Network for Development
Implementing wide-spread Multicast
Trials of QoS across the Network
Tests and Training of IPv6
2/21/2019

9. We learned from all the experiences!
What We Have Learned
Some ideas have worked
Some ideas are more complex than expected
Some had to go back to the drawing board
Some had a change in direction
We learned from all the experiences!
2/21/2019

10. High Capacity Development Networks
They Enable New Applications
2/21/2019

11. Gigapops have helped to complete the backbones in local and regional areas.
2/21/2019

12. High Capacity Development Networks
Enable New Applications
Many good Demonstrations
Real Time Application can now work
Transfers of Large Datasets
Provide the Test Bed for New Technologies
Packet over Sonet
New Measurement Techniques
Allows sharing of Ideas and Results
2/21/2019

13. Multicast
Robust Multicast Deployment Has Been Done
2/21/2019

14. Why Isn’t Multicast Used More?
For few to few applications it is still easier to use Application Hubs
Applications Hubs provide better control
One to many applications could benefit from Multicast but we have not yet reached “many” for application users.
There is still the control issue. How do you know who is connected?
2/21/2019

15. QoS Abilene Premium Service failed to get off the ground
It is a complex system that needs many parts before it can work. Some key part were not implemented for high speed links.
Premium Service requires all parts of the end-to-end path to participate and coordinate. This is against the current Internet operating model.
Allocation models for who gets Premium Service need a lot more work.
Trials are going on with Scavenger Service
Low Priority Best Effort Service
Only helps conjection. Jitter. Less words
2/21/2019

16. IPv6 An Operational IPv6 National Backbone
V6 works and is not hard
An Operational IPv6 National Backbone
Established Hands-on Training Workshops
2/21/2019

17. What Did Not Work QoS was too hard to implement at this time
Even with High Bandwidth available, researchers often did not see the expected performance. WHY?
The End-to-End Performance Initiative (E2Epi) was create to figure this out.
2/21/2019

18. The Wizard Gap
2/21/2019

19. Hey, this is not working right!
A Problem
Hey, this is not working right!
Others are
getting in ok
Not our problem
Applications
Developer
Applications
Developer
LAN
Administrator
LAN
Administrator
Talk to the other guys
System
Administrator
Everything is
AOK
System
Administrator
Campus
Networking
Campus
Networking
The computer
Is working OK
No other
complaints
Looks fine
Gigapop
Gigapop
All the lights
are green
How do you solve
a problem along a path?
Backbone
We don’t see
anything wrong
The network is lightly loaded
2/21/2019

20. True End-to-End Experience
User perception
Application
Operating system
Host IP stack
Host network card
Local Area Network
Campus backbone network
Campus link to regional network/GigaPoP
GigaPoP link to Internet2 national backbones
International connections
EYEBALL
APPLICATION
STACK
JACK
NETWORK
. . .
2/21/2019

21. Good Work Already Going On
Many projects and research is underway by others investigating performance on portions of the end-to-end path.
E2Epi seeks to bring these efforts together to look at the end-to-end path as a complete system.
2/21/2019

22. Packet Loss The Performance Killer
On high-speed, long-distance connections even one lost packet can greatly affect performance. For TCP the maximum transmission rate is:
MSS
0.7
Rate = *
As the equation shows the rate is limited by the packet size and the round-trip-time. The longer the distance the lower the rate possible. But to get the most out of that rate, the packet loss needs to be as small as possible. Packet loss can occur an any of a number of locations along the end-to-end path and not just in the network.
RTT P
Matt Mathis
MSS = Maximum Segment Size
RTT = Round Trip Time
P = Packet Loss
2/21/2019

23. Top Three Problems - #1 Ethernet Duplex Mismatch
Duplex negotiations failed
One side full duplex, one side half duplex
Works fine under light load
Heavy load, full duplex dominates
The top three problems according to a paper by Claudia DeLuna, JPL
2/21/2019

24. Top Three Problems - #2 Computer Tuning
Buffers not set big enough, can not fill pipe
Software options not set correctly
Hardware limitations
2/21/2019

25. Top Three Problems - #3 Bad Media – Copper or Fiber
Usually between wall and computer
Kinked copper wire
Dirty optic fibers
Causes random errors and packet loss
2/21/2019

26. Partners Campuses Faculty and discipline communities GigaPoPs
International partners
NSF-sponsored engineering efforts
NLANR,
Web100 Project,
Internet2 corporate members
Federal labs and agencies
2/21/2019

27. E2Epi as a Coordination Point
E2Epi has provided a rallying point for the many efforts going on in measurement and performance improvement.
These groups are trying to develop a common reference framework.
2/21/2019

28. A View of Measurement System Components
Data Collection
Hardware Platform
Test Programs
Measurement Attributes
Data Queue
Data Repository
Data Gathering
Database
Data Access
Exchange
Format
Exchange
Format
Problem Reporting
Contact Database
Report Format
Contact Method
Analysis
Data Retrieval
Path Capabilities
Problem Detection
2/21/2019

29. Internet2 Performance Package Application Beacons piPES
E2Epi Projects
Internet2 Performance Package
Application Beacons
piPES
2/21/2019

30. Internet2 Performance Package Project
Establish computer performance testing procedures and minimum capabilities
Gather test reports from the Internet2 community
Allow computer vendors to certify Internet2 Performance Packages for sale to the community
2/21/2019

31. Application Beacons Provide a method to check applications performance
Facilitate divide and conquer methods for locating problems
2/21/2019

32. Let’s Bounce it off a Beacon!
Application Beacons
Let’s Bounce it off a Beacon!
OK you test
your end
Applications
Developer
Applications
Developer
LAN
Administrator
LAN
Administrator
System
Administrator
System
Administrator
Campus
Networking
Campus
Networking
Beacon
Applications beacons can be used by end-users to create a “divide and conquer” technique to figure out where a performance problem lies.
Gigapop
Gigapop
How do you solve
a problem along a path?
Backbone
2/21/2019

33. Application Beacons My side tests OK Try from your end Beacon
Applications
Developer
Applications
Developer
LAN
Administrator
LAN
Administrator
System
Administrator
System
Administrator
Campus
Networking
Campus
Networking
Beacon
First one side test to the application beacon that is somewhere along the path and determines if the expected performance can be achieved.
Gigapop
Gigapop
Backbone
2/21/2019

34. Application Beacons Beacon Oh, My end has a lot of packets lost
Applications
Developer
Applications
Developer
LAN
Administrator
LAN
Administrator
System
Administrator
System
Administrator
Campus
Networking
Campus
Networking
Beacon
The other end also tests to the beacon to check performance. In this case it was determined that the problem is on the this half of the path. If other beacons are available along the path the location of the problem can be more closely determined.
Gigapop
Gigapop
Backbone
2/21/2019

35. E2E piPES
Project: End-to-End Performance Initiative Performance Environment System (E2E piPES)
Goal: To allow end-users and network operators to determine performance capabilities, locate problems, and contact the right person to get a problem resolved.
Approach: Collaborative project combining the best work of many organizations.
Introduced at Spring Member Meeting.
Have made good progress on architectural design.
Have made good progress on outreach to other organizations and initiating collaboration.
2/21/2019

36. The Measurement System
The New Abilene will have measurement devices as part of its structure
Abilene
PMP
PMP
PMP
PMP = Performance Measurement Point (at each Abilene Node)
2/21/2019

37. The Measurement System
Extend the System to Campuses
Campus X
Abilene
PMP
PMP
PMP
Campus PAS is a specified PC based computer ($2000 estimated) with a software load provided by Internet2
Campus Y
PMP
PMP
PMP at each Campus Border
2/21/2019

38. Performance Reporting
Perform regular tests to provide web based Performance
Reports across Abilene and for Campus-Abilene paths
Campus X
Abilene Capability
SNVA/DNVR – 10 Gbps
DNVR/IPLS – 10 Gbps
IPLS/NYCM – 10 Gbps
Abilene
PMP
PMP
PMP
At first test results will be reported on web pages. An end-user suspecting performance problems can check the reported performance from their campus to Abilene, across Abilene, and from Abilene to the destination campus. If there are no reported problems on any of these segments, the the end-user can suspect the campus networks on the ends, or the computer systems themselves.
Campus Y
Campus-Abilene Capability
Campus W – 155 Mbps
Campus X – 612 Mbps
Campus Y – 1000 Mbps
Campus Z – 2 Mbps
PMP
PMP
2/21/2019

39. Reaching the End-User
Provide a Testing Tool for the End-Users Computer
to test across campus and beyond
Campus X
Abilene
PMP
PMP
PMP
Host A
Host B
The tool will use test results stored in a data repository as much as possible to reduce the overall amount of active testing going on. The tools will probably be a JAVA Applet to make
it as portable as possible.
Campus Y
PMP
PMP
The tool reports performance of the path to a specified destination by using active tests
(like to the campus border) or using stored test results from regularly scheduled tests.
It also performs tests on the host system to check host tuning parameters
2/21/2019

40. Adding Path Information
Add PMP along paths to provide additional detail
Campus X
Abilene
PMP
Gigapop S
Regional
Network M
PMP
PMP
PMP
Host A
PMP
Host B
PMP
E2Epi is looking for the Quilt to lead in the deployment at the Gigapops and Regional Networks. Ideally the system would have measurement information for each administrative domain along each path. For larger networks, multiple Path Measurement Points may be desireable.
Campus Y
PMP
PMP
PMP
Regional
Network J
Gigapop T
Regular tests are performed between adjacent PMP and stored in a Data Repository.
This information is used by the End-User tool to reduce the number or active tests needed.
2/21/2019

41. segment in the path appears to have a problem
Who You Gonna Call?
E2E piPES will provide contact information when a
segment in the path appears to have a problem
Campus X
Abilene
PMP
Gigapop S
Regional
Network M
PMP
PMP
PMP
Host A
PMP
Host B
PMP
E2Epi need input on how to make this work. There are operational and policy issues here.
piPES Test Analysis
Looks like a problem
In Gigapop T. Pass
these test result to:
Campus Y
PMP
PMP
PMP
Regional
Network J
Gigapop T
The test results obtained by the End-User will be passed on to the contact person
2/21/2019

42. Access, Authentication, Authorization
Roles for Access
Standard End-User?
NOC Staff?
Network Engineer?
Shibboleth for Implementation
Each campus decides who in each role
Others trust campus designation
2/21/2019

43. ARENA Allows users to understand network speeds and route options
Database and web access to network link information
Manually updated by network operators
2/21/2019

44. DNS LINFO
Idea: Create DNS LINFO records for interfaces that indicate link speed, max MTU, and media type
Use: Modified Traceroute that will look up the LINFO and provide information on each link in the path
2/21/2019

45. Campus Network Infrastructure Guide
Write a Guide to help campuses in building Internet2 level networks
Draw upon the experiences of others
Provide advise on various technologies
2/21/2019

46. E2Epi Web Site Case Studies Lists of Tools
Other Projects working on Performance
A Place to Communicate
E2Epi-Interest list
2/21/2019

47. Network Magazine Article
Strategies & Issues: Measuring End-to-End Internet Performance
Discusses major cause of performance degradation.
2/21/2019

48. Future Works More use of passive measurements
Analysis techniques for measurement data
Make the Internet work as a system
2/21/2019

49. Peer-to-Peer and Grid Systems
Peer-to-Peer (P2P) and Grid Systems are global computers that use the network as the computer backplane
End-to-End Performance can greatly affect their usability.
P2P is a developing area that will affect and be affected by E2Epi
2/21/2019

50. Network Security Initial Internet Premise:
Security belongs on the end-system
Development of security capabilities has been slow.
Devices have been put into the network to try to help
Firewalls, NAT boxes, Filters
These often impede Applications.
Network devices themselves need to be protected
2/21/2019

51. 2/21/2019