1. Internet2 E2E piPEs Joining the Federation of Network Measurement Infrastructures
Eric L. Boyd
26 December 2018

2. Metcalf’s Law
Robert Metcalf's law states that the "value" or "power" of a network increases in proportion to the square of the number of nodes on the network.
Our version: The value of a performance measurement framework scales with the square of the deployment footprint
12/26/2018

3. Vision Statement Goal #1: A Federation of Measurement Frameworks
Approach: Develop a common language for measurement frameworks
Metcalf’s Law: Successful new measurement frameworks will be designed to speak the language
Goal #2: A Measurement Framework Federation
Approach: Develop a decentralized, locally administered, participatory framework
Metcalf’s Law: Autonomous network operators gain more value by joining than “going alone”
These goals are not mutually exclusive …
12/26/2018

4. Federation of Measurement Frameworks
Working within the GGF NMWG
NMWG is meeting 2 PM Wednesday Marriott Willow room on the fourth floor
Version 1 of the schema is all but “done”
Expect to announce it at Wednesday’s meeting
Version 2 of the schema is under development
Efforts such as piPEs, AMP, MonALISA, and NLANR DAST’s Advisor are interoperable
12/26/2018

5. Measurement Framework Federation (1)
Tool beacons for on-demand and regularly scheduled tests are released:
BWCTL (
Resource Allocation Broker for NLANR DAST’s Iperf (
NDT (
OWAMP (
And deployed:
15 organizations listed as of today
12/26/2018

6. Measurement Framework Federation (2)
piPEs Measurement Framework alpha is released:
And deployed on Abilene Observatory:
Shows results of 11 x 11 mesh of Iperf and OWAMP tests among Abilene nodes
Anyone can test to Abilene from their own BWCTL (need a key) or OWAMP node
Federation of NDT servers deployed on Abilene, with redirect to nearest server
12/26/2018

7. Internet2 / GÉANT2 JRA1: Exchange of Ideas
UCL E2E Monitoring Workshop 2003
Internet2, DANTE, CANARIE biannual meetings (12/03, 07/04) and ESnet (01/05)
Transatlantic Performance Monitoring Workshop 2004 (03/04)
Brussels Workshop (09/04)
12/26/2018

8. Internet2 / GÉANT2 JRA1: Working Together
Contribution to the GGF NM-WG
Both Internet2 and Dante provide a significant contribution to the effort (four to five people contributing regularly, 2 from Europe, 2-3 from US)
Contribute to design, early adoption and prototyping, feedback
Installation/evaluation of tools
Internet2 OWAMP, Internet2 BWCTL, Internet2 piPEs framework
DFN IPPM
Caltech <-> CERN Demo
March ’04 and November, December ‘04
Haystack, USA <-> Onsala, Sweden
12/26/2018

9. Internet2 / GÉANT2 JRA1: Joint Development (1)
Rather than build two separate interoperable measurement frameworks, why not jointly develop a single measurement framework?
Steps:
Agree to joint open source development √
General Framework Design √
Prototype (Summer ’05)
Detailed Design
Implementation
Seek participation from NRENs & campuses, particularly Internet2 & ESnet members
Thrice weekly conference calls
Very active mailing list
2-3 face-to-face meetings per year
12/26/2018

10. So what are we building? You could call it piPEs v2.0 or GFD
A Services-Based Measurement Framework for Building Dynamic, Self-Organizing Performance Communities
12/26/2018

11. Design Goals Services-oriented measurement framework
Dynamic, self-organizing characteristics identical to that of the network as a whole
Recognize and facilitate the ability of independent network entities to set policies and limits on the use of measurement resources locally
Encourage and facilitate the use of measurement resources by users interested in network paths that traverse remote administrative domains
Facilitate the widespread adoption of new performance tools in a broad, E2E framework
Allow framework to evolve over time
12/26/2018

12. Services Oriented Architecture
In a simple scenario, each domain consists of a set of services. All services are well defined and independent
Services within a domain represent the domain with the help of Authentication and Authorization – they respond to requests only if the Authentication service of the domain has authenticated the user and the policy of the given service authorizes it.
12/26/2018

13. Basic Services Lookup Authentication Measurement Point
Measurement Archive
Resource Protector (Authorization)
Transformation
Topology
12/26/2018

14. Lookup Service Initial discovery Lookup is not simply by name
Multicast / Anycast
Well known servers
Required servers (by administrative configuration)
Previously detected servers (organized in a P2P network – lookup services find out about other lookup services…
Lookup is not simply by name
Type of service, Community, Network path, Organization, Type of authentication required, etc.
Response contains
Contact information, Available services, Authentication required, etc.
12/26/2018

15. Authentication Service
Registers with lookup
Client requests “kind” of authentication token based on lookup results
Authentication grants time-limited token used to request service
Attribute service created to protect privacy and support role-based authorization
Allow new measurement points to be created as easily as possible
Allow new data consumers access as easily as possible
12/26/2018

16. Measurement Point Service
Service to wrap measurement tools
Interacts with resource protectors to protect shared resources
Registers with lookup service and specifies the authentication credentials required to interact
Registers with lookup service to indicate types of tests it can perform
Accepts requests for tests
12/26/2018

17. Measurement Archive Service
Subscribes to some set of data – either from a measurement point or from an aggregation service
May publish the derived data sets
12/26/2018

18. Resource Protector Service
Enables centralizing of resource allocation (not globally - this is within spheres of administrative control)
Multiple measurement points interact with a given resource protector to limit the shared resources
Resource protectors can be chained to control aggregations of shared resources across larger frameworks.
12/26/2018

19. Transformation Service
Pipelines data between other components in the framework
Subscribes and Publishes data
Provides:
Aggregation
Correlation
Caching
Duplication
Filtering
Translation
Event generation
Data analysis
12/26/2018

20. Topology Service Special case of a Transformation Service
Network topology information is necessary for measurement system optimization
Creates overviews/”maps” to illustrate network
Layered approach (domain level through to wavelengths and physical level)
Collects raw data from measurement points and pushes topology information into the measurement archive (allows topologically based queries by lookup service)
12/26/2018

21. Summary Open Source Shared Development
Sourceforge-based Sub-Projects
Modified Berkeley Licensing
Common Service-based Architecture
Architecture spans superset of deployment use cases
~Quarterly face-to-face meetings
~Thrice-Weekly phone conferences
Split development according to interest, resources
12/26/2018

22. Questions? Are you interested in participating? Talks of Interest
piPEfitters BoF for a detailed discussion of the architecture
piPEfitters BoF for a detailed discussion of NDT efforts
Tuesday, 1:00 PM, Parlor A
GGF NMWG meeting for a detailed discussion of a common measurement schema
Wednesday, 2:00 PM, Willow Room, 4th Floor
Talks of Interest
Measurement SIG
Tuesday, 7 PM, Marriot Ballroom 1
Transport Track session for a introduction into efforts of ad hoc working group aiming to get benefit of kernel level congestion control algorithm improvements into a user level bulk FTP tool
Wednesday, 10:25 AM, Saltair
12/26/2018