Internet2 E2E piPEs Joining the Federation of Network Measurement Infrastructures Eric L. Boyd 26 December 2018

Metcalf’s Law Robert Metcalf's law states that the "value" or "power" of a network increases in proportion to the square of the number of nodes on the network. Our version: The value of a performance measurement framework scales with the square of the deployment footprint 12/26/2018

Vision Statement Goal #1: A Federation of Measurement Frameworks Approach: Develop a common language for measurement frameworks Metcalf’s Law: Successful new measurement frameworks will be designed to speak the language Goal #2: A Measurement Framework Federation Approach: Develop a decentralized, locally administered, participatory framework Metcalf’s Law: Autonomous network operators gain more value by joining than “going alone” These goals are not mutually exclusive … 12/26/2018

Federation of Measurement Frameworks Working within the GGF NMWG NMWG is meeting 2 PM Wednesday Marriott Willow room on the fourth floor Version 1 of the schema is all but “done” Expect to announce it at Wednesday’s meeting Version 2 of the schema is under development Efforts such as piPEs, AMP, MonALISA, and NLANR DAST’s Advisor are interoperable 12/26/2018

Measurement Framework Federation (1) Tool beacons for on-demand and regularly scheduled tests are released: BWCTL (http://e2epi.internet2.edu/bwctl) Resource Allocation Broker for NLANR DAST’s Iperf (http://dast.nlanr.net/Projects/Iperf/) NDT (http://e2epi.internet2.edu/ndt) OWAMP (http://e2epi.internet2.edu/owamp) And deployed: http://e2epi.internet2.edu/pipes/pmp/pmp-dir.html 15 organizations listed as of today 12/26/2018

Measurement Framework Federation (2) piPEs Measurement Framework alpha is released: http://e2epi.internet2.edu/pipes/ And deployed on Abilene Observatory: Shows results of 11 x 11 mesh of Iperf and OWAMP tests among Abilene nodes http://abilene.internet2.edu/observatory/ Anyone can test to Abilene from their own BWCTL (need a key) or OWAMP node http://e2epi.internet2.edu/pipes/ami/pmp-info.html Federation of NDT servers deployed on Abilene, with redirect to nearest server 12/26/2018

Internet2 / GÉANT2 JRA1: Exchange of Ideas UCL E2E Monitoring Workshop 2003 http://people.internet2.edu/~eboyd/ucl_workshop.html Internet2, DANTE, CANARIE biannual meetings (12/03, 07/04) and ESnet (01/05) Transatlantic Performance Monitoring Workshop 2004 (03/04) http://people.internet2.edu/~eboyd/transatlantic_workshop.html Brussels Workshop (09/04) 12/26/2018

Internet2 / GÉANT2 JRA1: Working Together Contribution to the GGF NM-WG Both Internet2 and Dante provide a significant contribution to the effort (four to five people contributing regularly, 2 from Europe, 2-3 from US) Contribute to design, early adoption and prototyping, feedback Installation/evaluation of tools Internet2 OWAMP, Internet2 BWCTL, Internet2 piPEs framework DFN IPPM Caltech <-> CERN Demo March ’04 and November, December ‘04 Haystack, USA <-> Onsala, Sweden 12/26/2018

Internet2 / GÉANT2 JRA1: Joint Development (1) Rather than build two separate interoperable measurement frameworks, why not jointly develop a single measurement framework? Steps: Agree to joint open source development √ General Framework Design √ Prototype (Summer ’05) Detailed Design Implementation Seek participation from NRENs & campuses, particularly Internet2 & ESnet members Thrice weekly conference calls Very active mailing list 2-3 face-to-face meetings per year 12/26/2018

So what are we building? You could call it piPEs v2.0 or GFD A Services-Based Measurement Framework for Building Dynamic, Self-Organizing Performance Communities 12/26/2018

Design Goals Services-oriented measurement framework Dynamic, self-organizing characteristics identical to that of the network as a whole Recognize and facilitate the ability of independent network entities to set policies and limits on the use of measurement resources locally Encourage and facilitate the use of measurement resources by users interested in network paths that traverse remote administrative domains Facilitate the widespread adoption of new performance tools in a broad, E2E framework Allow framework to evolve over time 12/26/2018

Services Oriented Architecture In a simple scenario, each domain consists of a set of services. All services are well defined and independent Services within a domain represent the domain with the help of Authentication and Authorization – they respond to requests only if the Authentication service of the domain has authenticated the user and the policy of the given service authorizes it. 12/26/2018

Basic Services Lookup Authentication Measurement Point Measurement Archive Resource Protector (Authorization) Transformation Topology 12/26/2018

Lookup Service Initial discovery Lookup is not simply by name Multicast / Anycast Well known servers Required servers (by administrative configuration) Previously detected servers (organized in a P2P network – lookup services find out about other lookup services… Lookup is not simply by name Type of service, Community, Network path, Organization, Type of authentication required, etc. Response contains Contact information, Available services, Authentication required, etc. 12/26/2018

Authentication Service Registers with lookup Client requests “kind” of authentication token based on lookup results Authentication grants time-limited token used to request service Attribute service created to protect privacy and support role-based authorization Allow new measurement points to be created as easily as possible Allow new data consumers access as easily as possible 12/26/2018

Measurement Point Service Service to wrap measurement tools Interacts with resource protectors to protect shared resources Registers with lookup service and specifies the authentication credentials required to interact Registers with lookup service to indicate types of tests it can perform Accepts requests for tests 12/26/2018

Measurement Archive Service Subscribes to some set of data – either from a measurement point or from an aggregation service May publish the derived data sets 12/26/2018

Resource Protector Service Enables centralizing of resource allocation (not globally - this is within spheres of administrative control) Multiple measurement points interact with a given resource protector to limit the shared resources Resource protectors can be chained to control aggregations of shared resources across larger frameworks. 12/26/2018

Transformation Service Pipelines data between other components in the framework Subscribes and Publishes data Provides: Aggregation Correlation Caching Duplication Filtering Translation Event generation Data analysis 12/26/2018

Topology Service Special case of a Transformation Service Network topology information is necessary for measurement system optimization Creates overviews/”maps” to illustrate network Layered approach (domain level through to wavelengths and physical level) Collects raw data from measurement points and pushes topology information into the measurement archive (allows topologically based queries by lookup service) 12/26/2018

Summary Open Source Shared Development Sourceforge-based Sub-Projects Modified Berkeley Licensing Common Service-based Architecture Architecture spans superset of deployment use cases ~Quarterly face-to-face meetings ~Thrice-Weekly phone conferences Split development according to interest, resources 12/26/2018

Questions? Are you interested in participating? Talks of Interest piPEfitters BoF for a detailed discussion of the architecture piPEfitters BoF for a detailed discussion of NDT efforts Tuesday, 1:00 PM, Parlor A GGF NMWG meeting for a detailed discussion of a common measurement schema Wednesday, 2:00 PM, Willow Room, 4th Floor Talks of Interest Measurement SIG Tuesday, 7 PM, Marriot Ballroom 1 Transport Track session for a introduction into efforts of ad hoc working group aiming to get benefit of kernel level congestion control algorithm improvements into a user level bulk FTP tool Wednesday, 10:25 AM, Saltair 12/26/2018