Sponsored by the National Science Foundation GENI Instrumentation and Measurement WG: 2 nd I&M Workshop and Priority Architecture Topics GENI Engineering Conference 8 UC San Diego GPO System Engineer: Harry Mussman July 22,

Sponsored by the National Science Foundation 2 July 22, nd I&M Workshop Objective: –Gather contributors from key I&M prototyping projects that have already implemented pieces of I&M functionality in a manner consistent with GENI goals. –Define priority pieces of the I&M architecture by consensus –Assemble teams to complete the discussion and documentation –Draft a roadmap for implementations during Spirals 2 and 3. –See Initial set of priority topics from WG meeting at GEC7: –GENI I&M use cases –GENI I&M services –GENI measurement plane –Interfaces, protocols and schema for measurement data in GENI

Sponsored by the National Science Foundation 3 July 22, 2010 (continued) Organizing Committee: –Paul Barford - University of Wisconsin – Madison –Bruce Maggs – Duke University and Akamai –Harry Mussman – BBN/GPO –Vic Thomas - BBN/GPO –Evan Zhang – BBN/GPO Key projects who sent representatives: –OML (ORBIT Measurement Library) OMF (ORBIT Management Framework) –Instrumentation Tools –perfSONAR –Scalable Sensing Service (S3) –OnTimeMeasure for network measurements –GENI Meta-Operations Center and NetKarma –Virtual Machine Introspection (VMI) –Experiment Management Service – Digital Object Registry

Sponsored by the National Science Foundation 4 July 22, 2010 (continued) Approach: –Each priority topic was discussed in a structured manner –The organizers outlined an approach or solution, and representatives from the key projects discussed it – The group identified where they had a consensus, and where there were issues that needed further discussion –One new topic was identified: how to discover, specify, authorize and assign all of the various types of resources required for I&M capabilities –The interfaces topic was split into four topics, divided upon function –Finally, teams were assembled to discuss each priority topic, come to a consensus, and write a revised section for the architecture document

Sponsored by the National Science Foundation 5 July 22, 2010 (continued) Final set of priority topics: –Topic 1: GENI I&M Use Cases –Topic 2: GENI I&M Services –Topic 3: GENI I&M Resources –Topic 4: GENI I&M Measurement Plane and Interfaces –Topic 5: GENI I&M Interfaces and Messages/Flows/APIs : Manage Services –Topic 6: GENI I&M Interfaces and Messages/Flows/APIs : Data Flows and Data File Transfers –Topic 7: GENI I&M Interfaces and Messages/Flows/APIs : Service Registration and Discovery –Topic 8: GENI I&M Interfaces and Messages/Flows/APIs : GUIs –Topic 9: GENI Measurement Data Schema

Sponsored by the National Science Foundation 6 July 22, 2010 Topic 2: GENI I&M Services Team members: –Harry Mussman* and Evan Zhang – BBN/GPO –Giridhar Manepalli – CNRI –Chris Small and Beth Plale - Indiana Univ *agreed to organize first writing and discussion Summarize current view of services –Measurement Orchestration (MO) Service –Measurement Point (MP) Service –Measurement Information (MI) Service (added since GEC7) –Measurement Collection (MC) Service –Measurement Analysis and Presentation (MAP) Service –Measurement Data Archive (MDA) Service Have identified three different types of services: –Type 1: Service completely dedicated to an experiment –Type 2: Common service platform, with dedicated slivers for different experiments –Type 3: Common service, with data provided to multiple experiments

Sponsored by the National Science Foundation 7 July 22, 2010 GENI I&M Architecture: Services and Interfaces

Sponsored by the National Science Foundation 8 July 22, 2010 Topic 4: GENI I&M Measurement Plane and Interfaces Team members: –Harry Mussman* – BBN/GPO –Ezra Kissel – Univ Delaware –Chris Small - Indiana Univ *agreed to organize first writing and discussion Assume: Use IP backbone network (with Internet access) for most measurement traffic (like control traffic) Consider: Use Layer 2 (VLAN) connections only when necessary Issues with measurement traffic: –Which protocols are active? minimize number to simplify? –Need access to resources in aggregates, even when resources are in private address space, via proxies (i.e., gateways) –Need to provide authentication and authorization, without and with proxies –Need to be able to reserve bandwidth (provide QoS) to protect measurement traffic from other traffic, and vice-versa –Which of these requires the use of Layer 2 (VLAN) connections?

Sponsored by the National Science Foundation 9 July 22, 2010 GENI Measurement Traffic Flows

Sponsored by the National Science Foundation 10 July 22, 2010 GENI I&M Architecture: Services and Interfaces

Sponsored by the National Science Foundation 11 July 22, 2010 GENI I&M Interfaces and Messages/Flows/APIs 1) Discover Resources and Assign Slivers –EC srvc uses CF to discover resources, and then assign slivers to slice/researcher for I&M srvc’s 2) Configure and Program Slivers –EC srvc uses CF and/or ssh to load std or customized software images for I&M srvc’s –Note: 1) and 2) are not specific to I&M services 3) Manage Services –EC srvc and MO srvc use CF and/or https to check status of I&M srvcs, receive event notifications, and execute functions such as start, stop, reset, reboot, and checkpoint 4) Measurement Data Flows and Data File Transfers –Measurement data flows and data file transfers between I&M srvcs. Options: Pull, Push, Pub/Sub. 5) Register I&M Service –Operator configures I&M srvc to register with Lookup Srvc, advertising name, location, and available metadata 6) Discover I&M Service and Establish Meas Data Flow –ECS or I&M srvc discovers I&M srvc advertisement, and establishes data flow 7) Conduct and Observe Experiment –Researcher uses browser to interact with and observe services via web portals (GUIs)

Sponsored by the National Science Foundation 12 July 22, 2010 Topic 6: GENI I&M Interfaces and Messages/Flows/APIs: Data Flows and Data File Transfers Team members: –Harry Mussman* – BBN/GPO –Ivan Seskar – Rutgers WINLAB –Max Ott – NICTA –Ezra Kissel – Univ Delaware –Prasad Calyam - Ohio Supercomputing Ctr –Michael Zink - UMass Amherst *agreed to organize first writing and discussion Consider data flows and data file transfers between all services –Fundamental to I&M, and not yet defined in GENI –Linked to measurement data schema, which is primary

Sponsored by the National Science Foundation 13 July 22, 2010 (continued) Consensus to allow a wide range of options: –Both data flows and data files transfers –Types: Pull, Push, Pub/Sub –Protocols: SNMP, SCP, FTP, gridFTP, HTTP, XMPP, TCP, SCTP –Consider: Naming, Discovery, Connectivity, Authentication and authorization mechanisms Goal is to define an initial set: –Minimum set required for GENI –Mapped to current projects –Permit later additions

Sponsored by the National Science Foundation 14 July 22, 2010 Reports on Priority Topics Topic 1: Use Cases, by Prasad Calyam/Ohio Supercomputing Topic 3: I&M Resources: Naming, Discovery and Authorization, by Jim Griffioen/University of Kentucky Topic 5: I&M Interfaces and Messages/Flows/APIs: Manage Services, by Sonia Fahmy/Purdue Topic 7: I&M Interfaces and Messages/Flows/APIs : Service Registration and Discovery, by Jason Zurawski/Internet 2 Topic 8: I&M Interfaces and Messages/Flows/APIs : GUIs, by Guilherme Fernandes/Univ Delaware Topic 9: Measurement Data Schema, by Martin Swany/Univ Delaware