1. Sponsored by the National Science Foundation 1 March 15, 2011 GENI I&M Update: Gathering, Transferring and Sharing MD Goals Architecture Overview –Process –Functional Services –Measurement Data Flows/Transfers –Measurement Traffic Flows –Principal Use Cases –Types of Services –Control Framework Implications –Measurement Data Objects and Descriptors –Controlling Access to Measurement Data Objects –Current Status Current I&M Design and Prototyping Efforts I&M Service Deployments References: –I&M work in progress: http://groups.geni.net/geni/wiki/GeniInstMeas http://groups.geni.net/geni/wiki/GeniInstMeas –I&M capabilities catalog: http://groups.geni.net/geni/wiki/GENIIandMCAPCAT http://groups.geni.net/geni/wiki/GENIIandMCAPCAT –I&M architecture document: http://groups.geni.net/geni/wiki/GeniInstrumentationandMeasurementsArchitecture http://groups.geni.net/geni/wiki/GeniInstrumentationandMeasurementsArchitecture –Available I&M tools: http://groups.geni.net/geni/wiki/GeniInstMeas#AvailableGENIIMTools http://groups.geni.net/geni/wiki/GeniInstMeas#AvailableGENIIMTools

2. Sponsored by the National Science Foundation 2 March 15, 2011 GENI I&M Architecture Overview: Measurement Data (MD) Flows/Transfers MD packet flows and file transfers are carried from one I&M service to the next “Stitching” is done as I&M services are setup and configured Authorization mechanisms are required to protect MD Flows/transfers follow a wide range of protocols and schema, including both pull and push methods

3. Sponsored by the National Science Foundation 3 March 15, 2011 Measurement Data (MD) Flows/Transfers: Current Protocols and Schema Current range of protocols and schema: Option 1: Bytes of MD via SNMP Option 2: File of MD via SCP, FTP or 3 rd -party FTP Option 3: XML-formatted MD via HTTP (e.g., perfSONAR) Option 4: Tuples of MD via Custom OML Protocol Option 5: Tuples of MD via IPFIX over SCTP Option 6: NetCDF formatted files of data using LDM over TCP Opt 1 Opt 3 Opt 4 Opt 5 Opt 2

4. Sponsored by the National Science Foundation 4 March 15, 2011 GENI I&M Architecture Overview: Principal Use Cases Use Case 1: Experimenter gathering measurement data (MD) from their slice Use Case 2: Operator (or Service Provider) gathering measurement data (MD) from GENI infrastructure Use Case 3: Experimenter gathering measurement data (MD) from their slice and from GENI infrastructure Use Case 4: Experimenter (or Operator) moving measurement data (MD) to an archive, with an option to share with other Researchers

5. Sponsored by the National Science Foundation 5 March 15, 2011 Use Case 1: Experimenter gathering MD from their slice Each I&M service is setup as part of the Experimenter’s slice Includes: –Type 1 service: dedicated to a slice –Type 4 service: with portal to share MD The Experimenter that collects the MD “owns” the MD, and sets the policy for how it can be stored or disposed

6. Sponsored by the National Science Foundation 6 March 15, 2011 Use Case 2: Operator (or Service Provider) Gathering MD from GENI infrastructure An Operator (or Service Provider) can gather MD from GENI infrastructure into their slice –For their own use –To share with other operators, e.g., share with GMOC –The Operator (or Service Provider) that collects the MD “owns” the MD, and sets the policy for how it can be stored or disposed, consistent with policy of Infrastructure and/or GENI Includes: –Type 2 service: common service with multiple slivers –Type 3 service: common service with MD to multiple slices

7. Sponsored by the National Science Foundation 7 March 15, 2011 Type 2 I&M Service: Common service with multiple slivers Each Type 2 I&M service: –Common service, assembled, configured and managed by a Service Provider (or Operator) in their sliver using Control Framework (CF) mechanisms –Service Provider runs an AggMgr interface and offers slivers (e.g., MP service slivers, or UW service slivers, as shown above) to other Operators (or Experimenters) via CF mechanisms –Each sliver is a resource with an rspec –Example: Measurement System for packet capture from Univ Wisconsin –Example: User Workspace (UW) service prototype from CNRI

8. Sponsored by the National Science Foundation 8 March 15, 2011 Type 3 I&M Service: Common service with MD to multiple slices Each Type 3 I&M service: Common service assembled, configured and managed by a Service Provider (or Operator) in their sliver using Control Framework (CF) mechanisms Service Provider (or Operator) registers Descriptors of available MD Objects (e.g., MD flows, as shown above) with Measurement Information (MI) service Example: perfSONAR MP and MI services Operator (or Experimenter) finds and requests a flow from the I&M service Authorization mechanism required to make flow available to selected operators/experimenters. Obligated to follow policy established by MD “owner”

9. Sponsored by the National Science Foundation 9 March 15, 2011 Use Case 3: Experimenter gathering MD from their slice and from GENI infrastructure An Experimenter can gather MD from their slice and from GENI infrastructure –Combines Use Case 1 with Use Case 2 Includes Type 2 service: common service with multiple slivers –Policy required to make sliver available to selected experimenters Includes Type 3 service: common service with MD to multiple slices –Access mechanism required to make flow available to selected experimenters Includes Type 1 service: dedicated to a slice –Configured to stitch flow into experimenter’s I&M service

10. Sponsored by the National Science Foundation 10 March 15, 2011 GENI I&M Architecture Overview: Controlling Access to Measurement Data (MD) Objects MD Objects have associated Measurement Data Object Descriptors (MDODs, commonly known as “metadata”) –Each MDOD typically includes policy elements, to specify how the MD Object can be shared or disposed. –The I&M service holding the MD Object is obligated to follow the policy for sharing Measurement data flows and transfers between Type 1 services within a slice must be “stitched” as slice is setup –An access control mechanisms is required to prevent others from getting the MD A Type 3 service can register measurement data for others to discover and share MD via a standardized interface –An access control mechanism is required to admit selected GENI experimenters and operators –It should reuse accepted certificate/credential arrangements –It must follow policy in MDOD associated with Digital Object A Type 4 service can share measurement data via a web portal –An access control mechanism is required to admit selected GENI experimenters and operators –It should reuse accepted certificate/credential arrangements –A Digital Object Archive (DOA) service is required to make digital object available to selected users within GENI, selected users outside of GENI, and/or all others –It must follow policy in MDOD associated with Digital Object