1. GENI Instrumentation and Measurement System - Schema Martin Swany

2. Schema Generally: model of objects and their relationships We should talk in terms of the general “schema” in this sense Renderings later –XML schema –SQL schema –IPFIX –JSON

3. I&M Schema Requirements Consistent basic representation of measurement and instrumentation data inside and outside of a slice Flexibility of encoding and transport –timestamp and value or values Expressive metadata Easily extensible Reusable components and models Reasonable relationship to RSpec

4. perfSONAR Model perfSONAR is an internationally-adopted framework for instrumentation and measurement –Based on schemata and protocols defined in the *Grid Forum over the last 10 years –Recognized by the NSF as a key technology; recent NSF-funded workshop Again, consider the model in the abstract rather than the XML encoding or RNC files

5. perfSONAR Schema Key Goals: Extensibility, Normalization, Readability Break representation of performance measurements down into basic elements Data and Metadata Measurement Data –A set of of measurement events that have some value or values at a particular time Measurement Metadata –The details about the set of measurement data

6. Schema Normalization Can simplify the database representation for many types of measurement data –While optimizations are possible, many measurement types can be viewed as one value measured over time Assists Combination/Concatenation of metrics –Creating derived metrics Normalization helps with inferring relationships between types of metrics

7. Schema Basic Elements - Metadata Subject (Noun) –The measured/tested entity EventType (Verb) –What type of measurement or event occurred, or instrumented parameter was read –Characteristic, tool output, or generic event Parameters (Adjectives and Adverbs) –How, or under what conditions, did this event occur?

8. Schema Basic Elements - Data Some sort of value - Datum –Existence of an event might point to the case where there no additional value As in “Link up/down” or threshold events Time –Is extensible since various representations are appropriate in different cases E.g. UNIX timestamp vs NTP time

9. A Message Message MetadataData

10. An Object Store Store MetadataData

11. A Data is Linked to a Metadata Metadata someId Data someId

12. A Metadata may be linked to another Metadata someId Metadata someOtherId someId

13. Schema Namespaces Observation: all measurements have some sort of Data and Time All measurements can be described by the Metadata identifying who, what and how The specific structures of the Data and Metadata elements depend on the measurement Approach: Consistently use Data and Metadata elements and vary the namespaces of the specific elements

14. Schema Namespaces - 2 We encode the measurement/event type in the namespace –And as a standalone element Some components of the system can pass Data and Metadata elements through without understanding their specific structure Allows and implementation to decide whether it supports a particular type of data or not Allows validation based on extended (namespace-specific) schemata

15. Schema Namespaces and Extensibility One key to extensibility is the use of hierarchy with delegation –Similar to OIDs in the IETF management world The OGF NM-WG defined a hierarchy of network characteristics –Good starting point However, not all tools are cleanly mapped onto the Characteristic space –Often a matter of some debate

16. Schema Namespaces and Extensibility Organization-rooted tools namespace addresses this Some top-level tools ping, traceroute Easy to add new tools in organization- specific namespaces Performance Event Repository –Add a schema and get a URI –Add Java classes

17. Linking Metadata Metadata can be linked in two ways –Merge chaining allows for elements to be reused and a complete metadata can be built –Operation chaining requests or describes operations on data sets Representation of data provenance A A B B ABAB ABAB A A B B B(A)

18. Resources, Subjects, Topology perfSONAR has a topology schema called UNIS – Unified Network Information Schema Related to the control frameworks’ Resource Specification Measurement and Instrumentation must be related to the resources themselves

19. end

20. Chaining Metadata Merge chaining uses the idRef of the Metadata Operation chaining uses the idRef of the Subject