1. XSEDE GLUE2 Update 1

2. Current XSEDE Usage Using legacy TeraGrid information services Publishing compute information about clusters – Subset of XSEDE clusters publishing GLUE2 is not yet required – TeraGrid XML schema Relatively similar to the flat one we’re finalizing – 2 documents: Public, with system description entities Private, with ComputingActivities Published periodically (2-5 mins) Used for Condor-G metascheduling and queue prediction

3. Future XSEDE Usage Discussing a new XSEDE information system – Prototyping now Expect to use GLUE2 – Will adopt an official XML schema XML or JSON Expect to publish additional documents – Documents with individual ComputingActivity (job status updates) – Documents with ApplicationEnvironment (installed software) – Documents with Services? (centralized services) – Documents with storage information – May do other reorganizing Additional users – web portal, science gateways 3

4. JSON Rendering Discussion 4

5. Background Shiraz and Warren have been working with GLUE2 and JSON – Shiraz was describing endpoints in JSON – Warren was describing compute-related entities A few months back, we thought we’d start discussing a JSON rendering Anyone else interested? Please let us know! 5

6. JSON Overview JavaScript Object Notation – JSON isn’t just for web interfaces! Lightweight data representation format Easy for humans and programs to use Consists of a few types: – Basic values: “strings”, numbers, true/false, null – Arrays (lists of values): [value, value] – Objects (sets of key/value pairs): {“name”: value, …} Significantly simpler than XML JSON libraries in every popular programming language 6

7. Guidelines for Rendering Flat structure, rather than hierarchical – Like the current XML rendering Allow documents to contain one or many GLUE2 entities – Describing one or more systems Documents can contain the full context or assume there is external context – Will come back to this If there can be more than one value, the JSON value is always an array – There can always be more than one value ??? – Alternative is consumer has to check the type of the value Naming follows the names in the GLUE2 spec – Use Extension instead of Extension.Key – Use fooID instead of foo.ID – Don’t pluralize entity and attribute names, even when the value is an array (Activity, not Activities) Use an Extension key in entity objects to hold extensions – Extension value is an object 7

8. Approach Develop templates/examples – Help us with discussions – No commonly used schema language – Define rendering by template/example and narrative Define JSON schema and/or provide a validator, if practical – json-schema json-schema.org Still be trying to go through the IETF process There are validation tools available for it May be worth using… Rendering document 8

9. JSON Document Structure No external context { GLUE2_ENTITY_NAME = {(can be at most 1 GLUE2_ENTITY of this type) GLUE2_ATTRIBUTE_NAME =..., GLUE2_ATTRIBUTE_NAME =... },... GLUE2_ENTITY_NAME = [(can be more than 1 GLUE2_ENTITY) { GLUE2_ATTRIBUTE_NAME =..., GLUE2_ATTRIBUTE_NAME =... },... { GLUE2_ATTRIBUTE_NAME =..., GLUE2_ATTRIBUTE_NAME =... } ] } 9

10. JSON Document Structure External context External context identifies what GLUE2 entities are in the document – Context could be a messaging topic or a JSON database e.g. a topic/database that only contains ComputingActivity documents – Implies only one type of entity in a document? Purpose is to remove redundant information that could pose problems when searching or filtering { GLUE2_ATTRIBUTE_NAME =..., GLUE2_ATTRIBUTE_NAME =... } or [ { GLUE2_ATTRIBUTE_NAME =..., GLUE2_ATTRIBUTE_NAME =... },... { GLUE2_ATTRIBUTE_NAME =..., GLUE2_ATTRIBUTE_NAME =... } ] 10

11. GLUE2 JSON Usage FutureGrid – A distributed test-bed that is part of XSEDE – Publishing GLUE2 information about clusters and clouds Didn’t need to add any significant extensions to support IaaS clouds – Using a JSON rendering of GLUE2 Very similar to the previously described approach – 3 documents: System descriptions Jobs/VMs being managed (list of ComputingActivity) Job/VM state updates (single ComputingActivity) Texas Advanced Supercomputing Center – Monitoring configuration and load of local HPC systems for web and mobile interfaces – Same implementation as FutureGrid Shiraz? 11