1. Lattice 2004Chris Maynard1 QCDml Tutorial How to mark up your configurations

2. Lattice 2004Chris Maynard2 Contents  FAQs on using XML schema  Defining QCDml –Namespaces and validation  Example XML IDs –Ensemble and config Ensemble: actions, algorithms and management metadata Config: what goes where  Babble about BinX  Metadata catalogue demo

3. Lattice 2004Chris Maynard3 FAQs about XML schema  What is XML schema? –Collection of rules for XML documents –An XML schema is itself an XML document  Why do we need an XML schema? –Computers can read and understand XML IDs – 16 –Meaning of length is context dependent  Do I need to learn XML schema –No. Schema makes it easier to produce XML

4. Lattice 2004Chris Maynard4 QCDml1.0  Metadata split into two schemata –Ensemble XML –Config XML N.B. use lowerCamelConvention  ILDG website for XML schema files –http://www.lqcd.org/ildghttp://www.lqcd.org/ildg –Go to Metadata and follow links –Version 1.0 online and ready to use

5. Lattice 2004Chris Maynard5 Namespaces  Example XML ID for UKQCD data  XML Namespace defined by W3.org as  A collection of names identified by a URI reference

6. Lattice 2004Chris Maynard6 First namespace  URI defines namespace for QCDml  This is the default namespace  All elements of QCDml belong to this namespace

7. Lattice 2004Chris Maynard7 Second namespace  Namespace of XML schema itself  Prefix for elements of XML schema  XML ID is valid against WC3 XML schema

8. Lattice 2004Chris Maynard8 SchemaLocation  The namespace of the schema  The file which contains the schema  URI namespace can be URL of the schema instance – not compulsory

9. Lattice 2004Chris Maynard9 Logical filename  Unique URI for a file in a namespace  Uniquely identifies this ensemble in ILDG namespace

10. Lattice 2004Chris Maynard10 Validation  Verify XML ID is valid against a schema –Schema aware applications can use XML ID  Can write XML in vi,emacs etc  CMM uses XMLSpy for schema and ID manipulation –built in validator, create XML ID from schema  http://www.w3.org/XML/Schema http://www.w3.org/XML/Schema –Many different tools

11. Lattice 2004Chris Maynard11 QCDml Ensemble Split into quark and gluon sections UML representation of schema

12. Lattice 2004Chris Maynard12 Ensemble XML - actions  Inheritance tree - check for your action in schema

13. Lattice 2004Chris Maynard13 Which elements?  Schema defines required elements  UKQCD NP clover

14. Lattice 2004Chris Maynard14 UKQCD Ensemble example Glossary: not computer readable How c SW was determined References etc

15. Lattice 2004Chris Maynard15 NumberOfFlavours Number of degenerate flavours for which these coupling values apply

16. Lattice 2004Chris Maynard16 MILC 2+1 staggered Ensemble is array valued Non-degenerate flavours shown with different couplings Mass 0.02 Mass 0.05

17. Lattice 2004Chris Maynard17 Management  Metadata created when Ensemble registered with ILDG  Yet to be created middleware will do this

18. Lattice 2004Chris Maynard18 Algorithm  Algorithmic metadata split between ensemble and algorithm  Most metadata is unconstrained parameter pairs  Relevant information can be found –Glossary document for references etc  Hierarchical structure for algorithms is –difficult to create –difficult to make extenisble

19. Lattice 2004Chris Maynard19 Algorithm: Example Glossary for detailed information Unconstrained parameter pairs

20. Lattice 2004Chris Maynard20 Config XML Machine and code details In principle these could be different for configurations in the same ensemble

21. Lattice 2004Chris Maynard21 Config Management Checksum for config binary Zeroeth is generate data, as this occurs before submission to ILDG

22. Lattice 2004Chris Maynard22 Precision Precision (double or float) in which the calculation was done

23. Lattice 2004Chris Maynard23 markovStep Logical File name of the ensemble in the ILDG namespace

24. Lattice 2004Chris Maynard24 dataLFN Logical File name of the configuration in the ILDG namespace

25. Lattice 2004Chris Maynard25 The markov chain Where the configuration is in the trajectory of markov chain

26. Lattice 2004Chris Maynard26 avePlaquette Very useful metadata, can be used to check data transformations are correct

27. Lattice 2004Chris Maynard27 Config: UKQCD example Application codes can write this info either as QCDml Or tool can convert the IO to QCDml

28. Lattice 2004Chris Maynard28 BinX  XML markup for binary data  Library for manipulating marked up data  Production codes do not use BinX library –But easy to mark up data format in BinX style –ILDG middleware can use BinX for data manipulations –http://www.edikt.org/binxhttp://www.edikt.org/binx  BinX under discussion by Middleware + Metadata WG for file format.

29. Lattice 2004Chris Maynard29 Gauge config BinX Small Written once per ensemble write code on top of BinX library Change array order 2x3  3x3 average plaquette ILDG BinX based gauge config manipulator?

30. Lattice 2004Chris Maynard30 Correlator data Compact. No standard shape to correlators BinX will read in any shape

31. Lattice 2004Chris Maynard31 Array stripper BinX + BJ’s Xpath reader Code reads this XML Produces single slice array in text/XML From any size/shape array Schema for correlator channels ILDG middleware extract channel from any correlator

32. Lattice 2004Chris Maynard32 Correlator dictionary  Possible QCDml extension  Correlator AP code knows channel details –IO AP write dictionary Channel n is zero p pion  User requests pion –Stripper reads dictionary to find pion –Pulls channel n from correlator  Very easy to read other peoples data!

33. Lattice 2004Chris Maynard33 Metadata demonstration  UKQCD metadata catalogue –Browser is based on OGSA-DIA –Open source You can get it at www.forge.nesc.ac.ukwww.forge.nesc.ac.uk  Browser reads the schema –Build XPath query graphically –Result handler Display XML and GET data Render web page of results? Create XML IDs?

34. Lattice 2004Chris Maynard34 ILDG metadata  ILDG proposal: –All collaborations publish metadata  Example method –UKQCD metadata catalogue access is not authenticated –Anyone can read it  ILDG aggregation of metadata catalogues –Mark up data in QCDml –No extra effort required.