1. 1 Eric Guilyardi and the Metafor team Common Metadata for Climate Modelling Digital Repositories Metafor Dissemination Workshop Abingdon, 14 March 2011

2. 2 Outline Why Metafor Metafor goals How we are doing it What we have done – Common Information Model (CIM) – Controlled vocabulary – CMIP5 support and questionnaire Next steps

3. 3 Finding climate model data is hard Understanding data is harder (esp. for non-experts) Discriminating between two simulations/models is not easy Documentation is “patchy” and specific to modelling centres Documentation currently revolves around (at best) the runtime, but not the scientific detail and relevance of the model components Little or no documentation of the “simulation context”, i.e. the whys and wherefores and issues associated with any particular simulation. Why Metafor Current issues (before Metafor):

4. 4 Why Metafor 20 modelling centres 40+ models 60 numerical experiments 90,000 years of simulation 2 million output datasets Data to be available from “core-nodes” and “modelling- nodes” in a global federation (PetaBytes of data). Users need to find datasets, and discriminate between models, and between simulation characteristics. Example: the CMIP5 documentation challenge

5. Discovery, Documentation, Definition Why Metafor

6. CF-NETCDF IS-ENES Metrics Metafor-CIM GEOSS Journal Paper Discovery, Documentation, Definition & examples Why Metafor

7. 7 “The main objective of Metafor is to develop a Common Information Model (CIM) to describe climate data and the models that produce it in a standard way, and to ensure the wide adoption of the CIM” Requirements for success: Gather top field experts Engage with similar existing activities Work towards community adoption Capture wider community needs Ensure post-project governance Why Metafor

8. 8 Facts and Figures 12 partners EU contribution of 2.2M€ Started March 2008, duration 3.5 years – NCAS, University of Reading, UK (Coordinator) – BADC, Science and Technology Facilities Council, UK – CERFACS, France – Models and Data, Max Planck Institute for Meteorology, Germany – Institute Pierre-Simon Laplace, CNRS, France – University of Manchester, UK – Met Office, UK – Administratia Nationala de Meterologie, Romania – Météo France, CNRM, France – CLIMPACT, France – CICS, Princeton University, USA – University of Cantabria, Spain INFRA-2007-1.2.1 Scientific Digital Repositories

9. 9 METAFOR Year 1 meeting in Abingdon Feb. 2009

10. 10 Where Metafor came from PRISM project (FP5 2001-2004), ENES PRISM Sustained Initiative (PSI) – Code coupling and I/O – Integration and modelling environments – Data processing and management – Meta-data standards (key !) – Computing issues Metafor IS-ENES IS-ENES, Metafor ENES coordination

11. 11 Metafor organisation Metafor activities and work packages (WP) map onto the I3 structure. Project management, training and dissemination are organised in WP1 and WP7.

12. 12 the CIM builds on existing metadata standards used internationally in climate research (CF, NMM, Curator, FLUME, ISO-standards, etc.) + new bits the CIM defines a general structure over which a specific Controlled Vocabulary (CV) can be applied a CV consists of the terms (and their relationships) used to build the content of CIM instances. The CIM

13. 13 Lots of people talk about climate models and data; Some people even agree about those things (“the yolk”); We have a formal way of describing that (UML, CONCIM); That UML is constrained to follow a particular meta-model......so that it can be transformed into something usable (XSD, APPCIM) for particular users; Metadata instances conform to an APPCIM The CIM “metamodel”

14. 14 The CONCIM Climate Modelling = an activity using a software to produce data on a grid to be archived in a repository.

15. 15 The CONCIM Software Data Activity Grid CIM v 1.4 available on the Metafor website at: http://metaforclimate.eu/trac/browser/CIM http://metaforclimate.eu/trac/browser/CIM

16. 16 Conceptual Model Climate Modelling = an activity using a software to produce data on a grid to be archived in a repository. UML Application Model Application Model XSD RDF Instance @ PCMDI XML An essential aim of Metafor is that the conceptual model is not changed by the manner in which it is used or applied. Instance @ IPSL Instance @ BADC e.g. CIM e.g. CMIP5 The CONCIM to APPCIM More in Allyn’s presentation

17. 17 The Query and other CIM Tool(s) building a search interface for CIM instances building a CIM instance viewer building a CIM instance comparer More in Mark’s presentation

18. 18 Using the CIM to support CMIP5 The CMIP5 experimental archives will be ~500TB of model run data We need to be able to capture all the details of these experiments (and the component models and platforms used) to allow users of the archive to differentiate between the experiments and the models. To do this, Metafor has been tasked by WGCM/CMIP to define, collect and provide the CMIP5 model metadata.

19. 19 Novelty in community: Software CV Gathering a way of describing (in a community consistent way) the scientific properties of model subcomponents Controlled Vocabulary 570 controlled questions - hundred of choices ! Creating the CV mindmap

20. 20 Ocean Lateral physics CV More in Sébastien’s presentation

21. CMIP5 metadata collection CMIP5 Questionnaire @BADC More in Gerry’s presentation

22. ESG Federation Architecture for CMIP5

23. 23 How it all fits together Creating the CV mindmap CMIP5 Questionnaire @BADC CIM+DOI database Portals for user access (search & other tools) User perspective and access to data via ESGF

24. 24 Upcoming challenges CMIP5 questionnaire support – Deployed since November 2010 – Growing use (20 groups) – Feedback on CIM concepts Provide first CIM tools and services (WP4/5/6) – Articulate developments with IS-ENES and ESG Community outreach and buy-in – Develop CIM for other uses (Charlotte’s presentation) Set up long-term governance for CIM and CV – “standards committee” under WCRP

25. 25 “The open standard developed in Metafor will play a catalytic role in the way next generation climate data repositories, such as IPCC AR5, are organised, preserved and accessed” - METAFOR project proposal More on: http://metaforclimate.eu http://metaforclimate.eu “METAFOR is now a major international focal point for earth system modelling metadata definition” - Karl Taylor, PCMDI “The right people for the right project at the right time !” “There is evidence of excellent teamwork” - EU review A few quotes...