- EGU 2010 ESSI5 - 07 May 2010 - 1 Building on the CMIP5 effort to prepare next steps : integrate community related effort in the every day workflow to.

Slides:

Advertisements

Similar presentations

ESGF and ES-DOC Documenting climate models and their simulations ES-DOC current and future plans Working with ESGF Eric Guilyardi, Balaji, Cecelia DeLuca,

Advertisements

Climate Analytics on Global Data Archives Aparna Radhakrishnan 1, Venkatramani Balaji 2 1 DRC/NOAA-GFDL, 2 Princeton University/NOAA-GFDL 2. Use-case 3.

CMIP5: Overview of the Coupled Model Intercomparison Project Phase 5

Geophysical Fluid Dynamics Laboratory Review June 30 - July 2, 2009 Geophysical Fluid Dynamics Laboratory Review June 30 - July 2, 2009.

Part 1a: Overview of the UM system

NCAS Unified Model Introduction Part 1a: Overview of the UM system University of Reading, 3-5 December 2014.

Sergey Belov, LIT JINR 15 September, NEC’2011, Varna, Bulgaria.

Institut Pierre-Simon Laplace (IPSL) Bringing together key laboratories of the Paris area in the field of environmental studies.

CLIMATE SCIENTISTS’ BIG CHALLENGE: REPRODUCIBILITY USING BIG DATA Kyo Lee, Chris Mattmann, and RCMES team Jet Propulsion Laboratory (JPL), Caltech.

Overview of Coupled Model Intercomparison Project (CMIP) and IPCC AR5 Activities Ronald J Stouffer Karl Taylor, Jerry Meehl and many others June 2009.

TPAC Digital Library Talk Overview Presenter:Glenn Hyland Tasmanian Partnership for Advanced Computing & Australian Antarctic Division Outline: TPAC Overview.

Sergey Belov, Tatiana Goloskokova, Vladimir Korenkov, Nikolay Kutovskiy, Danila Oleynik, Artem Petrosyan, Roman Semenov, Alexander Uzhinskiy LIT JINR The.

Z EGU Integration of external metadata into the Earth System Grid Federation (ESGF) K. Berger 1, G. Levavasseur 2, M. Stockhause 1, and M. Lautenschlager.

Eric Guilyardi (LOCEAN/IPSL and Univ. Reading) and the Metafor team Common Metadata for Climate Modelling Digital Repositories IS-ENES kick-off meeting.

Grid for Coupled Ensemble Prediction (GCEP) Keith Haines, William Connolley, Rowan Sutton, Alan Iwi University of Reading, British Antarctic Survey, CCLRC.

CCSM Portal/ESG/ESGC Integration (a PY5 GIG project) Lan Zhao, Carol X. Song Rosen Center for Advanced Computing Purdue University With contributions by:

1 Eric Guilyardi and the Metafor team Common Metadata for Climate Modelling Digital Repositories Metafor Dissemination Workshop Abingdon, 14 March 2011.

IS-ENES [ees-enes] InfraStructure for the European Network for Earth System Modelling IS-ENES will develop a virtual Earth System Modelling Resource Centre.

IS-ENES Kick-off meeting Paris, March 2009 Overview of JRA2 European ESM: Performance Enhancement Graham Riley, University of Manchester IS-ENES Kick-off.

CIM – The Common Information Model in Climate Research

Presented by The Earth System Grid: Turning Climate Datasets into Community Resources David E. Bernholdt, ORNL on behalf of the Earth System Grid team.

Initiatives toward Climate Services in France and in the European Communities C. Déandreis (CNRS/IPSL); M. Plieger and W. Som de Cerff (KNMI); Ph. Dandin,

Integrating HPC into the ATLAS Distributed Computing environment Doug Benjamin Duke University.

Using ICENI to run parameter sweep applications across multiple Grid resources Murtaza Gulamali Stephen McGough, Steven Newhouse, John Darlington London.

DRIAS Project: providing Regional Climate Informations over France Julien Lémond 1-4, Ph. Dandin 1, S. Planton 4, R. Vautard 3, C. Pagé 2, M. Déqué 4,

Michael Lautenschlager World Data Center Climate Model and Data / Max-Planck-Institute for Meteorology German Climate Computing Centre (DKRZ)

- IS-ENES SA2/JRA4 24 th February IS-ENES SA2/JRA4 IPSL Status/Questions Sébastien Denvil, Ashish Bhardwaj, Mark Morgan IPSL, Global Climate Modeling.

기후모델 : 기후변화연구의 인프라 Climate Model: Infrastructure for Climate Change Research Wonsun Park Leibniz Institute of Marine Sciences Kiel, Germany KOFST Ultra.

GEM Portal and SERVOGrid for Earthquake Science PTLIU Laboratory for Community Grids Geoffrey Fox, Marlon Pierce Computer Science, Informatics, Physics.

Data Publication and Quality Control Procedure for CMIP5 / IPCC-AR5 Data WDC Climate / DKRZ:

The Climate-G testbed towards a large scale data sharing environment for climate change S. Fiore Scientific Computing and Operations Division, CMCC, Italy.

An Integrated Systems Solution to Air Quality Data and Decision Support on the Web GEO Architecture Implementation Pilot – Phase 2 (AIP-2) Kickoff Workshop.

Human fingerprints on our changing climate Neil Leary Changing Planet Study Group June 28 – July 1, 2011 Cooling the Liberal Arts Curriculum A NASA-GCCE.

Climate Change Working Group (CCWG) July, 2004 Co-chairs: Gerald A. Meehl, Ben Santer, and Warren Washington.

1 Earth System Modeling Framework Documenting and comparing models using Earth System Curator Sylvia Murphy: Julien Chastang:

CLIMARES WP 110 Climate model scenarios for the Arctic region for the next decades Current state: Klaus Dethloff, AWI WP Leader: Erich Roeckner, MPI Planing.

1 PRogramme for Integrated earth System Modelling (PRISM) An Infrastructure Project for Climate Research in Europe.

- Vendredi 27 mars PRODIGUER un nœud de distribution des données CMIP5 GIEC/IPCC Sébastien Denvil Pôle de Modélisation, IPSL.

ESG Observational Data Integration Presented by Feiyi Wang Technology Integration Group National Center of Computational Sciences.

The Climate-G testbed towards a large scale data sharing environment for climate change S. Fiore Scientific Computing and Operations Division, CMCC, Italy.

May 6, 2002Earth System Grid - Williams The Earth System Grid Presented by Dean N. Williams PI’s: Ian Foster (ANL); Don Middleton (NCAR); and Dean Williams.

The evolution of climate modeling Kevin Hennessy on behalf of CSIRO & the Bureau of Meteorology Tuesday 30 th September 2003 Canberra Short course & Climate.

WP6/SA2: Access to IS-ENES Data Federation SA2 is a European distributed data infrastructure providing access to data from ESM simulations produced in.

Modelling the climate system and climate change PRECIS Workshop Tanzania Meteorological Agency, 29 th June – 3 rd July 2015.

Lautenschlager + Thiemann (M&D/MPI-M) / / 1 Introduction Course 2006 Services and Facilities of DKRZ and M&D Integrating Model and Data Infrastructure.

1 Accomplishments. 2 Overview of Accomplishments  Sustaining the Production Earth System Grid Serving the current needs of the climate modeling community.

1 Overall Architectural Design of the Earth System Grid.

1 Gateways. 2 The Role of Gateways  Generally associated with primary sites in ESG-CET  Provides a community-facing web presence  Can be branded as.

Climate feedback on the marine carbon cycle in CarboOcean Earth System Models J. Segschneider 1, E. Maier-Reimer 1 L. Bopp 2, J. Orr 2 1 Max-Planck-Institute.

Earth System Curator and Model Metadata Discovery and Display for CMIP5 Sylvia Murphy and Cecelia Deluca (NOAA/CIRES) Hannah Wilcox (NCAR/CISL) Metafor.

Impact of solar UV variability on sudden stratospheric warming with LMDz-Reprobus A. Hauchecorne 1, S. Bekki 1, M. Marchand 1, C. Claud 2, P. Keckhut 1,

1 Summary. 2 ESG-CET Purpose and Objectives Purpose  Provide climate researchers worldwide with access to data, information, models, analysis tools,

Welcome to the PRECIS training workshop

WP4 - Strengthening the European Network on Earth System Modelling IS-ENES kick-off meeting – March 30-31, Paris Partners and general objectives CERFACS.

XMC Cat: An Adaptive Catalog for Scientific Metadata Scott Jensen and Beth Plale School of Informatics and Computing Indiana University-Bloomington Current.

Warren M. Washington NCAR The Parallel Climate Model (PCM) and Transition to a Climate Change Version of the Community Climate System Model (CCSM)

Application of RDF-OWL in the ESG Ontology Sylvia Murphy: Julien Chastang: Luca Cinquini:

Metadata Support for Model Intercomparison Projects Sylvia Murphy: Cecelia DeLuca: Julien.

PRODIGUER French Climate Modelling Information Services Platform.

PRODIGUER French Climate Modelling Information Services Platform.

Metafor Year 3 EU Review CIM Component Services. Of itself an Ontology is an inert artefact, i.e. a dictionary. CON-CIM Conceptual CIM.

WGCM19 Report from the French groups

AP7/AP8: Long-Term Archival of CMIP6 Data

VI-SEEM Data Repository

Grid Metadata Management

Ronald J Stouffer Karl Taylor, Jerry Meehl and many others

CMIP6 / ENES Data TF Meeting: DKRZ

Production and use of regional climate model projections at the Swedish Meteorological and Hydrological Institute Erik Kjellström Rossby Centre,

Digital Object Management for ENES: Challenges and Opportunities

Presentation transcript:

- EGU 2010 ESSI May Building on the CMIP5 effort to prepare next steps : integrate community related effort in the every day workflow to lower the data distribution and data management burden Sébastien Denvil, Mark Morgan, Ashish Bhardwaj, Martial Mancip, and Patrick Brockmann Climate Modeling Group, IPSL

- EGU 2010 ESSI May Context : coutdown of the IPCC report  2010  Mid 2011 : Climate simulations  End of 2010  ? : Data distribution  End of 2010  July 2012 : articles submission  September 2013 : IPCC AR5 WG1 plenary session  October 2014 : Nobel Prize?

- EGU 2010 ESSI May Management of data since years in many climate modeling groups Mainly centralized, store on a SAN OpenDap access on Supercomputing Centre Basic system of data retrieval Access to raw data Security/Authentication/Restriction to data access : not an issue No on demand post-processing No metadata integration No support for high level database query

- EGU 2010 ESSI May Emerging requirements for Data management Move the data a minimum, keep them close to supercomputing centres if possible  Data access protocol, strong links with computing centres When data needs to be moved do it quickly and with a minimum amount of human intervention  Management of storage resources, fast network Keep a track of what we got, particularly what is on deep storage  Metadata et data catalogues Exploiting a federation of sites  EarthSystemGrid software stack

- EGU 2010 ESSI May CMIP5 global data amount Raw Data amount lower bound  565 TB Raw Data amount higher bound  1000 TB CMIP5 Distribution (50%)  TB Global Storage (Raw+Distributed)  TB LMDz 0.5° (50 Km)

- EGU 2010 ESSI May Tropospheric chemistry & aerosols (INCA) Carbon / CO 2 (ORCHIDEE, NEMO/PISCES) Stratospheric chemistry / ozone (REPROBUS) Emissions Land use Volcanoes Solar irradiance Physic – Transport Atmosphere (LMDZ) Surface (ORCHIDEE) Ocean (NEMO/OPA) Sea ice (NEMO/LIM2) Coupler (OASIS) IPSL Earth System Model (ESM) Global climate Regional climate Various kind of Model Impacts studies Dynamical Downscaling (RCM) Statistical Downscaling

- EGU 2010 ESSI May Earth System Grid Federation to support CMIP5

- EGU 2010 ESSI May National Level: many partners International Level: many partners

- EGU 2010 ESSI May Data Node Architecture

- EGU 2010 ESSI May S 1... S N S 3 S 2 Simulation Execution Environment Input.ini.netCDF.make Events 100=Start 101=Stop Output.ini.netCDF SIMULATION MACHINE

- EGU 2010 ESSI May S 1... S N S 3 S 2 Simulation Execution Environment Input.ini.netCDF.make Events 100=Start 101=Stop Output.ini.netCDF Prodiguer Simulation Services Python (Async) Message Queues (RabbitMQ) Event Monitor Event Publisher SIMULATION MACHINE

- EGU 2010 ESSI May S 1... S N S 3 S 2 Simulation Execution Environment Input.ini.netCDF.make Events 100=Start 101=Stop Output.ini.netCDF Prodiguer Simulation Services Python (Async) Message Queues (RabbitMQ) Event Monitor Event Publisher SIMULATION MACHINE PRODIGUER AGGREGATING DATA NODE FIREWALL Base64

- EGU 2010 ESSI May Meta-Data Publication FRENCH SCIENTIFIC PARTNER/COMPUTING CENTRES DN-1 (CCRT) DN-N (Meteo-France) DN-3 (CERFACS) DN-2 (IDRIS)... (DN=Data Node)

- EGU 2010 ESSI May Meta-Data Publication FRENCH SCIENTIFIC PARTNER/COMPUTING CENTRES DN-1 (CCRT) DN-N (Meteo-France) DN-3 (CERFACS) DN-2 (IDRIS)... CORE (CMIP5)

- EGU 2010 ESSI May Meta-Data Publication FRENCH SCIENTIFIC PARTNER/COMPUTING CENTRES DN-1 (CCRT) DN-N (Meteo-France) DN-3 (CERFACS) DN-2 (IDRIS)... CORE (CMIP5)OPERATIONAL

- EGU 2010 ESSI May Meta-Data Publication FRENCH SCIENTIFIC PARTNER/COMPUTING CENTRES DN-1 (CCRT) DN-N (Meteo-France) DN-3 (CERFACS) DN-2 (IDRIS)... WEB SERVICES (RESTful, AtomPub) CORE (CMIP5)OPERATIONAL

- EGU 2010 ESSI May Meta-Data Publication FRENCH SCIENTIFIC PARTNER/COMPUTING CENTRES DN-1 (CCRT) DN-N (Meteo-France) DN-3 (CERFACS) DN-2 (IDRIS)... DATABASE(S) PostGres, RDF-Triple ESG – GATEWAY WEB SERVICES (RESTful, AtomPub) OPERATIONALCORE (CMIP5)

- EGU 2010 ESSI May Meta-Data Publication FRENCH SCIENTIFIC PARTNER/COMPUTING CENTRES DN-1 (CCRT) DN-N (Meteo-France) DN-3 (CERFACS) DN-2 (IDRIS)... DATABASE(S) PostGres, RDF-Triple ESG – GATEWAY WEB SERVICES (RESTful, AtomPub) PRODIGUER DATABASE(S) PostGres CORE (CMIP5)OPERATIONAL

- EGU 2010 ESSI May Meta-Data Publication FRENCH SCIENTIFIC PARTNER/COMPUTING CENTRES DN-1 (CCRT) DN-N (Meteo-France) DN-3 (CERFACS) DN-2 (IDRIS)... DATABASE(S) PostGres, RDF-Triple ESG – GATEWAY DATABASE(S) eXist, PostGres, RDF METAFOR / IS-ENES WEB SERVICES (RESTful, AtomPub) PRODIGUER DATABASE(S) PostGres CORE (CMIP5)OPERATIONAL

- EGU 2010 ESSI May Meta-Data Publication FRENCH SCIENTIFIC PARTNER/COMPUTING CENTRES DN-1 (CCRT) DN-N (Meteo-France) DN-3 (CERFACS) DN-2 (IDRIS)... DATABASE(S) PostGres, RDF-Triple ESG – GATEWAY DATABASE(S) eXist, PostGres, RDF METAFOR / IS-ENES WEB SERVICES (RESTful, AtomPub) PRODIGUER DATABASE(S) PostGres XML Base64 CORE (CMIP5)OPERATIONAL

- EGU 2010 ESSI May Meta-Data Publication FRENCH SCIENTIFIC PARTNER/COMPUTING CENTRES DN-1 (CCRT) DN-N (Meteo-France) DN-3 (CERFACS) DN-2 (IDRIS)... DATABASE(S) PostGres, RDF-Triple ESG – GATEWAY DATABASE(S) eXist, PostGres, RDF METAFOR / IS-ENES WEB SERVICES (RESTful, AtomPub) PRODIGUER DATABASE(S) PostGres XML HTTPS / X509 XML HTTPS / X509 XML Base64 CORE (CMIP5)OPERATIONAL

- EGU 2010 ESSI May Conclusions European response to climate simulation proliferation has been built in close collaboration with the ESG-CET American consortium. To come in support to CMIP5 require a work on software environment, data storage, their handling, distribution to users AND a work to describe simulations, their contexts, and their results. ESGF, IS-ENES and METAFOR has been built to support this. The every day workflow and then the every day simulation must benefit from the work done to achieve “CMIP5 like” exercise. The aggregating data node approach is the one we choose. It’s an integration activity, leveraging what’s been done to support CMIP5 like activity. Operational Data « CMIP5 like » Data