Overarching System Team Cecelia DeLuca NOAA Environmental Software Infrastructure and Interoperability (NESII) NOAA ESRL/University of Colorado HIWPP/NGGPS.

Slides:



Advertisements
Similar presentations
NSF NCAR | NASA GSFC | DOE LANL ANL | NOAA NCEP GFDL | MIT | U MICH Emergence of the Earth System Modeling Framework NSIPP Seasonal Forecast.
Advertisements

Earth System Curator Spanning the Gap Between Models and Datasets.
Metadata Development in the Earth System Curator Spanning the Gap Between Models and Datasets Rocky Dunlap, Georgia Tech.
Systems Engineering in a System of Systems Context
21 July 20054th ESMF Community Meeting Overview of the Battlespace Environments Institute Rick Allard Naval Research Laboratory Stennis Space Center, MS.
1 CW2015, Manchester, 04/ Working Session II - Future Issues Working Session II – Future Issues: interoperability, sharing of models/infrastructure,
Overview of NEMS infrastructure Jun Wang Mark Iredell NEMS-NMMB tutorial April 1,
Integrated Frameworks for Earth and Space Weather Simulation Timothy Killeen and Cecelia DeLuca National Center for Atmospheric Research, Boulder, Colorado.
Mesoscale & Microscale Meteorological Division / NCAR ESMF and the Weather Research and Forecast Model John Michalakes, Thomas Henderson Mesoscale and.
1 NGGPS Dynamic Core Requirements Workshop NCEP Future Global Model Requirements and Discussion Mark Iredell, Global Modeling and EMC August 4, 2014.
Common Infrastructure for Modeling the Earth
Interim Review Cupid: An IDE for Model Development and Modeler Training Cecelia DeLuca 1, Rocky Dunlap 2, Spencer Rugaber 2 1 NOAA ESRL/University of Colorado.
Tolman, May 7, 2015MAPP Webinar, 1/14 Transition to operations at EMC What works and what does not work Hendrik L. Tolman Director, Environmental Modeling.
RUP Fundamentals - Instructor Notes
NSF NCAR | NASA GSFC | DOE LANL ANL | NOAA NCEP GFDL | MIT | U MICH First Field Tests of ESMF GMAO Seasonal Forecast NCAR/LANL CCSM NCEP.
1 NOAA’s Environmental Modeling Plan Stephen Lord Ants Leetmaa November 2004.
CCSM Software Engineering Coordination Plan Tony Craig SEWG Meeting Feb 14-15, 2002 NCAR.
Project Overview GMAO Seasonal Forecast NCAR/LANL CCSM NCEP Forecast GFDL FMS Suite MITgcm NASA GMAO Analysis Climate Data Assimilation.
Metadata for the Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS) using the Earth System Modeling Framework (ESMF) Peter Bosler University.
NE II NOAA Environmental Software Infrastructure and Interoperability Program Cecelia DeLuca Sylvia Murphy V. Balaji GO-ESSP August 13, 2009 Germany NE.
ESMF Development Status and Plans ESMF 4 th Community Meeting Cecelia DeLuca July 21, 2005 Climate Data Assimilation Weather.
Initial Results from the Integration of Earth and Space Frameworks Cecelia DeLuca/NCAR, Alan Sussman/University of Maryland, Gabor Toth/University of Michigan.
1 Addressing Critical Skills Shortages at the NWS Environmental Modeling Center S. Lord and EMC Staff OFCM Workshop 23 April 2009.
Overarching System and NEMS Cecelia DeLuca NOAA Environmental Software Infrastructure and Interoperability (NESII) NOAA ESRL/University of Colorado NGGPS.
1 NUOPC National Unified Operational Prediction Capability 1 Review Committee for Operational Processing Centers National Unified Operational Prediction.
1 Reanalysis Coupled Data Assimilation at NCEP Presented By Suru Saha (EMC/NCEP) Contributors: Jack Woollen, Daryl Kleist, Dave Behringer, Steve Penny,
February 2012 Milestone Materials Implicit coupling design document NUOPC Layer software prototype bundled with ESMF Updated NUOPC Layer reference manual.
1 Global Model Development Priorities Presented By: Hendrik Tolman & Vijay Tallapragada (NWS/NCEP) Contributors: GCWMB (EMC), NGGPS (NWS)
CESM/ESMF Progress Report Mariana Vertenstein NCAR Earth System Laboratory CESM Software Engineering Group (CSEG) NCAR is sponsored by the National Science.
Software Engineering Committee Status Report: Preliminary Findings and Recommendations Richard Loft and Gerry Wiener SE Committee Co-chairs National Center.
1 Coupled Modeling for Seasonal to Interannual Presented By Suru Saha (EMC/NCEP) Contributors: Jack Woollen, Daryl Kleist, Dave Behringer, Steve Penny,
DTC HWRF Task AOP2009 & AOP /01/2009 Ligia Bernardet.
Earth System Modeling Framework Status Cecelia DeLuca NOAA Cooperative Institute for Research in Environmental Sciences University of Colorado, Boulder.
Tolman, May 13, 2015JCSDA NWS overview, 1/11 JCSDA Activities and Plans NWS overview & plans Hendrik L. Tolman Director, Environmental Modeling Center.
Ligia Bernardet, S. Bao, C. Harrop, D. Stark, T. Brown, and L. Carson Technology Transfer in Tropical Cyclone Numerical Modeling – The Role of the DTC.
PoDAG XXI: SEEDS SEED: NSIDC Potential Interactions NSIDC DAAC should prepare an evaluation of their desired future roles in "core activities" and in mission.
Slides for NUOPC ESPC NAEFS ESMF. A NOAA, Navy, Air Force strategic partnership to improve the Nation’s weather forecast capability Vision – a national.
ESMF Strategic Discussion Cecelia DeLuca NOAA ESRL/University of Colorado ESMF Executive Board/Interagency Meeting June 12, 2014.
NOAA Intra-Seasonal to Interannual Prediction (ISIP) and Climate Prediction Program for Americas (CPPA) Jin Huang NOAA Office of Global Programs November.
1 Group 1: Modeling and Data Assimilation Summary: 1. HPC access, HPC access, HPC access 2. resources: personnel inside (e.g. EMC) and externally funded.
1 NEMS and NGGPS integration Presented By: Hendrik Tolman (NWS/NCEP/EMC) Contributors: EMC senior staff.
CTB computer resources / CFSRR project Hua-Lu Pan.
1 National Environmental Modeling System (NEMS) Status M. Iredell and EMC Staff.
Rational Unified Process Fundamentals Module 4: Core Workflows II - Concepts Rational Unified Process Fundamentals Module 4: Core Workflows II - Concepts.
Earth System Curator and Model Metadata Discovery and Display for CMIP5 Sylvia Murphy and Cecelia Deluca (NOAA/CIRES) Hannah Wilcox (NCAR/CISL) Metafor.
Building Community and Capability through Common Infrastructure: ESMF and the Earth System Curator Cecelia DeLuca MAP Meeting College.
Transitioning unique NASA data and research technologies to operations Short-term Prediction Research and Transition (SPoRT) Project Future Directions.
ESMF Change Review Board Robert Ferraro ESMF Board Meeting Dec 17, 2008 Climate Data Assimilation Weather.
Emergence of a Common Modeling Architecture for Earth System Science American Geophysical Union December 13, 2010 Cecelia DeLuca NOAA/CIRES.
State of ESMF: The NUOPC Layer Gerhard Theurich NRL/SAIC ESMF Executive Board / Interagency Working Group Meeting June 12, 2014.
DTC Overview Bill Kuo September 25, Outlines DTC Charter DTC Management Structure DTC Budget DTC AOP 2010 Processes Proposed new tasks for 2010.
NOAA Environmental Modeling System Cecelia DeLuca NOAA Environmental Software Infrastructure and Interoperability (NESII) NOAA ESRL/University of Colorado.
Nesting Milestone Materials Nesting design document Updated NUOPC Layer software prototype bundled with ESMF Updated NUOPC Layer reference manual Nesting.
The NOAA Environmental Modeling System at NCEP Mark Iredell and the NEMS group NOAA/NWS/NCEP Environmental Modeling Center June 12, 2014.
Metadata Development in the Earth System Curator Spanning the Gap Between Models and Datasets Rocky Dunlap, Georgia Tech 5 th GO-ESSP Community Meeting.
A Professional Development Series from the CDC’s Division of Population Health School Health Branch Professional Development 101: The Basics – Part 1.
A Quick Tour of the NOAA Environmental Software Infrastructure and Interoperability Group Cecelia DeLuca Dr. Robert Detrick visit March 28, 2012
Project Management Approach
OAS Requirements Experience
Overarching System Team
GMAO Seasonal Forecast
UNIFIED GLOBAL COUPLED SYSTEM (UGCS) FOR WEATHER AND CLIMATE PREDICTION Saha-UMAC-09Aug2016.
NGGPS NGGPS Priorities: the three legs of the stool
Building a Distributed Earth System Model Community
ESPC Air-Ocean-Land-Ice Global Coupled Prediction
ESMF Governance Cecelia DeLuca NOAA CIRES / NESII April 7, 2017
Mariana Vertenstein (CGD)
Joint GEOS-Chem and NCAR Modeling Workshop:
Metadata Development in the Earth System Curator
A brief introduction to NEMS
Presentation transcript:

Overarching System Team Cecelia DeLuca NOAA Environmental Software Infrastructure and Interoperability (NESII) NOAA ESRL/University of Colorado HIWPP/NGGPS Program Meeting February 10, 2016

Objectives 1.Provide a portable, high performance, unified software infrastructure for use in operational prediction models at NCEP. 2.Coordinate and provide to NCEP a document on code, [data, and documentation] management for NEMS-based modeling applications and suites. 3.Promote communication and coordinated software development across the NGGPS teams and related efforts at EMC and NOAA.

Unified Software Infrastructure Coupled NEMS Deliveries Milestones Define and deliver a sequence of increasingly capable coupled modeling applications Delivery schedule for NEMS applications: bUOGM0frNPEMIO_ND28/edit#gid=0 bUOGM0frNPEMIO_ND28/edit#gid=0 Applications and activities include (but are not limited to): Unified Global Coupled System (UGCS) – Seasonal Scale Regional-Hydrology Whole Atmosphere Model – Ionosphere Plasmasphere Electrodynamics Integration Unified Global Coupled System – Weather Scale (HYCOM integration)

Unified Software Infrastructure UGCS-Seasonal 0.2 Milestone atm: GSM, ocean: MOM5, sea ice: CICE, coupler: NEMS mediator Technically valid 3-way interaction, capable of 9 month run Target September 2015 Status/Deliveries Running 15 days with full set of three-way exchange fields Testing 30 day runs but need restart to go longer Waiting on: Investigation of GSM memory growth every 6 th timestep/hour GSM version which has a restart capability Cold start implementation sequence (NESII and EMC) Anticipating February 2016 delivery if GSM issues are resolved

Unified Software Infrastructure UGCS-Seasonal 0.2 Lead and Collaborators Collaboration of NESII team (Fei Liu, Gerhard Theurich, Anthony Craig, Cecelia DeLuca, others), EMC (Xingren Wu, Jiande Wang, Bin Li, Mark Iredell, Suru Saha, Patrick Tripp, others), NCAR (David Bailey), COLA (Larry Marx, Jim Kinter), GFDL (Niki Zadeh), many others NESII organizes weekly calls with about 50 on distribution, high attendance

Unified Software Infrastructure UGCS-Seasonal 0.2 Sample results at 15 days GSM: T126 MOM5 and CICE: 0.5 degree with 0.25 degree tropics SST (K) 2m air temperature (K)

Unified Software Infrastructure Regional-Hydro 0.2 Milestone atm: GSM, ocean: MOM5, sea ice: CICE, land: LIS/Noah, hydro: WRF-Hydro, coupler: NEMS mediator Technically valid 5-way interaction Target December 2015 Status/Deliveries Delivery within the next week – release page: Lead and Collaborators Collaboration of NESII team (Rocky Dunlap, Dan Rosen, others), EMC (Jairui Dong, Mike Ek, others), NCAR (David Gochis, Wei Yu), NASA GSFC (Sujay Kumar, Jim Geiger), GLERL (Drew Gronewald), Navy (Sue Chen) NESII organizes weekly calls with about 20 on distribution

Components in NEMS Mediator Model: ROF Model: WAV Model: GLC Model: HYD Model: WAV Model: LND NEMS Mediator NUOPC Driver Model: ATM Model: ICE Model: OCN Model: IPE Model: ICE Model: OCN Model: ATM Model: ATM Mediator Space Wx Mediator Model: GLC Model: CHM Additional activities include space weather, wave, and aer/chem integration, with application milestones for each of these later in 2016.

Unified Software Infrastructure Software Training Milestone Provide an initial NUOPC/NEMS training offering in collaboration, target December 2015 Deliveries Developed one-day basic training course: Training courses at EMC on November 12-13, 2015 (8 trainees), January 28, 2016 (15 trainees), and January 29, 2016 (15 trainees) Additional courses will be provided as needed, along with sessions on more specialized topics Leads and Collaborators Fei Liu (NRL) and Rocky Dunlap (NOAA NESII) were instructors Suru Saha coordinated at EMC; EMC management provided input on the class structure and curriculum

Unified Software Infrastructure Software Training Delivery of guide on how to adapt a model component for NUOPC interfaces, November 2015:

Unified Software Infrastructure Code and Repository Management Milestones Complete draft of code management document in September 2015 Deliveries Completed draft code management document and extended it to include documentation and data management sections XIAUeThUPWD9Tg/edit#heading=h.ku78qulk21xh XIAUeThUPWD9Tg/edit#heading=h.ku78qulk21xh Plan to disseminate more broadly across NGGPS in the next month Leads and Collaborators Contributors/reviewers include DTC (Bernadet, Carson), ESRL (DeLuca, Theurich, Liu), EMC (Iredell, Tolman, Saha), NCAR CESM (Large, Vertenstein), VLab (Sperow), COLA (Kinter, Marx)

Unified Software Infrastructure Code and Repository Management Code, Data, and Documentation Management for NEMS Modeling Applications and Suites Table of Contents (sample topics) Terminology and Background Modes of Use and Implications Software Requirements and Expectations Tools and Options: Collaboration Environments and Workflow Software Tools and Options: Repository Software and Services Repository Access and Use Procedures NEMS Repository Guidelines for Software Checkins and Updates Proposed Processes for Coordinating Software Across NEMS Documentation Requirements and Expectations Tools and Options: Preparation of Documentation Documentation Current Practice and Recommended Evolution Input Data Requirements and Expectations Pilot Projects Acronyms

Unified Software Infrastructure Communication and Coordination Milestones Communication and coordination across NGGPS as needed Status/Deliveries Initiated documentation survey and gap analysis: Logical counterpart to development of documentation policies in the Code, Data, and Documentation Management Document Leads and Collaborators Organized mainly by DTC (Bernadet) and NESII (DeLuca, Liu); others from EMC, ESRL and elsewhere have been contributing ideas and information

Unified Software Infrastructure Communication and Coordination Milestones Communication and coordination across NGGPS as needed Status/Deliveries Initiated mailing lists (thanks to Steve Warren): New request: ALL NGGPS Rationale Effective, broad communication pathways are needed for coordination Need to easily and consistently communicate with people in specific roles (e.g. model component leads, NGGPS team leads)

Unified Software Infrastructure Communication and Coordination Milestones EMC would benefit from formation of NEMS “Change Review Board,” populated with NEMS application leads Goal is to manage shared technical resources effectively in the delivery of NGGPS and other milestones Status/Deliveries Completed draft terms of reference Tentative first meeting on February 16 – waiting on EMC to complete scheduling Lead and Collaborations EMC management leads, NESII support

Summary Major Accomplishment Physically reasonable 15+ day runs of coupled atmosphere-ocean-ice system – many issues remain, and in-depth analysis of behavior has not really begun Priority Focus Effort for FY2016 Training at EMC; support for delivery of applications Most Important Issue or Coordination Need Need for community support and analysis of the coupled system – for operational workflows and research workflows Believe this would benefit from discussion including EMC, DTC, CESM, COLA, and NESII Proposed “CIME” pilot with NCAR is important in this respect– leverages existing coupled system research workflow and components

Unified Software Infrastructure Strategic Motivation for CIME Pilot CESM already includes the 7+ components that NEMS anticipates integrating, plus the target dynamical cores (finite volume and MPAS). There is tremendous expertise in CGD in having spent decades in making that very complex coupled system work. NEMS applications will need a community-friendly environment for coupled model development. That requires the kinds of many- component coupled model management tools that CESM has developed over decades. It seems advantageous to leverage what has already proven valuable, accessible, and familiar to a broad coupled modeling community. It will benefit NEMS application developers (and collaborators such as DTC) to have a standing relationship with scientists with established expertise in many-component coupled model development, evaluation, and governance.

Unified Software Infrastructure Community Mediator - CIME Pilot Project CIME (Common Infrastructure for Modeling the Earth) is a github community repository for storing infrastructure software Created by the CESM team but not specific to CESM software Pilot would store the NEMS mediator in CIME, where it can be treated like other community-developed components NEMS components can be run within research workflows, with access to tests and diagnostics NEMS mediator can be scrutinized by coupling experts and tested with a variety of community components NUOPC interfaces provide a link back to operational systems Issues with license and access requirements must be addressed

Overarching System Team Cecelia DeLuca ESRL/CIRES/NESII Ligia Bernadet NCAR DTC Anthony Craig contracting for NESII Jim Doyle NRL MRY Mark Iredell NCEP EMC John Michalakes NOAA NWS Gerhard Theurich Fei Liu NRL/NESII Mariana Vertenstein NCAR CGD/CESM Valbona Kunkel NCEP EMC Patrick Tripp NCEP EMC In coordination with + EMC and external model component leads + modeling application leads, including NGGPS science/product lead

Supplementary Material

NEMS Basics The NOAA Environmental Modeling System (NEMS) is infrastructure for building coupled modeling systems ‐Examples of other coupled modeling systems: UK Met Office Unified Model, Community Earth System Model (CESM) NEMS is associated with a collection of model components External model components have a primary repository that is not at EMC In general, model components exchange data using the main NEMS mediator – often called a "coupler“ Custom NEMS mediators are being built for special interactions, such as optimized 3D coupling of the upper atmosphere to the ionosphere Introduction to NEMS:

NEMS Modeling Applications NEMS can be assembled into a number of different modeling applications, each associated with: ‐a purpose, such as seasonal forecasting ‐a set of model components ‐a set of parameters that represents a range of supported options, including grids and resolutions Different NEMS modeling applications can have different types and numbers of model components The same physical domain may be represented by different model components in different modeling modeling applications: ‐For example, in some NEMS modeling applications the ocean component may be HYCOM and in others it may be MOM5 Spreadsheet of NEMS modeling applications: bUOGM0frNPEMIO_ND28/edit#gid=0 bUOGM0frNPEMIO_ND28/edit#gid=0

NEMS Infrastructure NEMS is built using Earth System Modeling Framework (ESMF) infrastructure software, which provides: generation and application of interpolation weights, time management classes, and other utilities data structures for representing fields, grids, and model components in a standard way The National Unified Operational Prediction Capability (NUOPC) Layer increases interoperability by adding behavioral rules to ESMF, including: ‐a standard way of representing build dependencies ‐a standard syntax for initialization and run phases NUOPC wrappers or “caps” contain translations of native data structures (e.g. grids, field data, time quantities) into ESMF data structures. ESMF site: NUOPC Layer site: Performance reports: