1. Introduction to the Earth System Modeling Framework International Workshop on Next Generation Climate Models for Advanced High Performance Computing Facilities March 12, 2002 Cecelia DeLuca / Scientific Computing Division / NCAR Chris Hill / Massachusetts Institute of Technology V. Balaji / SGI – NOAA Geophysical Fluid Dynamics Laboratory

2. Outline Background and Motivation ESMF Design Overview ESMF Codes Milestones Development Plan Beyond 2004: ESMF Evolution Additional Information and Events

3. Trends in Earth System Modeling In climate research and numerical weather prediction … Increased emphasis on detailed representation of individual physical processes Requires many teams of specialists to contribute components to a coupled system In computing technology … Increase in complexity and variety in high-performance computing hardware Community Response Abstraction of underlying hardware to provide uniform programming model across architectures Modular design for interchangeable dynamical cores and larger components Development of prototype frameworks - GFDL (FMS), NASA/GSFC (GEMS), others The Earth System Modeling Framework (ESMF) aims to unify and extend these efforts

4. Project Background Collaborators:NASA/GSFC-DAO, NASA/GSFC-NSIPP, LANL, ANL, University of Michigan, MIT, NCAR-SCD, NCAR-CGD, NCAR-MMM, NOAA/NCEP, NOAA/GFDL Term:3 years, start March 2002 Budget:$9.8M NASA’s Earth Science Technology Office (ESTO) proposed the creation of the ESMF in a September 2000 Cooperative Agreement Notice Large collaboration developed three linked proposals in response: Part I: Core ESMF Development (PI: Killeen, NCAR) Part II: Modeling Applications (PI: Marshall, MIT) Part III: Data Assimilation Applications (PI: da Silva, NASA DAO)

5. Project Organization Prognostic Model Deployment NASA ESTO NSF NCAR PIMIT PINASA DAO PI Data Assimilation Deployment Core Framework Development Part I Proposal Specific Milestones Part II Proposal Specific Milestones Part III Proposal Specific Milestones Joint Milestones Part IPart IIPart III Joint Specification Team Requirements Analysis System Architecture API Specification

6. Objectives of the ESMF 1.Facilitate the exchange of scientific codes (interoperability) 2.Promote the reuse of standardized technical software while preserving computational efficiency 3.Focus community resources to preserve performance portability in a volatile computing environment 4. Share overhead costs of the housekeeping aspects of software development 5.Provide greater institutional continuity to model development efforts

7. Framework Architecture Components and Coupling gridded component interface data transfers/coupling collective I/O Fields and Grids Fields field and field bundle data and metadata field I/O Grids physical grid description grid decomposition Parallel Utilities transpose, halo, etc. abstracted machine layout Low-Level Utilities event alarms performance profiling I/O primitives communication primitives, etc.

8. Application Architecture External Libraries Low Level Utilities Fields and Grids Layer Model Layer Coupling Layer ESMF Infrastructure User Code ESMF Superstructure BLAS, MPI, NetCDF, …

9. ESMF Codes: Modeling Applications GFDLFMS B-grid atmosphere at N45L18 FMS spectral atmosphere at T63L18 FMS MOM4 ocean model at 2°x2°xL40 FMS HIM isopycnal C-language ocean model at 1/6°x1/6°L22 MITMITgcm coupled atmosphere/ocean at 2.8°x2.8°, atmosphere L5, ocean L15 MITgcm regional and global ocean at 15kmL30 NSIPPNSIPP atmospheric GCM at 2°x2.5°xL34 coupled with NSIPP ocean GCM at 2/3°x1.25°L20 NCAR/LANLCCSM2 including CAM with Eulerian spectral dynamics and CLM at T42L26 coupled with POP ocean and data ice model at 1°x1°L40

10. ESMF Codes: Data Assimilation Applications DAOPSAS interface layer with 2O0K observations/day CAM with finite volume dynamics at 2°x2.5°L55, including CLM NCEPGlobal atmospheric spectral model at T170L42 SSI analysis system with 250K observations/day, 2 tracers WRF regional atmospheric model at 22km resolution CONUS forecast 345x569L50 NSIPPODAS with OI analysis system at 1.25°x1.25°L20 resolution with ~10K observations/day MITMITgcm 2.8° century / millennium adjoint sensitivity

11. Final Milestones Tested, optimized core framework software and documentation –Support for many platforms, including commodity clusters All ESMF codes will achieve full ESMF compliance –Will use coupling superstructure and utilities 8 Interoperability demonstrations –Example: NCAR CAM with MITgcm ocean –Multiple time scales: NWP, mid-range forecast experiments Coupled seasonal forecasts Interannual/decadal variability Centennial simulations –Will require tuning and validation to be scientifically useful

12. Development Plan Requirements –Intensive interaction involving ESMF collaborators and broader community, ESMF Community Requirements Meeting –High-level results presented in General Requirements Document, detailed requirements documents prepared later Design –Design study includes examination of FMS, GEMS, MCT, and other existing Earth science frameworks –Results presented in Architecture Report, detailed design documents for specific components prepared later –Community review of ESMF interface specification

13. Development Plan, cont. Implementation –Implementation study will determine implementation language, language interoperability strategy, programming model –Examine viability of supporting software tools, e.g. CCA, Cactus –Results presented in Implementation Report Best Practices –Early distribution of Software Developer’s Guide will encourage consistent standards, conventions and practices throughout the project

14. Beyond 2004:ESMF Evolution Maintenance and management –NCAR commitment to maintain and support core ESMF software –Persistence of ESMF Executive Committee and Advisory Board Technical evolution –Functional extension: support for advanced data assimilation algorithms, additional grids, new domains –Earth System Modeling Environment, including web/GUI interface, databases of components and experiments, links to GRID services

15. Additional Information and Upcoming Events ESMF Community Requirements Discussion and Review Open Meeting In conjunction with the Spring American Geophysical Union Meeting May 30, 2002 ESMF website: http://www.esmf.ucar.edu Questions: esmf@ucar.eduesmf@ucar.edu