1. The SciDAC2 CCSM Consortium Project 2007 - 2012 Presented by John Drake Atmospheric Model Working Group Meeting January 31, 2007, NCAR, Boulder

2. The SciDAC2 CCSM Consortium Project: A Scalable and Extensible Earth System Model Lead PI: John B. Drake, Oak Ridge National Laboratory, Co-Lead PI: Phil Jones, Los Alamos National Laboratory Argonne National Laboratory (ANL) Robert Jacob, Ray Loy, Brookhaven National Laboratory (BNL) Robert McGraw; Lawrence Berkeley National Laboratory (LBNL) Michael Wehner,Lawrence Livermore National Laboratory (LLNL) Phillip Cameron-Smith, Arthur Mirin; Los Alamos National Laboratory (LANL) Scott Elliot, Philip Jones, William Lipscomb, Mat Maltrud; National Center for Atmospheric Research (NCAR) Peter Gent, William Collins, Tony Craig, Jean-Francois Lamarque, Mariana Vertenstein, Warren Washington; Oak Ridge National Laboratory (ORNL) Marcia Branstetter, John Drake, David Erickson, Forrest Hoffman,W. M. Post, Trey White, Patrick Worley; Pacific Northwest National Laboratory (PNNL) Steven Ghan, Xiaohong Liu, Richard Easter, Rahul Zaveri, Sandia National Laboratories (SNL) Mark Taylor Current Activities (cont.) Contact Info: drakejb@ornl.gov, pwjones@lanl.govdrakejb@ornl.govpwjones@lanl.gov Project Web Page: www.scidac.gov/climate/earth.html Scientific Challenge To determine the range of possible climate changes over the 21 st century and beyond through simulations using a more accurate climate system model that includes the full range of human and natural climate feedbacks with increased realism and spatial resolution Objective The primary objective of this proposal is to develop, test, and exploit a first generation of Earth system models based upon the CCSM. We will bring to the community a new well- validated version of CCSM that will run efficiently on thousands of processors and include significant model enhancements. Improvements to the representation of carbon and chemical processes for treatment of greenhouse gas emissions and aerosol feedbacks will be performed in collaboration with the DOE Atmospheric Science Program, DOE Atmospheric Radiation Measurement Program, and DOE Terrestrial Carbon Program. This project will enable climate change simulations required for scheduled national and international climate change assessments to which the CCSM is committed as part of the National Climate Change Science Program (CCSP) strategy. Goals and Deliverables Over the next five years, we will pursue this objective through four integrated goals: 1.Extend the capabilities of the Community Climate System Model (CCSM) to include representations of biological, ecological, chemical, and aerosol processes that will allow scientists and policy-makers to simulate climate and climate change using a comprehensive Earth system model, 2.Provide the necessary software and modeling expertise to rapidly integrate new methods and model improvements, 3.Pursue the development and evaluation of innovative methods in the coupled context of the CCSM, and 4.Improve the performance, portability and scalability of the CCSM on available and future computing architectures for use in national and international assessments of climate change. Biogeochemical Coupling: Scalability and Performance: Model Integration and Evaluation: Collaborations Scientific Applications (SAs) and Partnerships(SAPs) Brookhaven National Laboratory Robert McGraw Oak Ridge National Laboratory Patrick Worley Argonne National Laboratory Kotamarthi Rao Centers for Enabling Technology Collaborations Earth System Grid (ESG) - Dean Williams Performance Engineering (PER) – Bob Lucas VACET – Wes Bethel ; SDM - Ari Shoshani TOPS – David Keyes; ITAPS - Lori Diachin APDEC - Phil Colella Aerosols and Atmospheric Chemistry Tasks and Organization Earth System Model  Terrestrial carbon cycle and dynamic vegetation  Atmospheric chemistry and aerosol dynamics  Ocean ecosystems and biogeochemical coupling  Feedbacks between atmospheric composition and biogenic emissions Model and Evaluation Model Integration and Evaluation  Integration and unit testing  New icesheet and ocean models  New atmospheric dynamics: finite volume (cubed sphere), discontinuous Galerkin, others(icosahedral)  Frameworks for model evaluation Computational Computational Performance  Scalablity to thousands of processors, load balance, (fault recovery) Computational Climate End Station for Climate Change Science - Warren Washington (PI) Software enabling integration: Model Coupling Toolkit (MCT 2.3.0) released (1/10/07) Optimized coupling strategies New methods for model evaluation (w/ Rao) Software enabling scalability: Collecting performance data to quantify and identify scalability limiters. Modifying code to improve scalability: Enabling different degrees of parallelism in physics and dynamics of atmosphere model. Identifying and eliminating nonscalable memory allocations. Working with Pnetcdf developers and CCSM software engineers to implement a single parallel I/O interface across all CCSM components. Validating software solutions on large Cray XT3 and IBM BlueGene systems. Carbon Land Model Intercomparison (C-LAMP): Determine the best set of terrestrial biogeochemistry process representations to include a future release of CCSM Three models coupled to CCSM3: CLM3-CASA', CLM3- CN, and LSX-IBIS Experiments are being performed with changing climate, CO2, nitrogen, and land use Comparison with high frequency, high quality flux tower measurements Atmospheric CO2 resulting from ecosystem respiration from the CLM3-CASA’ Ice sheet model: Largest missing piece of physical climate model and necessary for sea level rise prediction Variable resolution for melting at margins of ice sheet Ocean vertical grids: Arbitrary Lagrangian-Eulerian vertical grid for deep ocean circulation with good resolution in the surface mixed layer Ocean horizontal grids: Eddy-resolving simulations required for adequate simulation of ocean circulation Unstructured grids provide ability to focus grids in eddy- active regions at reduced cost High Resolution Ocean and Cryosphere: Aerosols alter cloud properties and the distribution of radiation in the atmosphere Development of a faster chemistry package for the troposphere and lower stratosphere Introduction of modal and moment representations of aerosol dynamics modeling aerosol size distribution Modeling the surface emissions of pollutants changes atmospheric radiation Current Activities Sea surface temperatures simulated on 0.1 o displaced pole grid using the POP2 code CVT Delaunay tessellation of the North Atlantic with 85K triangles - Gunzburger and Ju

3. AMWG Related Activities n Atm chemistry and aerosol dynamics (J.-F. Lamarque) –Aerosol Indirect effect - Steve Ghan, Bill Collins –Moment methods - Bob McGraw –Atm chem fast mechanisms - Philip Cameron-Smith – Sulfur cycle - Dave Erickson n Model Integration and Evaluation (M. Vertenstein) –Integration and unit testing (Rob Jacobs, Tony Craig) –FV (cubed sphere), DG, others (Mark Taylor, Art Mirin, Pat Worley, J. Drake) –Evaluation Frameworks (K. Rao et al.) n Computational Performance (Pat Worley, Art Mirin, Ray Loy) –Scalability, load balance, (fault recovery)

4. 20042005200620072008 2009 100 TF 1000 TF 250 TF 18.5 TF Cray TBD 50 TF Supercomputers at NCCS Leadership Computing Facility (also BG/L at Argonne) Leadership Computing Facility (also BG/L at Argonne) 18 TF Cray X1 / X1E Will not expand 1024 processors Vector processing for sustained performance 50 TF Cray XT3 5212 dual core Opterons 21 Terabytes memory 100TB scratch 25 TF 60 TF Cray XT4 6296 dual core Opterons New interconnect Acceptance testing

5. Computer Allocation: Computational End Station for Climate Change Science –Extensible community models available for computational science –Coordination of effort among agencies and institutions (talk to Lawrence Buja, J. Drake or one of the partner representatives) PI: Warren Washington (NCAR), partners: CCSM, COSIM, PCMDI, SciDAC, NASA-GSFC, PNNL, CCRI(Universities)

6. What does DOE want? n Relation to Aerosol Science Program and Terrestrial Carbon Program n Coordinated enterprise, relation to CETs and SAPs n Reporting –Impacts and revised scope statement –30 days: 4-6 slides for Dr. Orbach –60 days: Management Plan, website, performance baseline –6months: progress reports, highlights DOE OBER Performance Targets 2006 Deliver new measurements of clouds where observations are missing 2007 Include realistic cloud simulations in a climate model 2008 Measure ecosystem responses to climate change 2010 Develop/validate new models predicting effect of aerosols on climate forcing 2010 Provide climate model that links the Earth climate system with Earth’s biological systems 2013 Reduce differences between observed temperature & model simulations at sub continental scales using several decades of recent data 2015 Deliver improved climate data & models for policy makers to determine safe levels of greenhouse gasses.