1. Lessons learned from building and managing the Community Climate System Model David Bailey PCWG liaison (NCAR) Marika Holland PCWG co-chair (NCAR) Elizabeth Hunke PCWG co-chair (LANL) David Lawrence LMWG co-chair (NCAR) Steve Vavrus University collaborator (U. Wisconsin)

2. A brief history 1993 – Initial meetings of NCAR scientists 1994 – Development of the first coupled model 1996 - First successful coupled simulation (little drift); first Breckenridge workshop; Working groups formed; first release to the research community 2000 – name changed to Community Climate System Model 2004 – CCSM3 released 2009 – CCSM4?

3. CCSM Management Scientific Steering Committee (SSC) Chair: Peter Gent Provide scientific leadership; oversight of activities of working groups, coordination of model experiments, decision making on model definition and development priorities An advisory committee consisting of university faculty, members of national laboratories CCSM Advisory Board (CAB) Working Groups: Development and Applications CCSM Sponsored NSF and DOE

4. Chemistry Climate Chemistry Climate BioGeo Chemistry BioGeo Chemistry Software Engineering Climate Variability Polar Climate Polar Climate Land Model Land Model PaleoClimate Ocean Model Ocean Model CCSM Working Groups DevelopmentDevelopment Application Atm Model Atm Model Climate Change CCSM is primarily sponsored by the National Science Foundation and the Department of Energy

5. Makeup of CCSM Working groups 2 or 3 co-chairs, at least one external Scientific liaison Software engineer (ideally one per WG) ~10 or more hands-on model developers >40 total participants, majority are external Participation is voluntary, long-term participation is common

6. Developing and maintaining community participation Meetings (webcast when possible) –Plenary (13 th annual CCSM workshop, June 17-19, Breckenridge) –Component –Frequent local CCSM scientist Encourages community involvement Setting priorities, coordination of activities, putting names to tasks

7. Lessons: Resources Local dedicated computing and software engineering resources Sufficient staff with base funding –Scientific model developers and users with considerable local in-house scientific expertise and scientific investment throughout project –Software engineers (6 people minimum component liaison, coupling, run scripts, testing) Supplemental grants for specific projects

8. Lessons: Management of model Well-documented, user-friendly, tested, and efficient model Policies and procedures for ownership and distribution of model and model data Process for integrating community model improvements Policies for community support / external user problem solving

9. Lessons: Model development Started small, built up from models existing at NCAR Tuning a coupled model is a slow process (6 months or more for CCSM3) Deadlines, such as release dates or IPCC, keep progress moving along Conflicts will happen (SSC, Working group co-chairs) Compromises may be required –Near-surface ocean eddy flux change killed Atlantic layer –Snow cover fraction –Sea ice/snow albedo tuning Applications community needs to be involved

10. Not all good news in CCSM3.5 - Arctic Ocean Profiles CCSM3.5 OBS CCSM3 Distinct Atlantic layer missing in CCSM3.5 Runs. Does not appear to be related to ice model changes Considerable cooling of waters at depth compared to CCSM3 Salinity profiles still look quite good.

11. Lessons: Model development Starting small, built up from models existing at NCAR Tuning a coupled model is a slow process (6 months or more for CCSM3) Importance of setting deadlines (release dates) Version control (e.g. Subversion) Conflicts will happen (SSC, Working group co-chairs) Compromises may be required –Near-surface ocean eddy flux change killed Atlantic layer –Snow cover fraction –Sea ice/snow albedo tuning Applications community needs to be involved

12. Lessons: Coupling is hard High quality component models does not ensure high quality coupled simulations Tuning the coupled system is not easy Difficult to define and agree on metrics Examples of unforeseen problems –CLM3.5 crashed ocean model in Arctic (runoff spikes) –Smagorinsky parameter caused sea ice bias only in FV RunoffRunoff clim

13. Lessons: Coupling is hard Case of the wayward oceanic Smagorinsky parameter Spectral core FV core

14. Lessons: Software development Version control (e.g. Subversion) –Software gatekeeper for each component –Permits new model development to keep up and not conflict with accepted model code Regression testing Flexible build/run system User-friendly –Scripts –Code –Diagnostic packages –Validation packages

16. Summary Lessons learned from CCSM experience –Building and maintaining a vibrant model- development and user community is critical –Requires availability of sufficient resources from computing to software engineering –Coupling is hard and compromises may sometimes be required –Deadlines help, driving force –User-friendly is key

17. Timeline of Climate Model Development