WP9/JRA3: ESM Evaluation: developing an Infrastructure Participants: AA, CNRS-IPSL, DLR, FMI, SMHI, MPG OBJECTIVES Create an interdisciplinary infrastructure that will facilitate the scientific evaluation of complex ESMs Standardize, harmonize and simplify the tools and methodologies used in existing or past model intercomparison projects to evaluate the basic quality of an ESM model. Create a public archive of easy-to-use toolkits that will include software to prepare required model diagnostics as well as links to pre-assembled most recent state-of- the-art observational datasets. Add to the IS-ENES portal an evaluation component that can be used as a ‘one- stop-shop’ for modellers who want to evaluate the performance of their ESM.

TASKS Task 1. Survey of methods and problems in model intercomparison projects Identify synergies among the different established model intercomparisons in the field (CFMIP, AEROCOM, PMIP, CCMVal, C4MIP…). Investigate the common problems involved in documenting model quality and model progress. Develop a joint implementation plan to improve the functioning of these intercomparisons. Organize a workshop to examine the implementation plan in collaboration with international groups such as PCMDI, NASA, and NCAR. Task 2. Creation of support structure to facilitate model evaluation tests Provide support structures to ESM modelling teams to implement the appropriate model diagnostics and formatting tools (including CMOR, instrument simulators, variable meaning and unit standards), so that standard evaluation tests can be easily applied. Develop interfaces to bridge the format gaps between model and observational output. Task 3. Provision of cataloguing and access services to observational datasets and evaluation tools Provide catalogue and access services for the evaluation toolkits together with the corresponding observational datasets. Define standards for documentation of a toolkit. Implement these documentation standards for a set of previously used evaluation procedures in the above mentioned intercomparisons. Task 4. Pilot study on the implementation of evaluation toolkits on ESM output Perform a pilot study of the application of the evaluation toolkits to one ESM.

FIRST ACTIONS T.1 Survey of methods and problems in model intercomparison projects AA: Clouds, radiation and precipitation (e.g. CFMIP) CNRS-IPSL: Aerosols (e.g. AEROCOM), paleoclimate (e.g. PMIP), and carbon- climate couplings (e.g. C4MIP) SMHI: Ocean property and circulation projects (e.g. OMIP, AOMIP) MPG(MPIMET): Surface fluxes (e.g. SEAFLUX) DLR: Atmospheric chemistry (e.g. CCMVal) Larger interdisciplinary model evaluation projects such as CMIP and CCMVal will be surveyed in collaboration with international groups like PCMDI and NCAR. M1. Workshop on synergies and differences between model intercomparison projects. D1. Report on differences, synergies and potential methodology transfers between the surveyed model intercomparison projects. Joint implementation plan to improve the functioning of ESM intercomparisons.