1. Thomas Phillips and Celine Bonfils Program for Climate Model Diagnosis and Intercomparison (PCMDI) Lawrence Livermore National Laboratory Evaluating CMIP Simulations of Historical Continental Climate Using Koeppen Bioclimatic Metrics ____________________________________________________________________________________________________________________________________________________________________ Background LLNL-POST- 605935 Koeppen Vegetation Maps: CMIP Simulations vs Observationally Based (OBS) Estimates Summary Koeppen vegetation types 1 (Polar Desert) and 3 ( Evergreen Forest) are simulated most successfully, and vegetation types 11 (Tropical Moist Evergreen Forest) and 13 (Semiarid Vegetation) least successfully, by the CMIP models. Regions where the CMIP simulations are generally most problematic include: Amazonia, northwestern China, southwestern North America, southern Africa, and central Australia. Arid/semiarid zones are especially difficult regions for the typical CMIP model to correctly simulate T & P. Metrics VH and VA are stringent measures of CMIP model performance in simulating the correct amplitude and phase of T & P annual cycles. Even the best-performing models are able to display only ~ 70% correct one-to-one matches with the OBS vegetation types, and only ~ 90 % matches in their areas. The CMIP5 models generally perform incrementally better than their CMIP3 antecedents (especially in temperate climatic zones). A few CMIP5 models (e.g. BCC-CSM1-1, CNRM-CM5, CCSM4, GFDL-CM3, IPSL-CM5A, MIROC4h, etc.) show marked performance improvements relative to their CMIP3 antecedents. The selected examples of CMIP5 Earth Systems Models (ESMs) that include a prognostic carbon cycle responding to prescribed greenhouse-gas emissions display similar levels of performance as those of the corresponding physical-climate GCM(s) (from the same modeling center) with prescribed GHG concentrations. Summary Koeppen vegetation types 1 (Polar Desert) and 3 ( Evergreen Forest) are simulated most successfully, and vegetation types 11 (Tropical Moist Evergreen Forest) and 13 (Semiarid Vegetation) least successfully, by the CMIP models. Regions where the CMIP simulations are generally most problematic include: Amazonia, northwestern China, southwestern North America, southern Africa, and central Australia. Arid/semiarid zones are especially difficult regions for the typical CMIP model to correctly simulate T & P. Metrics VH and VA are stringent measures of CMIP model performance in simulating the correct amplitude and phase of T & P annual cycles. Even the best-performing models are able to display only ~ 70% correct one-to-one matches with the OBS vegetation types, and only ~ 90 % matches in their areas. The CMIP5 models generally perform incrementally better than their CMIP3 antecedents (especially in temperate climatic zones). A few CMIP5 models (e.g. BCC-CSM1-1, CNRM-CM5, CCSM4, GFDL-CM3, IPSL-CM5A, MIROC4h, etc.) show marked performance improvements relative to their CMIP3 antecedents. The selected examples of CMIP5 Earth Systems Models (ESMs) that include a prognostic carbon cycle responding to prescribed greenhouse-gas emissions display similar levels of performance as those of the corresponding physical-climate GCM(s) (from the same modeling center) with prescribed GHG concentrations. Continental temperature and precipitation (hereafter, T & P) are of crucial importance for biological organisms, and so it is essential that climate models accurately simulate these key variables. Particular thresholds of T & P are also central to determining where particular life forms are found on Earth. For example, the classic bioclimatic classification scheme devised by Wladimir Koeppen (1900 Geographische Zeitschrift) associates certain generic vegetation types (e.g. grassland, tundra, deciduous or evergreen forests, etc.) with specific thresholds of climatological monthly temperature and precipitation, and with their respective seasonalities (warm vs cool summers, dry vs wet winters, etc.). The Koeppen scheme (e.g. as adapted by Gnanadesikan and Stouffer, 2006 Geophysical Research Letters) distinguishes 5 climate zones according to surface-temperature thresholds: A, tropical (coolest month warmer than 18 C); B, arid (insufficient yearly rainfall to balance potential evaporation); C, temperate (coolest month between -3 C and 18 C); D, boreal forest and snow (warmest month > 10 C, but coldest month < -3 C); and E, cold snowy climates (warmest month < -3 C). Each climate zone is subdivided, in turn, into regional classes (e.g. designated by lower-case letters such as s, w, m, or f, depending on whether there is a summer dry season, a winter dry season, a monsoonal climate, or no clear-cut dry season). See the green table below for further details. OBS VegType1 Veg Type 14 The Koeppen bioclimatic classification thus supplies both a concise and stringent modality for evaluating and diagnosing model performance in simulating T & P in naturally defined regions. Koeppen bioclimatic classification also can provide a first-order reality test for the more complex Earth System Models (ESMs) that incorporate a prognostic carbon cycle and dynamical representations of vegetation. That is, even though the ESM vegetation types may differ from the generic Koeppen varieties, ESM dynamical vegetation schemes still will depend on accurate simulation of regional T & P. Thus, a Koeppen-based evaluation of the ESMs can readily uncover problematical aspects of the simulated historical climate that will impact accurate simulation of dynamical vegetation. Finally, Koeppen-based evaluation offers a convenient means for comparing CMIP5 model performance in simulating T & P with that of antecedent CMIP3 models. Numerous examples of such CMIP3  CMIP5 model performance changes are displayed in the orange-shaded table to the right. The OBS and CMIP simulations of Koeppen vegetation types are derived from 1980-1999 monthly annual-cycle climatologies of continental T & P that are interpolated to a common 2.5 x 2.5 degrees latitude-longitude grid. Note, the Koeppen classification results for each CMIP model are obtained from only a single simulated realization of T & P. The surface T & P observationally based datasets used to estimate the “observed” (OBS) Koeppen vegetation evaluation standard are taken from the National Center for Environmental Prediction/ National Center for Atmospheric Research (NCEP/NCAR) continental surface air temperature reanalysis, and from the Global Precipitation Climatology Project (GPCP) gauge-observed continental precipitation data. Modeling Centers (Country ) CMIP3 Models Metric VH (max = 1400) Metric VA (max = 100) CMIP5 Models Metric VH (max = 100) Metric VA (max = 100) Beijing Climate Center (China)BCC-CM12718BCC-CSM1-157, 58 * 90, 91* Canadian Centre for Climate Modelling &Analysis (Canada) CCCCMA-CGCM3-15877 CanCM46080 CCMA-CGCM3-1-T635883 CanESM2 5983 National Center for Atmospheric Research (USA)CCSM3-05973 CCSM46784 Meteo France Centre National de Recherches Meteorologiques (France)CNRM-CM35571 CNRM-CM56084 Commonwealth Scientific and Industrial Research Organization/ Queensland Climate Change Centre of Excellence (Australia) CSIRO-Mk-3-05781CSIRO-Mk-3-6-06276 Geophysical Fluid Dynamics Laboratory (USA) GFDL-CM2-05375 GFDL-CM36280 GFDL-ESM2G58. 57 * 75, 74* GFDL-CM2-15177GFDL-ESM2M5780 NASA Goddard Institute for Space Studies (USA) GISS-E-H4885 GISS-E2-H5577 GISS-E-R5076 GISS-E2-R 5780 Met Office Hadley Centre (UK) HadCM3 6288 HadCM36389 HadGEM2-CC6585 HadGEM1 6491 HadGEM2-ES6885 Institute for Numerical Mathematics (Russia)INM-CM3-05277 INM-CM456, 56 * 66, 66* Institut Pierre-Simon Laplace (France)IPSL-CM45072 IPSL-CM5A-LR 5774 IPSL-CM5A-MR5981 Center for Climate System Research (Univ. Tokyo), National Institute for Environmental Studies, and Japan Agency for Marine-Earth Science and Technology (Japan) MIROC3.2 MEDRES6186 MIROC4h6788 MIROC5 59 84 MIROC-ESM 59 83 MIROC-ESM-CHEM 60 83 Max Planck Institute for Meteorology (Germany)MPI-ECHAM56383MPI-ESM1-LR6383 Meteorological Research Institute (Japan)MRI-CGCM2-3-2a6382 MRI-CGCM36081 Norwegian Climate Centre (Norway)BCCR-BCM2.05574NorESM1-M6279 * ESM historical simulation with prescribed greenhouse-gas emissions Koeppen Zones/Classes/Vegetation Types * Climatic Criteria * 1 Ef: Polar Desert VegetationT max < 0 deg C 2 Et: Tundra Vegetation0 C < T max < 10 C and T min < -3 C 3Dc: Evergreen Forest (cold winters, cool summers) T min 4 months warmer than 10 C, but not zones/classes BS or BW (see Koeppen zones/classes/vegetation types !3 and 14 below) 4Dab: Deciduous Forest (cold winters, warm summers) T min 10 C and > 4 months warmer than 10 C, but not zones/classes/vegetation types BS or BW 5Cw: Evergreen Forest (temperate, wet summers) -3 C 10P min with P max occurring in summer and P min in winter, but not zones/classes/vegetation types BS or BW 6 Cs: Evergreen Broad-Leaf (temperate, wet winters) 18 C > T min > -3 C and P max > 3P min, with P max occurring in winter and P min in summer, but not zones/classes/vegetation types BS or BW 7Cfc: Needle-Tree Forest (cool, moist temperate) -3 C < T min < 18 and T max < 22C, and with < 4 months warmer than 10 C, but not zones/classes/vegetation types BS, BW, Cs, or Cw 8Cfb: Broad-Leaf Forest (warm, moist temperate) -3 C 4 months warmer than 10C, but not zones/classes/vegetation types BS, BW, Cs, or Cw 9Cfa: Broad-Leaf Forest (hot, moist temperate) -3C 22C, but not zones/classes/vegetation types BS, BW, Cs, or Cw 10 Af: Tropical Wet Evergreen Forest T min > 18C and P min > 6 cm, but not zones/classes/vegetation types BS or BW 11 Am: Tropical Moist Evergreen Forest T min > 18C and (250 cm – P year )/25 < P min < 6cm, but not zones/classes/vegetation types BS or BW 12 Aw: Tropical Dry Vegetation (savanna/woodland) T min > 18C and P min < (250 cm -P year )/25, but not zones/classes/vegetation types BS or BW 13BS: Semiarid Vegetation (bush, grassland) (T avg + P off ) < P year < 2(T avg + P off ) 14BW: Desert Vegetation (wasteland, cactus) P year < (T avg + P off ) * Here the regional climatic zones/vegetation types are defined after Gnanadesikan and Stouffer (2006 GRL), using the following notation: T min,max,avg are the minimum monthly, maximum monthly, and annual-average continental temperature T in degrees Celsius. P min,max,year are the minimum monthly, maximum monthly, and annually integrated continental precipitation amount P in centimeters. In addition, precipitation seasonality parameter P off = 0 if > 30% of P year falls in winter; P off = 7 if there is no distinctly wet season; and P off = 14 if > 30% of P year falls in summer. CMIP3 vs CMIP5 Historical (1980-1999) Climate Simulations: Global Performance Metrics by Model Model CCSM4 Vegetation Map Model Performance Metrics To quantify the degree of agreement between the simulated Koeppen vegetation types and those derived from the OBS, we define globally aggregated performance metrics VH and VA. VH measures percentage of a model simulation’s correct one-to-one “hits” of each OBS vegetation type, averaged over all 14 types (with a maximum possible value of 100 %). The hits by vegetation type can be expressed graphically in an x-y plot, where the 14 OBS vegetation types are arrayed along the x-axis and the modeled vegetation types along the y-axis. The hits then occur along the diagonal y = x, and the percentage of hits by vegetation type is indicated by a color scheme where higher aggregated percentages are expressed by “warmer” colors, and lower percentages by “cooler” colors. The model “misses” then are indicated graphically by grey off-diagonal swaths (see example below). Metric VA measures the aggregate deviations in geographical areas of the modeled vegetation types from those of the OBS: VA = 100 -  i = 1,14 |  v i |, where |  v i | is the absolute deviation in percentage area of modeled vegetation type i from that of the same OBS type. In the Model CCSM4 example (rightmost panel, immediately below) these deviations are equivalent to the gaps between the ___ MODEL and ___ OBS lines for each vegetation type. VA = 84 An Example: The Model CCSM4 Historical Climate Simulation  The map of 14 vegetation types derived from the CCSM4 historical climate simulation (near-right figure) illustrates the similarities and differences with respect to the OBS vegetation map above. CMIP3 vs CMIP5 Collective Performance  Here, the collective percentage “hits” by vegetation type for the CMIP3 versus CMIP5 models is shown. Overall, the CMIP5 models display an incrementally higher level of performance in simulating T & P than do the CMIP3 models, especially for temperate vegetation types 4-9. Furthermore, some individual CMIP5 models perform substantially better than their CMIP3 antecedents (see the tabulation of VH and VA by individual model below). Average CMIP3 vs CMIP5 % Hits Vegetation Types CMIP5 ___ CMIP3 ___ An Example of Regional Error Deconstruction: CCSM4’s Problematical Simulation of T & P near Mexico 1 14 CCSM4 Veg Hits & Misses 0% 100% OBS Vegetation Types Model Vegetation Types 14 VH = 67 CCSM4 vs OBS Veg Areas Vegetation Types % of Total Land Area VA = 84 The plot of CCSM4 vegetation “hits and misses” relative to that of the OBS and the aggregate VH metric are shown below-left. The comparative areas occupied by each vegetation type, along with the aggregate VA metric, are shown below-right. CCSM4’s Koeppen vegetation map in central Mexico is notably different from the observed semi-arid.vegetation types (compare maps displayed above). CCSM4 deviations  from observed variables T avg and P year (see maps immediately to the right) that define Koeppen semi-arid zone BS (cf. left Table), reveal the model errors responsible for generating anomalous regional vegetation types: In CCSM4, T avg is biased too low, and P year is biased too high relative to the OBS (in the CCSM4 deviation maps displayed to the right, “cool” colors denote negative biases, and “warm” colors denote positive biases). OBS VegetationCCSM4 Vegetation  T avg  P year  = [CCSM4 – OBS]