Anthony DeAngelis [ [

Outline  Introduction  Data, Models, and Methodology  Results  Spatial Comparisons over United States  Analysis of Resolution  Ranking of Model Performance  Conclusions  Future Directions

Importance of Precipitation  Agriculture, water resources, power, etc.  Extreme Precipitation  Flooding takes 140 lives in the United States each year (USGS 2006 ).  Observational evidence of increases in the frequency and intensity of extreme precipitation throughout the world over 20 th century (e.g., Groisman et al )  Model projections of future increases in heavy precipitation in response to increasing greenhouse gases (e.g., Pall et al )  Quantification of future changes in precipitation relies on model simulations

How well do models simulate precipitation?  IPCC AR 4 –Fairly realistic mean precipitation by ensemble of PCMDI coupled models

Mean Precipitation [IPCC AR 4, Ch 8, Fig. 8.5] CMAP Observations  Multi-model mean of AOGCMs 

How well do models simulate precipitation?  IPCC AR 4 –Fairly realistic mean precipitation by ensemble of PCMDI coupled models  Sun et al –Overestimation of light precipitation and underestimation of heavy precipitation by CMIP 3 models

observations Sun et al. ( 2007) Figure 1 model average

How well do models simulate precipitation?  IPCC AR 4 –Fairly realistic mean precipitation by ensemble of PCMDI coupled models  Sun et al. ( 2007) –Overestimation of light precipitation and underestimation of heavy precipitation by CMIP 3 models  Kiktev et al – HadAM 3 has little skill in simulating precipitation trends over  Higher resolution models perform better  Iorio et al – NCAR CCM 3 – mean and extreme precipitation  Kimoto et al – MIROC 3.0 – extreme precipitation  Models with embedded cloud resolving models or certain convective parameterizations perform better for extreme precipitation (Iorio et al. 2004, Emori et al )

What did I do?  Compared 20 th century simulations from CMIP 3 models with observations over the contiguous United States  Looked at differences in spatial pattern of precipitation characteristics for individual models  Used a longer and consistent time period for comparison ( ) than previous studies  Compared two gridded observational datasets  Assessed the role of resolution on model performance for all models collectively

Observational Data and Climate Models  Observations  Climate Prediction Center’s Daily United States Unified Precipitation (CPC) ° x 0.25 ° lon-lat ( ) [Higgins et al. 2007]  David Robinson’s daily gridded precipitation (DAVR) ° x 1.0 ° lon-lat ( ) [Dyer and Mote 2006]  Climate Models  20 th century simulations – forced with observed atmospheric composition  18 CMIP 3 models with daily precipitation from and a standard (non 360 day) calendar  One ensemble member for each model  Meehl et al. ( 2007 )

Model #Modeling GroupCountryModel IDSpatial Resolution (approximate - lon x lat) 1Bjerknes Centre for Climate ResearchNorwayBCCR BCM ° x 2.81° 2Canadian Centre for Climate Modelling & AnalysisCanadaCCCMA CGCM 3.1 T473.75° x 3.75° 3Canadian Centre for Climate Modelling & AnalysisCanadaCCCMA CGCM 3.1 T632.81° x 2.81° 4Centre National de Recherches MétéorologiquesFranceCNRM CM 32.81° x 2.81° 5CSIRO Atmospheric ResearchAustraliaCSIRO MK ° x 1.88° 6CSIRO Atmospheric ResearchAustraliaCSIRO MK ° x 1.88° 7Geophysical Fluid Dynamics LaboratoryUSAGFDL CM ° x 2.00° 8Geophysical Fluid Dynamics LaboratoryUSAGFDL CM ° x 2.00° 9Goddard Institute for Space StudiesUSAGISS AOM4.00° x 3.00° 10Goddard Institute for Space StudiesUSAGISS E H5.00° x 3.91° 11Goddard Institute for Space StudiesUSAGISS E R5.00° x 3.91° 12Institute of Atmospheric PhysicsChinaIAP FGOALS 1.0 G2.81° x 3.00° 13Institute for Numerical MathematicsRussiaINM CM ° x 4.00° 14Center for Climate System Research, National Institute for Environmental Studies, and Frontier Research Center for Global Change JapanMIROC 3.2 MEDRES2.81° x 2.81° 15Max Planck Institute for MeteorologyGermanyMPI ECHAM 51.88° x 1.88° 16Meteorological Research InstituteJapanMRI CGCM ° x 2.81° 17National Center for Atmospheric ResearchUSANCAR CCSM ° x 1.41° 18National Center for Atmospheric ResearchUSANCAR PCM 12.81° x 2.81° CMIP 3 Models Used More information:

Spatial Comparisons  Linear re-gridding to 2.5 ° x 2.5 ° lon-lat  Typical model resolution that is fine enough to resolve the coastlines  Precipitation Quantities for  Mean  Frequency of wet days (precip. ≥ mm/day)  Standard deviation for wet days divided by mean for wet days – precipitation variability  99 th percentile for all days  Generalized extreme value normalized scale parameter for yearly maximum daily precipitation distribution – extreme precipitation variability

Mean Precipitation (mm/day) Convective parameterizations? [Iorio et al ] Improper terrain representation? Agreement with IPCC AR 4

Mean Precipitation [IPCC AR 4, Ch 8, Fig. 8.5] CMAP Observations  Multi-model mean of AOGCMs 

Mean for Wet Days (mm/day)

Frequency of Wet Days (days/year)

Normalized Standard Deviation for Wet Days (dimensionless ) Could be related to too many wet days

99 th Percentile for All Days (mm/day) Convective parameterizations again?

Seasonal 99 th Percentile  Model Average – Observations Average (%) Convection season- consistent with idea of bad convective representation

Example Generalized Extreme Value (GEV) Distribution Representative of New Jersey in Observations location parameter- center of distribution ( 45 mm/day) scale parameter- spread of distribution ( 9 mm/day) I plot scale/location ( 0.2 in this case)

GEV Normalized Scale Parameter for Yearly Maximum (dimensionless) Not enough variability of precipitation extremes

GEV Normalized Scale Parameter for Yearly Maximum (dimensionless)

Does Spatial Resolution Make a Difference?  Linear re-gridding to 5.0 ° x 4.0 ° lon-lat  Error- root mean square of absolute difference between each model and observations average (Iorio et al )  Plot error against finite grid equivalent resolution (# of global grid cells)  Fit least squares linear regression to error vs. resolution plot

Error vs. Resolution Results  Statistically significant improvement in the frequency of wet days with higher resolution

= model average

Model #Modeling GroupCountryModel IDSpatial Resolution (approximate - lon x lat) 1Bjerknes Centre for Climate ResearchNorwayBCCR BCM ° x 2.81° 2Canadian Centre for Climate Modelling & AnalysisCanadaCCCMA CGCM 3.1 T473.75° x 3.75° 3Canadian Centre for Climate Modelling & AnalysisCanadaCCCMA CGCM 3.1 T632.81° x 2.81° 4Centre National de Recherches MétéorologiquesFranceCNRM CM 32.81° x 2.81° 5CSIRO Atmospheric ResearchAustraliaCSIRO MK ° x 1.88° 6CSIRO Atmospheric ResearchAustraliaCSIRO MK ° x 1.88° 7Geophysical Fluid Dynamics LaboratoryUSAGFDL CM ° x 2.00° 8Geophysical Fluid Dynamics LaboratoryUSAGFDL CM ° x 2.00° 9Goddard Institute for Space StudiesUSAGISS AOM4.00° x 3.00° 10Goddard Institute for Space StudiesUSAGISS E H5.00° x 3.91° 11Goddard Institute for Space StudiesUSAGISS E R5.00° x 3.91° 12Institute of Atmospheric PhysicsChinaIAP FGOALS 1.0 G2.81° x 3.00° 13Institute for Numerical MathematicsRussiaINM CM ° x 4.00° 14Center for Climate System Research, National Institute for Environmental Studies, and Frontier Research Center for Global Change JapanMIROC 3.2 MEDRES2.81° x 2.81° 15Max Planck Institute for MeteorologyGermanyMPI ECHAM 51.88° x 1.88° 16Meteorological Research InstituteJapanMRI CGCM ° x 2.81° 17National Center for Atmospheric ResearchUSANCAR CCSM ° x 1.41° 18National Center for Atmospheric ResearchUSANCAR PCM 12.81° x 2.81° CMIP 3 Models Used More information:

Error vs. Resolution Results  Statistically significant improvement in the frequency of wet days with higher resolution  All other quantities showed decreasing error with higher resolution, but the linear fit was not statistically significant  All quantities showed low percentage of model error variability explained by the linear fit (r 2 )

= model average

Other potential reasons for variability in model error  Different vertical resolutions  Different grid types (e.g., spectral resolution vs. finite grid)  Different cloud and convective parameterizations  Different microphysics schemes  Different ocean components  Different radiation schemes

Ranking of Model Performance  Ratio: Root mean square error for each model divided by the average root mean square error for all models for each precipitation quantity  Eliminates biases from quantities with different units (e.g., mean precipitation, frequency of wet days)  Take average of ratio over precipitation quantities for each model and rank them

CMIP 3 Model Ranking All Precipitation QuantitiesMean and Frequency of Wet Days99 th Percentile and GEV Normalized Scale Model RankAverage ErrorModel RankAverage ErrorModel RankAverage Error 1. MPI ECHAM MPI ECHAM GFDL CM MRI CGCM NCAR CCSM GFDL CM NCAR CCSM MRI CGCM MPI ECHAM CSIRO MK CSIRO MK CCCMA CGCM 3.1 T GFDL CM BCCR BCM Model Average Model Average Model Average CNRM CM CCCMA CGCM 3.1 T CCCMA CGCM 3.1 T CCCMA CGCM 3.1 T BCCR BCM GISS E R MIROC 3.2 MEDRES GFDL CM GFDL CM MRI CGCM CCCMA CGCM 3.1 T CSIRO MK CSIRO MK MIROC 3.2 MEDRES INM CM INM CM INM CM MIROC 3.2 MEDRES NCAR CCSM CSIRO MK CCCMA CGCM 3.1 T CSIRO MK CNRM CM GFDL CM NCAR PCM NCAR PCM GISS E H BCCR BCM IAP FGOALS 1.0 G NCAR PCM IAP FGOALS 1.0 G GISS E H CNRM CM GISS AOM GISS AOM IAP FGOALS 1.0 G GISS E H GISS E R GISS AOM GISS E R More information:

CMIP 3 Model Ranking All Precipitation QuantitiesMean and Frequency of Wet Days99 th Percentile and GEV Normalized Scale Model RankAverage ErrorModel RankAverage ErrorModel RankAverage Error 1. MPI ECHAM MPI ECHAM GFDL CM MRI CGCM NCAR CCSM GFDL CM NCAR CCSM MRI CGCM MPI ECHAM CSIRO MK CSIRO MK CCCMA CGCM 3.1 T GFDL CM BCCR BCM Model Average Model Average Model Average CNRM CM CCCMA CGCM 3.1 T CCCMA CGCM 3.1 T CCCMA CGCM 3.1 T BCCR BCM GISS E R MIROC 3.2 MEDRES GFDL CM GFDL CM MRI CGCM CCCMA CGCM 3.1 T CSIRO MK CSIRO MK MIROC 3.2 MEDRES INM CM INM CM INM CM MIROC 3.2 MEDRES NCAR CCSM CSIRO MK CCCMA CGCM 3.1 T CSIRO MK CNRM CM GFDL CM NCAR PCM NCAR PCM GISS E H BCCR BCM IAP FGOALS 1.0 G NCAR PCM IAP FGOALS 1.0 G GISS E H CNRM CM GISS AOM GISS AOM IAP FGOALS 1.0 G GISS E H GISS E R GISS AOM GISS E R More information:

CMIP 3 Model Ranking All Precipitation QuantitiesMean and Frequency of Wet Days99 th Percentile and GEV Normalized Scale Model RankAverage ErrorModel RankAverage ErrorModel RankAverage Error 1. MPI ECHAM MPI ECHAM GFDL CM MRI CGCM NCAR CCSM GFDL CM NCAR CCSM MRI CGCM MPI ECHAM CSIRO MK CSIRO MK CCCMA CGCM 3.1 T GFDL CM BCCR BCM Model Average Model Average Model Average CNRM CM CCCMA CGCM 3.1 T CCCMA CGCM 3.1 T CCCMA CGCM 3.1 T BCCR BCM GISS E R MIROC 3.2 MEDRES GFDL CM GFDL CM MRI CGCM CCCMA CGCM 3.1 T CSIRO MK CSIRO MK MIROC 3.2 MEDRES INM CM INM CM INM CM MIROC 3.2 MEDRES NCAR CCSM CSIRO MK CCCMA CGCM 3.1 T CSIRO MK CNRM CM GFDL CM NCAR PCM NCAR PCM GISS E H BCCR BCM IAP FGOALS 1.0 G NCAR PCM IAP FGOALS 1.0 G GISS E H CNRM CM GISS AOM GISS AOM IAP FGOALS 1.0 G GISS E H GISS E R GISS AOM GISS E R More information:

CMIP 3 Model Ranking All Precipitation QuantitiesMean and Frequency of Wet Days99 th Percentile and GEV Normalized Scale Model RankAverage ErrorModel RankAverage ErrorModel RankAverage Error 1. MPI ECHAM MPI ECHAM GFDL CM MRI CGCM NCAR CCSM GFDL CM NCAR CCSM MRI CGCM MPI ECHAM CSIRO MK CSIRO MK CCCMA CGCM 3.1 T GFDL CM BCCR BCM Model Average Model Average Model Average CNRM CM CCCMA CGCM 3.1 T CCCMA CGCM 3.1 T CCCMA CGCM 3.1 T BCCR BCM GISS E R MIROC 3.2 MEDRES GFDL CM GFDL CM MRI CGCM CCCMA CGCM 3.1 T CSIRO MK CSIRO MK MIROC 3.2 MEDRES INM CM INM CM INM CM MIROC 3.2 MEDRES NCAR CCSM CSIRO MK CCCMA CGCM 3.1 T CSIRO MK CNRM CM GFDL CM NCAR PCM NCAR PCM GISS E H BCCR BCM IAP FGOALS 1.0 G NCAR PCM IAP FGOALS 1.0 G GISS E H CNRM CM GISS AOM GISS AOM IAP FGOALS 1.0 G GISS E H GISS E R GISS AOM GISS E R More information:

CMIP 3 Model Ranking All Precipitation Quantities Mean, Frequency of Wet Days, and Normalized Standard Deviation for Wet Days 99 th Percentile and GEV Normalized Scale Model RankAverage ErrorModel RankAverage ErrorModel RankAverage Error 1. MPI ECHAM MPI ECHAM GFDL CM CSIRO MK CSIRO MK GFDL CM MRI CGCM MRI CGCM MPI ECHAM GFDL CM NCAR CCSM CCCMA CGCM 3.1 T NCAR CCSM Model Average Model Average Model Average MIROC 3.2 MEDRES CNRM CM CCCMA CGCM 3.1 T GFDL CM CCCMA CGCM 3.1 T MIROC 3.2 MEDRES CCCMA CGCM 3.1 T MIROC 3.2 MEDRES CCCMA CGCM 3.1 T CSIRO MK MRI CGCM GFDL CM CCCMA CGCM 3.1 T CSIRO MK INM CM INM CM INM CM CSIRO MK BCCR BCM NCAR CCSM BCCR BCM GFDL CM CSIRO MK CNRM CM GISS E R NCAR PCM NCAR PCM IAP FGOALS 1.0 G BCCR BCM IAP FGOALS 1.0 G NCAR PCM IAP FGOALS 1.0 G GISS E H CNRM CM GISS AOM GISS AOM GISS E H GISS E H GISS E R GISS AOM GISS E R More information:

CMIP 3 Model Ranking All Precipitation Quantities Mean, Frequency of Wet Days, and Normalized Standard Deviation for Wet Days 99 th Percentile and GEV Normalized Scale Model RankAverage ErrorModel RankAverage ErrorModel RankAverage Error 1. MPI ECHAM MPI ECHAM GFDL CM CSIRO MK CSIRO MK GFDL CM MRI CGCM MRI CGCM MPI ECHAM GFDL CM NCAR CCSM CCCMA CGCM 3.1 T NCAR CCSM Model Average Model Average Model Average MIROC 3.2 MEDRES CNRM CM CCCMA CGCM 3.1 T GFDL CM CCCMA CGCM 3.1 T MIROC 3.2 MEDRES CCCMA CGCM 3.1 T MIROC 3.2 MEDRES CCCMA CGCM 3.1 T CSIRO MK MRI CGCM GFDL CM CCCMA CGCM 3.1 T CSIRO MK INM CM INM CM INM CM CSIRO MK BCCR BCM NCAR CCSM BCCR BCM GFDL CM CSIRO MK CNRM CM GISS E R NCAR PCM NCAR PCM IAP FGOALS 1.0 G BCCR BCM IAP FGOALS 1.0 G NCAR PCM IAP FGOALS 1.0 G GISS E H CNRM CM GISS AOM GISS AOM GISS E H GISS E H GISS E R GISS AOM GISS E R More information:

CMIP 3 Model Ranking All Precipitation Quantities Mean, Frequency of Wet Days, and Normalized Standard Deviation for Wet Days 99 th Percentile and GEV Normalized Scale Model RankAverage ErrorModel RankAverage ErrorModel RankAverage Error 1. MPI ECHAM MPI ECHAM GFDL CM CSIRO MK CSIRO MK GFDL CM MRI CGCM MRI CGCM MPI ECHAM GFDL CM NCAR CCSM CCCMA CGCM 3.1 T NCAR CCSM Model Average Model Average Model Average MIROC 3.2 MEDRES CNRM CM CCCMA CGCM 3.1 T GFDL CM CCCMA CGCM 3.1 T MIROC 3.2 MEDRES CCCMA CGCM 3.1 T MIROC 3.2 MEDRES CCCMA CGCM 3.1 T CSIRO MK MRI CGCM GFDL CM CCCMA CGCM 3.1 T CSIRO MK INM CM INM CM INM CM CSIRO MK BCCR BCM NCAR CCSM BCCR BCM GFDL CM CSIRO MK CNRM CM GISS E R NCAR PCM NCAR PCM IAP FGOALS 1.0 G BCCR BCM IAP FGOALS 1.0 G NCAR PCM IAP FGOALS 1.0 G GISS E H CNRM CM GISS AOM GISS AOM GISS E H GISS E H GISS E R GISS AOM GISS E R More information:

CMIP 3 Model Ranking All Precipitation Quantities Mean, Frequency of Wet Days, and Normalized Standard Deviation for Wet Days 99 th Percentile and GEV Normalized Scale Model RankAverage ErrorModel RankAverage ErrorModel RankAverage Error 1. MPI ECHAM MPI ECHAM GFDL CM CSIRO MK CSIRO MK GFDL CM MRI CGCM MRI CGCM MPI ECHAM GFDL CM NCAR CCSM CCCMA CGCM 3.1 T NCAR CCSM Model Average Model Average Model Average MIROC 3.2 MEDRES CNRM CM CCCMA CGCM 3.1 T GFDL CM CCCMA CGCM 3.1 T MIROC 3.2 MEDRES CCCMA CGCM 3.1 T MIROC 3.2 MEDRES CCCMA CGCM 3.1 T CSIRO MK MRI CGCM GFDL CM CCCMA CGCM 3.1 T CSIRO MK INM CM INM CM INM CM CSIRO MK BCCR BCM NCAR CCSM BCCR BCM GFDL CM CSIRO MK CNRM CM GISS E R NCAR PCM NCAR PCM IAP FGOALS 1.0 G BCCR BCM IAP FGOALS 1.0 G NCAR PCM IAP FGOALS 1.0 G GISS E H CNRM CM GISS AOM GISS AOM GISS E H GISS E H GISS E R GISS AOM GISS E R More information:

Conclusions  CMIP 3 models underestimate mean and extreme precipitation amounts near the Gulf Coast  Convective parameterizations (Iorio et al )  CMIP 3 models produce precipitation days too frequently, especially in the north and west  Higher resolution models perform much better  CMIP 3 models have too little variability in all precipitation and extreme precipitation in the northern interior west  The MPI ECHAM 5 is the best, the model average is better than the majority of individual models, and the GISS models are the worst with 20 th century precipitation characteristics over the US

Future Directions  Understand the reasons for differences in model performance  What makes the MPI ECHAM 5 so good?  Evaluate the ability of CMIP 3 models to simulate precipitation changes  Time period used here is too short for a reliable analysis  Expand the evaluation of CMIP 3 precipitation to other regions

References  Dyer, J. L., and T. L. Mote, 2006: Spatial variability and patterns of snow depth over North America, Geophys. Res. Lett., 33, L16503, doi: /2006GL  Emori, S., A. Hasegawa, T. Suzuki, and K. Dairaku, 2005: Validation, parameterization dependence and future projection of daily precipitation simulated with an atmospheric GCM, Geophys. Res. Lett., 32, L06708, doi: /2004GL  Groisman, P. Y., R. W. Knight, D. R. Easterling, T. R. Karl, G. C. Hegerl, and V. N. Razuvaev, 2005: Trends in intense precipitation in the climate record, J. Clim., 18,  Higgins, R. W., W. Shi, E. Yarosh, and R. Joyce, 2000: Improved United States precipitationquality control system and analysis. NCEP/Climate Prediction Center Atlas No. 7, published online at  Iorio, J. P., P. B. Duffy, B. Govindasamy, S. L. Thompson, M. Khairoutdinov, and D. Randall, 2004: Effects of model resolution and subgrid scale physics on the simulation of precipitation in the continental United State, Clim. Dyn., 23, 243–258, doi: /s y.  Kiktev, D., D. M. H. Sexton, L. Alexander, and C. K. Folland, 2003: Comparison of modeled and observed trends in indices of daily climate extremes, J. Clim., 16, 3560–3571.  Kimoto, M., N. Yasutomi, C. Yokoyama, and S. Emori, 2005: Projected changes in precipitation characteristics near Japan under the global warming, Scientific Online Letters on the Atmosphere, 1, 85–88, doi: /sola

References Continued  Meehl, G. A., C. Covey, T. Delworth, M. Latif, B. McAvaney, J. F. B. Mitchell, R. J. Stouffer, and K. E. Taylor, 2007: The WCRP CMIP3 multi-model dataset: A new era in climate change research, Bull. Amer. Meteor. Soc., 88,  Pall, P., M. R. Allen, and D. A. Stone, 2007: Testing the Clausius-Clapeyron constraint on changes in extreme precipitation under CO2 warming, Clim. Dyn., 28,  Randall, D. A. and Coauthors, 2007: Climate Models and Their Evaluation. In: Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change [Solomon, S., D. Qin, M. Manning, Z. Chen, M. Marquis, K. B. Averyt, M. Tignor and H. L. Miller (eds.)]. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA.  Sun, Y., S. Solomon, A. Dai, and R. W. Portmann, 2007: How often will it rain? J. Clim., 20,  United States Department of the Interior, United States Geological Survey, 2006: Fact Sheet: Flood Hazards- A National Threat. Available at  For more plots, see