Representing model uncertainty in weather and climate: stochastic versa multi-physics representations Judith Berner, NCAR

Key Points There is model error in weather and climate models from the need to parameterize subgrid-scale fluctuations This model error leads to overconfident uncertainty estimates and possibly model bias We need a model error representation Hierarchy of simulations where statistical output from one level is used to inform the next (e.g., stochastic kinetic energy backscatter) Reliability of ensemble systems with stochastic parameterizations start to become comparable to that of ensembles systems with multi-physics

“Domino Parameterization strategy” Higher-resolution model inform output of lower-resolution model Stochastic kinetic energy backscatter scheme provides such a framework … But there are others, e.g. Cloud-resolving convective parameterization or super-parameterization

Multiple scales of motion 1mm 10 m 100 m 1 km 10 km 100 km 1000 km 10000 km Micro- physics Turbulence Cumulus clouds Cumulonimbus clouds Mesoscale Convective systems Extratropical Cyclones Planetary waves Large Eddy Simulation (LES) Model Cloud System Resolving Model (CSRM) Numerical Weather Prediction (NWP) Model Global Climate Model

The spectral gap … (Stull)

Atmospheric Scientists Nastrom and Gage, 1985

.... The link between climate forcing and climate impact involves processes acting on different timescales …

Resolved microphysics NPW model Climate model Cloud resolving model Cloud resolving model Large Eddy simulation Resolved microphysics Attempt to capture Multi-scale nature of atmospheric motion

Resolved microphysics Hierarchical Parameterization Strategy NPW model Climate model Cloud resolving model Large Eddy simulation Resolved microphysics Related: Grabowski 1999, Shutts and Palmer, 2007

Validity of spectral gap …

Atmospheric Scientists The spectral gap … Mathematicians Atmospheric Scientists

Atmospheric Scientists The spectral gap … M pathematicians Atmospheric Scientists

Spectral gap not necessary for stochastic parameterizations

Kinetic energy spectra in 500hPa Rotational part Rotational part Kinetic energy spectrum is closer to that of T799 analysis !

Limited vs unlimited predictability Rotunno and Snyder, 2008 Lorenz 1969; see also: Tribbia and Baumhefner, 2004

Stochastic parameterizations have the potential to reduce model error Stochastic parameterizations can change the mean and variance of a PDF Impacts variability of model (e.g. internal variability of the atmosphere) Impacts systematic error (e.g. blocking, precipitation error) Weak noise Strong noise PDF Unimodal Multi-modal

Outline Acknowledgements Parameterizations in numerical weather prediction models and climate models A stochastic kinetic energy backscatter scheme Impact on synoptic probabilistic weather forecasting (short/medium-range) Impact on systematic model error (seasonal to climatic time-scales) Acknowledgements Aime Fournier, So-young Ha, Josh Hacker, Thomas Jung, Tim Palmer, Paco Doblas-Reyes, Glenn Shutts, Chris Snyder, Antje Weisheimer

Sensitivity to initial perturbations

Representing initial state uncertainty by an ensemble of states RMS error spread ensemble mean analysis Slow Represent initial uncertainty by ensemble of states Flow-dependence: Predictable states should have small ensemble spread Unpredictable states should have large ensemble spread Ensemble spread should grow like RMS error True atmospheric state should be indistinguishable from ensemble system

Underdispersion of the ensemble system Systems Underdispersion of the ensemble system The RMS error grows faster than the spread Ensemble is underdispersive Ensemble forecast is overconfident ------- spread around ensemble mean RMS error of ensemble mean Underdispersion is a form of model error Forecast error = initial error + model error + boundary error These examples demonstarted that esnemble precition systems are a very good way to assess the uncertainty of a particuklar forecast due to the uncertainly if the initial condition. Sometimes this is referred to as predicting the predictability. It is of importance that the predictaibilty (in other words the ensemble soread) doies deopend n the stateas of the system. Eg.g. it will be easier to oredicta The evolution og the trajectiores if there large-scale state is in a quasio-stabel equilibrium, as oiised to a highly transient state. Now we come to the shortcoming of the current EPS. The uncertainilty we want to assess with the EPS is the difference of The trajectory of a forecatse from the truth. Theerefore we want the ensemble to contain the trajkectory of the truth at all Times. In other words the spread of the ensemble should grow as the mean error. It is know that the error of current EOPS ystems is larger than the ensemble spread. This could be countercated by increaseng the pertubations og the intial State to unrealistically large values, but this would degrade the forecast for smaller leads times. Hence there seems to be a Source of error that does not come from the initial conditions, namely model error. [trajectory] Buizza et al., 2004

Manifestations of model error In medium-range: Underdispersion of ensemble system (Overconfidence) Can “extreme” weather events be captured? On seasonal to climatic scales: Systematic Biases Not enough internal variability To which degree do e.g. climate sensitivity depend on a correct estimate of internal variability? Shortcomings in representation of physical processes: Underestimation of the frequency of blocking Tropical variability, e.g. MJO, wave propagation

Representing model error in ensemble systems The multi-parameterization approach: each ensemble member uses a different set of parameterizations (e.g. for cumulus convection, planetary boundary layer, microphysics, short-wave/long-wave radiation, land use, land surface) The multi-parameter approach: each ensemble member uses the control pysics, but the parameters are varied from one ensemble member to the next Stochastic parameterizations: each ensemble member is perturbed by a stochastic forcing term that represents the statistical fluctuations in the subgrid-scale fluxes (stochastic diabatic tendencies) as well as altogether unrepresented interactions between the resolved an unresolved scale (stochastic kinetic energy backscatter)

Recent attempts at remedying model error in NWP Using conventional parameterizations Stochastic parameterizations (Buizza et al, 1999, Lin and Neelin, 2000) Multi-parameterization approaches (Houtekamer, 1996, Berner et al. 2010) Multi-parameter approaches (e.g. Murphy et al,, 2004; Stainforth et al, 2004) Multi-models (e.g. DEMETER, ENSEMBLES, TIGGE, Krishnamurti et. al 1999) Outside conventional parameterizations Cloud-resolving convective parameterization (CRCP) or super-parameterization (Grabowski and Smolarkiewicz 1999, Khairoutdinov and Randall 2001) Nonlocal parameterizations, e.g., cellular automata pattern generator (Palmer, 1997, 2001) Stochastic kinetic energy backscatter in NWP (Shutts 2005, Berner et al. 2008,2009,…)

Stochastic kinetic energy backscatter schemes Stochastic kinetic energy backscatter LES Mason and Thompon, 1992, Weinbrecht and Mason, 2008 Stochastic kinetic energy backscatter in simplified models Frederiksen and Keupert 2004 Stochastic kinetic energy backscatter in NWP IFS ensemble system, ECMWF: Shutts and Palmer 2003, Shutts 2005, Berner et al. 2009a,b, Steinheimer MOGREPS, MetOffice Bowler et al 2008, 2009; Tennant et al 2010 Canadian Ensemble System Li et al 2008, Charron et al. 2010 AFWA mesoscale ensemble system, NCAR Berner et al. 2010

Forcing streamfunction spectra by coarse-graining CRMs from Glenn Shutts

“Domino Parameterization strategy” Higher-resolution model inform output of lower-resolution model Stochastic kinetic energy backscatter scheme provides such a framework … But there are others, e.g. Cloud-resolving convective parameterization or super-parameterization

Model error in weather forecasting and climate models A stochastic kinetic energy backscatter scheme (SPBS) Impact of SPBS on probabilistic weather forecasting (medium-range) -> Martin’s talk Impact of SPBS on systematic model error Impact in a mesoscale model and comparison to a multi-physics scheme

Forecast error growth For perfect ensemble system: the true atmospheric state should be indistinguishable from a perturbed ensemble member forecast error and model uncertainty (=spread) should be the same Since IPs are reduced, forecast error is reduced for small forecast times More kinetic energy in small scales

Model error in weather forecasting and climate models A stochastic kinetic energy backscatter scheme: SPectral Backscatter Scheme Impact of SPBS on probabilistic weather forecasting (medium-range) Impact of SPBS on systematic model error Impact in a mesoscale model and comparison to a multi-physics scheme

Experimental Setup for Seasonal Runs “Seasonal runs: Atmosphere only” Atmosphere only, observed SSTs 40 start dates between 1962 – 2001 (Nov 1) 5-month integrations One set of integrations with stochastic backscatter, one without Model runs are compared to ERA40 reanalysis (“truth”)

No StochasticBackscatter Stochastic Backscatter Reduction of systematic error of z500 over North Pacific and North Atlantic No StochasticBackscatter Stochastic Backscatter

Increase in occurrence of Atlantic and Pacific blocking ERA40 + confidence interval Stochastic Backscatter No StochasticBackscatter

No Stochastic Backscatter Wavenumber-Frequency Spectrum Symmetric part, background removed (after Wheeler and Kiladis, 1999) Observations (NOAA) No Stochastic Backscatter

Improvement in Wavenumber-Frequency Spectrum Observations (NOAA) Stochastic Backscatter Backscatter scheme reduces erroneous westward propagating modes

Model error in weather forecasting and climate models A stochastic kinetic energy backscatter scheme: SPectral Backscatter Scheme Impact of SPBS on probabilistic weather forecasting (medium-range) Impact of SPBS on systematic model error Impact in a mesoscale model and comparison to a multi-physics scheme

Experiment setup Ensemble A/B: 10 member ensemble with and without SPBS Ensemble C: 10 member multi-physics suite Weather Research and Forecast Model 30 cases between Nov 2008 and Feb 2009 40km horizontal resolution and 40 vertical levels Limited area model: Continuous United States (CONUS) Started from GFS initial condition (downscaled from NCEPs Global Forecast System)

Multiple Physics packages

Control Physics Ensemble WRF short-range ensemble: 60h-forecast for Oct 13, 2006: SLP and surface wind Control Physics Ensemble

Stochastic Backscatter Ensemble WRF short-range ensemble: 60h-forecast for Oct 13, 2006: SLP and surface wind Stochastic Backscatter Ensemble

Spread-Error Relationship Control Backscatter Multi-Physics PHYS_STOCH

Brier Score, U Control Backscatter Multi-Physics PHYS_STOCH

Scatterplots of verification scores Both, Stochastic backscatter and Multi-physics are better than control Stochastic backscatter is better than Multi-physics is better Their combination is even better

Multiple Physics packages

Brier Score Control Multi-Physics Backscatter

Spread-Error Relationship Control Backscatter Multi-Physics PHYS_STOCH

Seasonal Predication Uncalibrated Calibrated Stochastic Ensemble Multi-model Curtosy: TimPalmer

Summary and conclusion Stochastic parameterization have the potential to reduce model error by changing the mean state and internal variability. It was shown that the new stochastic kinetic energy backscatter scheme (SPBS) produced a more skilful ensemble and reduced certain aspects of systematic model error Increases predictability across the scales (from mesoscale over synoptic scale to climatic scales) Stochastic Backscatter outperforms Multi-physics Ens. Stochastic backscatter scheme provides a framework for hierarchical parameterization strategy, where stochastic parameterization for the lower resolution model is informed by higher resolution model

Future Work Understand the nature of model error better Inform more parameters from coarse-grained high-resolution output Impact on climate sensitivity Consequences for error growth and predictability

Challenges How can we incorporate the “structural uncertainty” estimated by multi-models into stochastic parameterizations?

Bibliography Berner, J., 2005: Linking Nonlinearity and non-Gaussianity by the Fokker-Planck equation and the associated nonlinear stochastic model, J. Atmos. Sci., 62, pp. 2098-2117 Shutts, G. J., 2005: A kinetic energy backscatter algorithm for use in ensemble prediction systems. Quart. J. Roy. Meteor. Soc., 612, 3079-3102 Berner, J., F. J. Doblas-Reyes, T. N. Palmer, G. Shutts, and A. Weisheimer, 2008: Impact of a quasi-stochastic cellular automaton backscatter scheme on the systematic error and seasonal predicition skill of a global climate model, Phil. Trans. R. Soc A, 366, pp. 2561-2579, DOI: 10.1098/rsta.2008.0031. Berner J., G. Shutts, M. Leutbecher, and T.N. Palmer, 2009: A Spectral Stochastic Kinetic Energy Backscatter Scheme and its Impact on Flow-dependent Pre- dictability in the ECMWF Ensemble Prediction System, J. Atmos. Sci.,66,pp.603-626 T.N. Palmer, F.J. Doblas-Reyes, A. Weisheimer, G.J. Shutts, J. Berner, J.M. Murphy, 2008: Towards the Probabilistic Earth-System Model, J.Clim., in preparation