1. Representing model uncertainty in weather and climate: stochastic versa multi-physics representations
Judith Berner, NCAR

2. Key Points There is model error in weather and climate models
from the need to parameterize subgrid-scale fluctuations
This model error leads to overconfident uncertainty estimates and possibly model bias
We need a model error representation
Hierarchy of simulations where statistical output from one level is used to inform the next (e.g., stochastic kinetic energy backscatter)
Reliability of ensemble systems with stochastic parameterizations start to become comparable to that of ensembles systems with multi-physics

3. “Domino Parameterization strategy”
Higher-resolution model inform output of lower-resolution model
Stochastic kinetic energy backscatter scheme provides such a framework
… But there are others, e.g. Cloud-resolving convective parameterization or super-parameterization

4. Multiple scales of motion
1mm
10 m
100 m
1 km
10 km
100 km
1000 km
10000 km
Micro-
physics
Turbulence
Cumulus
clouds
Cumulonimbus
clouds
Mesoscale
Convective systems
Extratropical
Cyclones
Planetary
waves
Large Eddy Simulation (LES) Model
Cloud System Resolving Model (CSRM)
Numerical Weather Prediction (NWP) Model
Global Climate Model

5. The spectral gap …
(Stull)

6. Atmospheric Scientists
Nastrom and Gage, 1985

7. .... The link between climate forcing and climate impact involves processes acting on different timescales …

8. Resolved microphysics
NPW model
Climate model
Cloud resolving model
Cloud resolving model
Large Eddy simulation
Resolved microphysics
Attempt to capture Multi-scale nature of atmospheric motion

9. Resolved microphysics
Hierarchical Parameterization Strategy
NPW model
Climate model
Cloud resolving model
Large Eddy simulation
Resolved microphysics
Related: Grabowski 1999, Shutts and Palmer, 2007

10. Validity of spectral gap …

11. Atmospheric Scientists
The spectral gap …
Mathematicians
Atmospheric Scientists

12. Atmospheric Scientists
The spectral gap …
M pathematicians
Atmospheric Scientists

13. Spectral gap not necessary for stochastic parameterizations

14. Kinetic energy spectra in 500hPa
Rotational part
Rotational part
Kinetic energy spectrum is closer to that of T799 analysis !

15. Limited vs unlimited predictability
Rotunno and Snyder, 2008
Lorenz 1969;
see also: Tribbia and Baumhefner, 2004

16. Stochastic parameterizations have the potential to reduce model error
Stochastic parameterizations can change the mean and variance of a PDF
Impacts variability of model (e.g. internal variability of the atmosphere)
Impacts systematic error (e.g. blocking, precipitation error)
Weak noise
Strong noise
PDF
Unimodal
Multi-modal

17. Outline Acknowledgements
Parameterizations in numerical weather prediction models and climate models
A stochastic kinetic energy backscatter scheme
Impact on synoptic probabilistic weather forecasting
(short/medium-range)
Impact on systematic model error
(seasonal to climatic time-scales)
Acknowledgements
Aime Fournier, So-young Ha, Josh Hacker, Thomas Jung, Tim Palmer, Paco Doblas-Reyes, Glenn Shutts, Chris Snyder, Antje Weisheimer

18. Sensitivity to initial perturbations

19. Representing initial state uncertainty by an ensemble of states
RMS error
spread
ensemble mean
analysis
Slow
Represent initial uncertainty by ensemble of states
Flow-dependence:
Predictable states should have small ensemble spread
Unpredictable states should have large ensemble spread
Ensemble spread should grow like RMS error
True atmospheric state should be indistinguishable from ensemble system

20. Underdispersion of the ensemble system
Systems
Underdispersion of the ensemble system
The RMS error grows faster than the spread
Ensemble is underdispersive
Ensemble forecast is overconfident
spread around ensemble mean
RMS error of ensemble mean
Underdispersion is a form of model error
Forecast error = initial error + model error + boundary error
These examples demonstarted that esnemble precition systems are a very good way to assess the uncertainty of a particuklar forecast due to the uncertainly if the initial condition. Sometimes this is referred to as predicting the predictability. It is of importance that the predictaibilty
(in other words the ensemble soread) doies deopend n the stateas of the system. Eg.g. it will be easier to oredicta
The evolution og the trajectiores if there large-scale state is in a quasio-stabel equilibrium, as oiised to a highly transient state.
Now we come to the shortcoming of the current EPS. The uncertainilty we want to assess with the EPS is the difference of
The trajectory of a forecatse from the truth. Theerefore we want the ensemble to contain the trajkectory of the truth at all
Times. In other words the spread of the ensemble should grow as the mean error. It is know that the error of
current EOPS ystems is larger than the ensemble spread. This could be countercated by increaseng the pertubations og the intial
State to unrealistically large values, but this would degrade the forecast for smaller leads times. Hence there seems to be a
Source of error that does not come from the initial conditions, namely model error.
[trajectory]
Buizza et al., 2004

21. Manifestations of model error
In medium-range:
Underdispersion of ensemble system (Overconfidence)
Can “extreme” weather events be captured?
On seasonal to climatic scales:
Systematic Biases
Not enough internal variability
To which degree do e.g. climate sensitivity depend on a correct estimate of internal variability?
Shortcomings in representation of physical processes:
Underestimation of the frequency of blocking
Tropical variability, e.g. MJO, wave propagation

22. Representing model error in ensemble systems
The multi-parameterization approach: each ensemble member uses a different set of parameterizations (e.g. for cumulus convection, planetary boundary layer, microphysics, short-wave/long-wave radiation, land use, land surface)
The multi-parameter approach: each ensemble member uses the control pysics, but the parameters are varied from one ensemble member to the next
Stochastic parameterizations: each ensemble member is perturbed by a stochastic forcing term that represents the statistical fluctuations in the subgrid-scale fluxes (stochastic diabatic tendencies) as well as altogether unrepresented interactions between the resolved an unresolved scale (stochastic kinetic energy backscatter)

23. Recent attempts at remedying model error in NWP
Using conventional parameterizations
Stochastic parameterizations (Buizza et al, 1999, Lin and Neelin, 2000)
Multi-parameterization approaches (Houtekamer, 1996, Berner et al. 2010)
Multi-parameter approaches (e.g. Murphy et al,, 2004; Stainforth et al, 2004)
Multi-models (e.g. DEMETER, ENSEMBLES, TIGGE, Krishnamurti et. al 1999)
Outside conventional parameterizations
Cloud-resolving convective parameterization (CRCP) or super-parameterization (Grabowski and Smolarkiewicz 1999, Khairoutdinov and Randall 2001)
Nonlocal parameterizations, e.g., cellular automata pattern generator (Palmer, 1997, 2001)
Stochastic kinetic energy backscatter in NWP (Shutts 2005, Berner et al. 2008,2009,…)

24. Stochastic kinetic energy backscatter schemes
Stochastic kinetic energy backscatter LES
Mason and Thompon, 1992, Weinbrecht and Mason, 2008
Stochastic kinetic energy backscatter in simplified models
Frederiksen and Keupert 2004
Stochastic kinetic energy backscatter in NWP
IFS ensemble system, ECMWF:
Shutts and Palmer 2003, Shutts 2005, Berner et al. 2009a,b,
Steinheimer
MOGREPS, MetOffice
Bowler et al 2008, 2009; Tennant et al 2010
Canadian Ensemble System
Li et al 2008, Charron et al. 2010
AFWA mesoscale ensemble system, NCAR
Berner et al. 2010

25. Forcing streamfunction spectra by coarse-graining CRMs
from Glenn Shutts

26. “Domino Parameterization strategy”
Higher-resolution model inform output of lower-resolution model
Stochastic kinetic energy backscatter scheme provides such a framework
… But there are others, e.g. Cloud-resolving convective parameterization or super-parameterization

27. Model error in weather forecasting and climate models
A stochastic kinetic energy backscatter scheme (SPBS)
Impact of SPBS on probabilistic weather forecasting (medium-range) -> Martin’s talk
Impact of SPBS on systematic model error
Impact in a mesoscale model and comparison to a multi-physics scheme

28. Forecast error growth
For perfect ensemble system:
the true atmospheric state should be indistinguishable from a perturbed ensemble member
forecast error and model uncertainty (=spread) should be the same
Since IPs are reduced, forecast error is reduced for small forecast times
More kinetic energy in small scales

29. Model error in weather forecasting and climate models
A stochastic kinetic energy backscatter scheme: SPectral Backscatter Scheme
Impact of SPBS on probabilistic weather forecasting (medium-range)
Impact of SPBS on systematic model error
Impact in a mesoscale model and comparison to a multi-physics scheme

30. Experimental Setup for Seasonal Runs
“Seasonal runs: Atmosphere only”
Atmosphere only, observed SSTs
40 start dates between 1962 – 2001 (Nov 1)
5-month integrations
One set of integrations with stochastic backscatter, one without
Model runs are compared to ERA40 reanalysis (“truth”)

31. No StochasticBackscatter Stochastic Backscatter
Reduction of systematic error of z500 over North Pacific and North Atlantic
No StochasticBackscatter
Stochastic Backscatter

32. Increase in occurrence of Atlantic and Pacific blocking
ERA40 + confidence interval
Stochastic Backscatter
No StochasticBackscatter

33. No Stochastic Backscatter
Wavenumber-Frequency Spectrum Symmetric part, background removed (after Wheeler and Kiladis, 1999)
Observations (NOAA)
No Stochastic Backscatter

34. Improvement in Wavenumber-Frequency Spectrum
Observations (NOAA)
Stochastic Backscatter
Backscatter scheme reduces erroneous westward propagating modes

35. Model error in weather forecasting and climate models
A stochastic kinetic energy backscatter scheme: SPectral Backscatter Scheme
Impact of SPBS on probabilistic weather forecasting (medium-range)
Impact of SPBS on systematic model error
Impact in a mesoscale model and comparison to a multi-physics scheme

36. Experiment setup
Ensemble A/B: 10 member ensemble with and without SPBS
Ensemble C: 10 member multi-physics suite
Weather Research and Forecast Model
30 cases between Nov 2008 and Feb 2009
40km horizontal resolution and 40 vertical levels
Limited area model: Continuous United States (CONUS)
Started from GFS initial condition (downscaled from NCEPs Global Forecast System)

37. Multiple Physics packages

38. Control Physics Ensemble
WRF short-range ensemble: 60h-forecast for Oct 13, 2006: SLP and surface wind
Control Physics Ensemble

39. Stochastic Backscatter Ensemble
WRF short-range ensemble: 60h-forecast for Oct 13, 2006: SLP and surface wind
Stochastic Backscatter Ensemble

40. Spread-Error Relationship
Control
Backscatter
Multi-Physics
PHYS_STOCH

41. Brier Score, U
Control
Backscatter
Multi-Physics
PHYS_STOCH

42. Scatterplots of verification scores
Both, Stochastic backscatter and Multi-physics are better than control
Stochastic backscatter is better than Multi-physics is better
Their combination is even better

43. Multiple Physics packages

44. Brier Score
Control
Multi-Physics
Backscatter

45. Spread-Error Relationship
Control
Backscatter
Multi-Physics
PHYS_STOCH

46. Seasonal Predication Uncalibrated Calibrated Stochastic Ensemble
Multi-model
Curtosy: TimPalmer

47. Summary and conclusion
Stochastic parameterization have the potential to reduce model error by changing the mean state and internal variability.
It was shown that the new stochastic kinetic energy backscatter scheme (SPBS) produced a more skilful ensemble and reduced certain aspects of systematic model error
Increases predictability across the scales (from mesoscale over synoptic scale to climatic scales)
Stochastic Backscatter outperforms Multi-physics Ens.
Stochastic backscatter scheme provides a framework for hierarchical parameterization strategy, where stochastic parameterization for the lower resolution model is informed by higher resolution model

48. Future Work Understand the nature of model error better
Inform more parameters from coarse-grained high-resolution output
Impact on climate sensitivity
Consequences for error growth and predictability

49. Challenges
How can we incorporate the “structural uncertainty” estimated by multi-models into stochastic parameterizations?

50. Bibliography
Berner, J., 2005: Linking Nonlinearity and non-Gaussianity by the Fokker-Planck equation and the associated nonlinear stochastic model, J. Atmos. Sci., 62, pp
Shutts, G. J., 2005: A kinetic energy backscatter algorithm for use in ensemble prediction systems. Quart. J. Roy. Meteor. Soc., 612,
Berner, J., F. J. Doblas-Reyes, T. N. Palmer, G. Shutts, and A. Weisheimer, 2008: Impact of a quasi-stochastic cellular automaton backscatter scheme on the systematic error and seasonal predicition skill of a global climate model, Phil. Trans. R. Soc A, 366, pp , DOI: /rsta
Berner J., G. Shutts, M. Leutbecher, and T.N. Palmer, 2009: A Spectral Stochastic Kinetic Energy Backscatter Scheme and its Impact on Flow-dependent Pre- dictability in the ECMWF Ensemble Prediction System, J. Atmos. Sci.,66,pp
T.N. Palmer, F.J. Doblas-Reyes, A. Weisheimer, G.J. Shutts, J. Berner, J.M. Murphy, 2008: Towards the Probabilistic Earth-System Model, J.Clim., in preparation