1. Uncertainties and climate simulation ensembles
RCM workshop
Meteo Rwanda, Kigali
17th – 20th July 2017

2. Aims of this session
Describe sources of uncertainty in projections of future regional climate
Demonstrate how ensembles of climate model simulations are used to explore different types of uncertainty
Appreciate the practical considerations for designing ensembles of Regional Climate Model simulations

3. Outline of the session Exercise: Why use ensembles?
Talk: Uncertainties in climate projections
Talk: Designing ensemble downscaling experiments
Exercise: Designing an ensemble
Discussion: What have we learnt?

4. Exercise 1

5. Ensembles
general: “a group of items viewed as a whole rather than individually”
climate science: “the results from several model simulations, so that each single simulation is considered only in context of the results of all of the simulations”
What do ensembles tell us?
The range and distribution of different future climates from the available models
What DON’T ensembles tell us (on their own)?
The most likely future climate outcome (the mean is not the most likely)
The finite range of plausible outcomes (e.g. worst case scenario)
Explain the general and climate-specific definitions of ensembles, followed by what they do and don’t tell us. The primary objective of using an ensemble is to explore the range of possible future climate and help quantity uncertainty.
Often a central estimate is interpreted as a ‘most likely’ outcome but this is not true – without other information, all ensemble members should be treated as equally likely. Moreover, the most commonly generated projection is not necessarily more likely than any other.
Finally, the range of projections might be larger if a new model is added and the range of projections from an ensemble may not include all possibilities!

6. Uncertainty = lack of certainty
What is uncertainty?
Uncertainty = lack of certainty
State that the principal reason for running ensemble experiments is because of uncertainty.
Ask the participants what their definition of uncertainty is? If there is a white board, write up some of the key words (probably along the lines of error, probability, chance, ignorance, unknowns etc). A number of different answers will be given and state that all are valid (probably) as uncertainty means different things to different people. However it is useful to remember that uncertainty simply means lack of certainty – every prediction of the future (be it economic, political, about the planets etc) is uncertain. Yet climate scientists tend to have a very specific framework for thinking about uncertainties in climate prediction and how to quantify them.
The cartoon shows that having more than one clock makes you uncertain about what the time actually is.
© Crown copyright Met Office

7. The “cascade of uncertainty”
Discuss the different components in the “cascade of uncertainty”. Some of them are inherent and irreducible, others are the subject of intense scientific endeavours to try and reduce the uncertainties – but no matter what, we must account for a range of futures.
The box draws attention to the focus of this workshop. Explain that the ensembles talk will go into more detail on the choices that can be made – i.e. How to sample the uncertainty – but that this still forms only a partial component of the overall cascade of uncertainties that ultimately decision makers must consider.
Wilby and Dessai, 2010
Robust Adaptation to Climate Change
© Crown copyright Met Office

8. Sources of climate projection uncertainty
Scenario uncertainty
Human and natural emissions of greenhouse gases
Translating emissions into concentrations of greenhouse gases and their effect on system radiative forcings
Initial Condition uncertainty
Sparse/incomplete observations in time and space
Erroneous and uncertain observations
Model uncertainty
Model error and inadequacy
Parameter uncertainty
Briefly introduce that the uncertainty relevant to climate prediction can be categorised into three primary sources.
© Crown copyright Met Office

9. 1. Emission Scenario Uncertainty
Uncertainties in the key assumptions and relationship about future population, socio-economic development and technical changes.
We are currently working with 2 sets of scenarios:
SRES (used for CMIP3 / IPCC AR4)
RCPs (used for CMIP5 / IPCC AR5)
The IPCC does not assign probabilities to these scenarios.
Uncertain future emissions due to uncertainties in future path taken by society. Therefore the IPCC SRES have developed a series of potential pathways to explore the future world.
There are inherent uncertainties in the key assumptions and relationship about future population, socio-economic development and technical changes that are the bases of the IPCC SRES Scenarios. However the uncertain nature of these emissions paths have been well documented (Morita and Robinson, 2001).
State that the way we deal with emissions uncertainty has changed in CMIP5 – we now use RCPs, explained in following slides.
© Crown copyright Met Office

10. 1. Emission Scenario Uncertainty
Representative Concentration Pathways (RCPs)
Explain what the RCP means – i.e. It is the global radiative difference in 2100 compare to pre-industrial in CO2 equivalent.
An RCP is representative of a range of possible scenarios. Because of uncertainties in the Earth’s carbon cycle, different emissions scenarios can lead to the same concentration of GHGs and/or forcing on the climate system – as shown by the figure on the left. Show that all the red emissions scenarios lead to a 8.5 W/m^2 difference by However, in general a low GHG emissions future is more likely to lead to less of a radiative forcing (e.g. RCP 2.6), while others are consistent with a higher GHG emissions future (e.g. RCP8.5) .
The right panel compares new RCPs used in CMIP5 to the SRES scenarios used in CMIP3.
Fuss et al. 2014
Source: Climate Futures
© Crown copyright Met Office

11. 1. Emission Scenario Uncertainty
This shows the impact of different RCPs on global average surface temperature. So for a given RCP, there are a range of global responses, but all show global warming, even RCP2.6.
McSweeney and Jones, In Press

12. 2. Initial Condition Uncertainty
This graph shows the mean central England temperature, showing annual anomalies relative to the average. The red line is a 21-point binomial filter, which is roughly equivalent to a 10-year running mean. It is shown to simply illustrate that the climate of any location is variable – even a 10 year average shows substantial variability.

13. 2. Initial Condition Uncertainty
CCSM3 model
40 member ensemble
DJF temperature trend
using A1B
Figure shows data taken from a 40 member ensemble run of the Community Climate System Model version 3 (CCSM3)
Each ensemble member undergoes the same A1B greenhouse-gas forcing scenario. The plots show the December–January–February (DJF) temperature trends during 2005–2060. The top panel shows the average of the 40 model runs; middle and bottom panels show the model runs with the largest and smallest trends for the contiguous United States as a whole, respectively.
The message is that even for the same model and forcing scenario, different ICs can lead to different projections, sometime even of a different sign.
Deser et al (2012) Nature Climate Change

14. 3. Model Uncertainty Only one planet Earth Take a breather.
Then say that we have a whole host of global climate models, but only one Earth. There are different ways to the model the Earth system, and no single way is the best way (possibly joke that the Met Office obviously has the best approach). Why is this...?

15. What is parameterization?
It is a method of replacing processes that are too small-scale or complex to be physically represented in the model by a simplified process
Say parameterization is a term that comes up a lot in this section – do they know what it means?
Parameterization in a weather or climate model within numerical weather prediction is a method of replacing processes that are too small-scale or complex to be physically represented in the model by a simplified process.
Convective clouds which can’t be resolved by the model grid is an example.

16. Why and how do models differ?
Structural Uncertainty
(e.g. CMIP3/5 multi-model ensemble)
Models within CMIP5 differ in the way they describe/represent/approximate different processes. Often these schemes will be common to several models, sometime large parts of code are shared – e.g. CSIRO’S ACCESS models share the same atmospheric physics as Met office HadGEM2-ES, but use different landsurface schemes and ocean/sea ice.
Parameter Uncertainty
(e.g. QUMP perturbed physics ensemble)

17. List of Current CMIP GCMs
Center
Models
Institution
Country
BCC
BCC-CSM1.1, BCC-CSM1.1(m)
Beijing Climate Center, China Meteorological Administration
China
CCCma
CanAM4, CanCM4, CanESM2
Canadian Centre for Climate Modelling and Analysis
Canada
CMCC
CMCC-CM, CMCC-CMS,   CMCC-CESM
Centro Euro-Mediterraneo per I Cambiamenti Climatici
Italy
CNRM-CERFACS
CNRM-CM5, CNRM-CM5-2
Centre National de Recherches Meteorologiques
France
COLA and NCEP
CFSv2-2011
Center for Ocean-Land-Atmosphere Studies and National Center for Environmental Prediction
USA
CSIRO-BOM
ACCESS1.0, ACCESS1.3
CSIRO (Commonwealth Scientific and Industrial Research Organisation) and BOM (Bureau of Meteorology)
Australia
CSIRO-QCCCE
CSIRO-Mk3.6.0
CSIRO (Commonwealth Scientific and Industrial Research Organisation) and the Queensland Climate Change Centre of Excellence
EC-EARTH
EC-EARTH consortium
International consortium
FIO
FIO-ESM
The First Institute of Oceanography, SOA
GCESS
BNU-ESM
College of Global Change and Earth System Science, Beijing Normal University
INM
INM-CM4
Institute for Numerical Mathematics
Russia
IPSL
IPSL-CM5A-LR, IPSL-CM5A-MR,   IPSL-CM5B-LR
Institut Pierre-Simon Laplace
LASG-CESS
FGOALS-g2
LASG, Institute of Atmospheric Physics, Chinese Academy of Sciences; and CESS, Tsinghua University
LASG-IAP
FGOALS-gl, FGOALS-s2
LASG, Institute of Atmospheric Physics, Chinese Academy of Sciences
MIROC
MIROC4h, MIROC5, MIROC-ESM, MIROC-ESM-CHEM
Japan Agency for Marine-Earth Science and Technology, Atmosphere and Ocean Research Institute (The University of Tokyo), and National Institute for Environmental Studies
Japan
MOHC
HadCM3, HadCM3Q, HadGEM2-A, HadGEM2-CC, HadGEM2-ES
Met Office Hadley Centre UK
MPI-M
MPI-ESM-LR, MPI-ESM-LR, MPI-ESM-P
Max Planck Institute for Meteorology (MPI-M)
Germany
MRI
MRI-AGCM3.2H, MRI-AGCM3.2S, MRI-CGCM3, MRI-ESM1
Meteorological Research Institute
NASA GISS
GISS-E2-H, GISS-E2-H-CC, GISS-E2-R, GISS-E2-R-CC
NASA Goddard Institute for Space Studies
NASA GMAO
GEOS-5
NASA Global Modeling and Assimilation Office
NCAR
CCSM4
National Center for Atmospheric Research (NCAR)
NCC
NorESM1-M, NorESM1-ME
Norwegian Climate Centre
Norway
NICAM
NICAM.09
Nonhydrostatic Icosahedral Atmospheric Model Group
NIMR/KMA
HadGEM2-AO
National Institute of Meteorological Research/Korea Meteorological Administration
South Korea
NOAA GFDL
GFDL-CM2.1, GFDL-CM3, GFDL-ESM2G, GFDL-ESM2M, GFDL-HIRAM-C180, GFDL-HIRAM-C360
Geophysical Fluid Dynamics Laboratory
NSF-DOE-NCAR
CESM1
National Science Foundation, Department of Energy, National Center for Atmospheric Research
This slide is crowded – but that is the point. There are a large number of available GCMs that have been developed to feed into CMIP. This table shows the different modelling centres, the host countries, and the range of models available from each institution. Some of the modelling institutions share model components (e.g. MOHC and CSIRO-BOM models are very similar) whilst others are very different. Note that there is a growing literature on model independence and the “genealogy” of models.

18. Perturbed parameter ensembles (PPEs)
Systematic sampling can be done by perturbing parameter values in a single GCM (but structural uncertainty is not sampled)
Each ensemble member has different sets of parameter values chosen from likely ranges
The CMIP ensemble is an “ensemble of opportunity” that does not systematically sample uncertainty – it is hard to assign probabilities to different simulation outcomes and it cannot be guaranteed that plausible extreme climate changes are being sampled.
PPEs offer the opportunity to do systematic sampling. This is done by taking a closer look at parameters in a single GCM that are very uncertain.
A disadvantage is that structural uncertainty is not accounted for – the fundamental equations in the different ensemble members are all the same.
The pink and purple lines on the diagram represent different ensemble members with different settings of key parameters (in reality there would be many more parameters)

19. Perturbed parameter ensembles (PPEs)
Systematic sampling can be done by perturbing parameter values in a single GCM (but structural uncertainty is not sampled)
Each ensemble member has different sets of parameter values chosen from likely ranges
Parameter
Range of values
Ice fall speed
Degree of cloud overlap
Roughness of sea surface
Roughness of forests
Depth of plant roots :
The CMIP ensemble is an “ensemble of opportunity” that does not systematically sample uncertainty – it is hard to assign probabilities to different simulation outcomes and it cannot be guaranteed that plausible extreme climate changes are being sampled.
PPEs offer the opportunity to do systematic sampling. This is done by taking a closer look at parameters in a single GCM that are very uncertain.
A disadvantage is that structural uncertainty is not accounted for – the fundamental equations in the different ensemble members are all the same.
The pink and purple lines on the diagram represent different ensemble members with different settings of key parameters (in reality there would be many more parameters)

20. Land surface processes
Parameters perturbed in the Met Office QUMP PPE
Boundary layer
Turbulent mixing coefficients: stability-dependence, neutral mixing length
Roughness length over sea: Charnock constant, free convective value
Large Scale Cloud
Ice fall speed
Critical relative humidity for formation
Cloud droplet to rain: conversion rate and threshold
Cloud fraction calculation
Dynamics
Diffusion: order and e-folding time
Gravity wave drag: surface and trapped lee wave constants
Gravity wave drag start level
Convection
Entrainment rate
Intensity of mass flux
Shape of cloud (anvils) (*)
Cloud water seen by radiation (*)
Land surface processes
Root depths
Forest roughness lengths
Surface-canopy coupling
CO2 dependence of stomatal conductance (*)
The diagram in the previous slide is a simplified example. In reality there are many more parameters that can be perturbed.
An example of a PPE that can be used with PRECIS is QUMP (Quantifying Uncertainty in Model Predictions), in which different versions of HadCM3 were generated by perturbing parameters.
This slide shows the different parameters that are perturbed in QUMP – there is no need to go through each – the point is that there are many.
HadCM3 is an old Met Office climate model – to do a decent ensemble size, we must be able to run a fast model (often an old model).
We therefore span uncertainty at the expense of leaving out some of the advanced features of the latest climate model.
Radiation
Ice particle size/shape
Cloud overlap assumptions
Water vapour continuum absorption (*)
Sea ice
Albedo dependence on temperature
Ocean-ice heat transfer

21. Contributions to overall uncertainty
Take another breather.
This graph – and link to the interactive online tool – shows the three main sources of climate uncertainty and how the relative contributions change with prediction lead time. Model uncertainty and internal variability dominate at shorter lead times, scenario uncertainty becomes increasingly more important on longer lead times.
Play with the interactive tool and show how the contributions change as a function of variable, aggregation time and spatial scale.
Hawkins and
Sutton, 2009
© Crown copyright Met Office

22. Designing Ensemble Downscaling Experiments
In this section you will get to consider how ensembles of downscaling experiments are designed.

23. Regional climate projections for impact studies
Uncertainty in large-scale climate (GCM ensemble)
Multiple RCMs / downscaling methods
Large resource implications!
Strategically sample large-scale climates from GCM simulations
Downscale these using one or more Regional Climate Models / statistical methods
Communicate results and provide impact-relevant datasets (more on this later)
Let’s assume we are generating regional climate projections to examine climate impacts.
It’s a good idea to sample large scale climate by using a GCM ensemble and perhaps even uncertainties in downscaling by using different RCMs, and perhaps even different statistical methods too.
The approaches using multiple GCMs and RCMs need a lot of computing resources, so we must think carefully about how we design our model ensembles.
An other challenge with ensembles is communicating the results – we’ll deal with this near the end of the workshop.

24. Designing ensembles: What is a good strategy?
I can only downscale one GCM. Shall I choose the one that best simulates the observed climate?
I might choose 4 GCMs that span the widest possible range of future rainfall changes.
I could use a higher resolution RCM, but do I have enough computing resource?
A picture of a scientist trying to work out how she might design an ensemble. Show this just before the upcoming exercise. If participants get stuck during the exercise, this might be a good slide to come back to and leave on the screen during the exercise or to have a group discussion on.
I see a lot of variability in the historical climate. Perhaps I need to use simulations with different initial conditions.

25. Exercise 2

26. Assessing the impact of climate change on New Devon by 2050
Read the instructions carefully!
Select from the given experimental designs
Be prepared to report back on the reasons for your selection 15 mins from now
There is no right answer – your reasoning is more important than your ensemble selection
Feel free to ask questions!
Emphasis that there is no right answer to the exercise and that at the end the important thing is for each group to be able to talk about their reasons for selecting or rejecting different experimental designs.

27. GCM simulations that can be downscaled
Simulations available
Realisations
Reproduction of
observed climate A
RCP4.5 and RCP8.5 1
Good B
Moderate C
RCP4.5 4 D
RCP8.5 E
Poor
Show this slide at the end of the exercise, go through each of the experimental designs and ask the groups to say if they selected the design and why. Also, ask why groups who have not selected the design rejected it. The upper table and diagram are to aid the discussion. All of this material is reproduced from the exercise hand outs.
Experimental designs 1
The RCP8.5 simulation of GCM A downscaled to 25km 2
All four simulations of GCM C downscaled to 50km 3
The RCP8.5 simulations of GCMs A, B, D and the C4 simulation of GCM C downscaled to 50km 4
The RCP8.5 simulations of GCMs A, B, E and the C4 simulation of GCM C downscaled to 50km 5
The RCP4.5 and RCP8.5 simulations of GCMs A and B downscaled to 50km

28. End of Exercise 2!

29. Recommended approach:
Evaluation: Eliminate any models which are so poor that we have good reason to not trust the projections
Future projections: sample the range of future outcomes
Both can be analysed using GCM output from the CMIP dataset
Carol McSweeney at the Met Office has written a paper that might help you select GCMs for downscaling. The approach first rejects models with poor representation of key processes and then selects models from the remainder that sample uncertainties in simulated future climate changes.

30. Example: Simulation of Asian Monsoon Circulation in CMIP5
Evaluation: Eliminate any GCMs that are so poor that we do not trust the projections
Example: Simulation of Asian Monsoon Circulation in CMIP5
An example of Carol’s first GCM selection step – model evaluation. Simulated Asian monsoon flow is compared with observed (top left panel). The MIROC-ESM GCMs (in red box) are rejected because the eastward flow does not extend eastwards enough. For example, east-west convergence in these GCMs is over Indochina rather than over West Pacific.

31. Example: Seasonal climate changes for southern Asia
Future projections: Select simulations that span the range in simulated future climate changes
Example: Seasonal climate changes for southern Asia
Change in precipitation by 2080s (mm/day)
Don’t go through this in detail. The gist is that diagrams of simulated relevant future climate changes (here precip and temp change by the 2080s for the 4 seasons) are plotted up and GCMs are selected out of those not previously rejected that give the largest span in results. These diagrams are analogous to the one used in the exercise, but there is more complexity here due to different seasons.
Change in temperature by 2080s (C)

32. PRECIS Ensembles Multi-RCP Ensemble (HadGEM2-ES)
PRECIS can currently downscale RCP2.6, RCP4.5 and RCP8.5 simulations of HadGEM2-ES.
Multi-model Ensemble (CMIP5)
PRECIS 2.1 due for release soon will feature ability to downscale multiple models from CMIP5 (initially GFDL-CM3 and CNRM-CM5). Other models to follow.
Perturbed-Physics ensemble (‘QUMP’)
17-member ensemble of HadCM3 (HadCM3 Q0-16)
We can select a sub-set of the 17 models in order to run a computationally ‘affordable’ experiment.
Initial conditions ensemble
In progress
You can use PRECIS with different types of ensemble. PRECIS can already downscale HadGEM2-ES for two RCPs and for the QUMP PPE. The ability to downscale more CMIP5 GCMs and for individual GCMs with different initial conditions is currently being worked on.

33. What have we learnt?

34. Summary
Ensembles of model simulations help us sample uncertainty in future climate outcomes
With PRECIS, we can sample a range of uncertainties due to
Emissions scenarios
GCMs imperfections
Climate variability
BUT
Big ensemble → Big computer!
Use of additional RCMs may more fully sample uncertainties
No ensemble will fully sample uncertainties in the real world
“No ensemble will fully sample uncertainties in the real world” – this is an important point. Because all our models are imperfect, there is a chance that the real world will behave in a way that is not covered by our ensemble – this possibility should not be forgotted.

35. END: Discussion and questions

36. 2. Initial Condition Uncertainty
Use initial condition ensembles for understanding variability, changes in variability or improving detectability of climate change signals.
A single realisation may incompletely sample the natural cycle of variability. 1st example shows a single realisation which could be interpreted as indicating a reduction in variable Y. By showing multiple realisation we can see that what could look like a ‘signal’ looks more like natural variability in the context of multiple realisations.
In the second example, a large amount of noise makes it difficult to identify if any signal exists – by using a larger sample the signal is clearer.
Also useful for: detecting changes in the characteristics of variability
Years of Simulation

37. Which kinds of uncertainty can we explore using ensembles?
Uncertainty Source
Commonly represented in climate scenarios?
Ways to incorporate in climate scenarios
Scenario Uncertainty: Atmospheric emissions concentrations of GHGs
Yes
CMIP3: Emissions scenarios
CMIP5: RCPs
Model uncertainty:
Global and large-scale regional climate response to forcing
Use multiple members of GCM ensemble
IC uncertainty:
Natural variability
Sometimes
Use multiple realisations to increase sample size
Downscaling uncertainty:
Adding high resolution detail (statistical or dynamical downscaling)
Use multiple approaches
at different resolutions
Explain that different sources of uncertainty are sampled in climate modelling, to differing degrees. Also, discuss that downscaling adds another level of uncertainty and PRECIS is only one model.