1 Isaac Newton Workshop on Probabilistic Climate Prediction University of Exeter Sep 2010 Professor David B. Stephenson Exeter Climate Systems Mathematics Research Institute

2 Aims of the workshop No universal agreement exists on what constitutes a reliable and robust framework for inferring and evaluating predictions of real-world climate. This workshop aims - to debate the strengths and weaknesses of existing frameworks for inferring and evaluating predictions of real-world climate -identify and formulate pressing and potentially solvable problems in the mathematics of probabilistic climate prediction. Workshop style: Interactive, constructive, stimulating. Overview talks on the main issues and existing methods will set the scene and then these will be followed by small thematic breakout working groups. Working groups will reconvene to share ideas.

3 How should we use multi-model simulations...

4... to make inference about future observables? Projections Observations Uncertainty around projections scenarios

5 Forecast Assimilation Data Assimilation Forecast Assimilation Stephenson, D.B., Coelho, C.A.S., Balmaseda, M. and Doblas-Reyes, F.J. (2005): Forecast Assimilation: A unified framework for the combination of multi-model weather and climate predictions, Tellus A, 57 (3), pp

6 The Multi-Model Ensemble A “fruit bowl of opportunity” {X 1,X 2,...,X m } Note: Not a random sample from one homogeneous population (and it does not include all possible fruit!)

7 What does reality look like? actual true climate Y – inferred from observations Z It could not have been drawn out of my fruit bowl! How can we infer properties of this from the fruit in the fruitbowl? An inconvenient truth

8 Smoothies (multi-model means) A smoothie is a weighted average of fruits. It is not an item of real fruit! (important information has been lost by averaging) Non-unique choice of weights for making smoothies.  We require modelling frameworks for obtaining samples of real fruit from the posterior distribution p(Y|X) (not smoothies E(X) or E(X|Y)).

9 Should we use everything in the fruit bowl? Should we select subsets? How should we weight the fruits? “All fruit are wrong, but some are tasty” - Granny Smith

10 Homogeous samples How to relate Y to X? Are the {X i } independent draws from a distribution centred on Y? Are the {X i } second-order exchangeable with each other and Y? How best to model model discrepancy Y-X i ?

11 Breakout themes Theme A: Frameworks for quantifying uncertainty (Chair: Jonty Rougier in LT1) What frameworks are available for quantifying uncertainty in climate predictions (e.g. Bayesian probability, maximum likelihood, interval probabilities, random/fuzzy, etc.) and what are their respective strengths and limitations? Theme B: Calibration of climate predictions (Chair: David Stephenson in room D) What grounds do we have for believing that our predictions are well-calibrated and how should we go about calibrating outputs from climate models (e.g. bias correction of extremes)? Theme C: Evaluation of climate predictions (Chair: Chris Ferro in room E) How should we go about evaluating probabilistic climate predictions? How can we best determine the skill and reliability of probabilistic climate predictions at various space and time scales? Theme D: Model processes and inadequacies (Chair: Mat Collins in room F) How should we represent and quantify known limitations in physical processes simulated by climate models? i.e. the ability to simulate long-lasting blocking events, the correct intensity of extreme storm events, etc. Theme E: Problem area to be decided (Chair: Richard Chandler in Margaret Room) There are many other pressing and interesting questions in probabilistic climate prediction that require statistical research. Each theme should have a rapporteur who will report back to the main sessions and record ideas on the wiki page (user=newton pass=apple). Participants can move around themes.

12 Theme B: Calibration strategies John Ho, Mat Collins, Simon Brown, Chris Ferro How to infer distribution of Y’ from distributions of Y, X and X’? 1. No calibration Assume Y’ and X’ have identical distributions (i.e. no model biases!) i.e. F Y’ = F X’ 2. Bias correction Assume Y’=B(X’) where B(.)=F Y -1 (F X (.)) 3. Change factor Assume Y’=C(Y) where C(.)=F X’ -1 (F X (.)) 4. Other e.g. Adjust parameters in parametric fits e.g. X Y Y’=? X’ Y’=B(X’) Y’=C(Y)

13 Example: daily summer temperatures in London Daily mean air temperatures Y=Observations from E- OBS gridded dataset (Haylock et al., 2008) X=HadRM3 standard run (25 km resolution) forced by HadCM3; SRES A1B scenario. n=30*120=3600 days Black line = sample mean Red line = 99th percentile

14 Probability density functions Black line = pdf of obs data Blue line = pdf of climate data Red line = pdf of climate data

15 Linear calibration (constant shape)  Two approaches give different future mean temperatures!

16 Change in 10-summer level from No calibration T g’ - T o Bias correctionChange factor  Substantial differences between different estimates!

17

18 Breakout themes Theme A: Frameworks for quantifying uncertainty (Chair: Jonty Rougier in LT1) What frameworks are available for quantifying uncertainty in climate predictions (e.g. Bayesian probability, maximum likelihood, interval probabilities, random/fuzzy, etc.) and what are their respective strengths and limitations? Theme B: Calibration of climate predictions (Chair: David Stephenson in room D) What grounds do we have for believing that our predictions are well-calibrated and how should we go about calibrating outputs from climate models (e.g. bias correction of extremes)? Theme C: Evaluation of climate predictions (Chair: Chris Ferro in room E) How should we go about evaluating probabilistic climate predictions? How can we best determine the skill and reliability of probabilistic climate predictions at various space and time scales? Theme D: Model processes and inadequacies (Chair: Mat Collins in room F) How should we represent and quantify known limitations in physical processes simulated by climate models? i.e. the ability to simulate long-lasting blocking events, the correct intensity of extreme storm events, etc. Theme E: Problem area to be decided (Chair: Richard Chandler in Margaret Room) There are many other pressing and interesting questions in probabilistic climate prediction that require statistical research. Each theme should have a rapporteur who will report back to the main sessions and record ideas on the wiki page (user=newton pass=apple). Participants can move around themes.

19 Ideas for Theme E? Theme E: Emergent problem area to be decided (Chair: Richard Chandler) There are many other pressing and interesting questions in probabilistic climate prediction that require statistical research. Some such questions include: What is the purpose of introducing stochastic components into climate simulators? How can we determine what data (i.e. model simulations, observed data, paleoclimate data) are required to inform the development of improved climate projections? How should we best account for shared sources of uncertainty across a multi- model ensemble? What formal role is there for simpler simulators? Do we believe the ergodic assumptions that are implicitly made in climate science? How best to visualise probability forecasts? Others???

20 Tuesday 21 September: Overview talks and small group brainstorming 09:30-10:15 “Outstanding problems in probabilistic prediction of climate” David Stephenson 10:15-11:00 “Model inadequacies and physical processes” Mat Collins 11:00-11:30 Morning coffee/tea 11:30-13:00 Small group breakout sessions on the 5 main themes 13:00-14:00 Photo and Buffet lunch 14:00-14:45 “Methodologies for probabilistic uncertainty assessment” Richard Chandler 14:45-15:30 “Probabilistic methodology used for UKCIP” David Sexton 15:30-16:00 Afternoon tea/coffee 16:00-16:15 "Probabilistic use of climate catastrophe multi-models" Gero Michel 16:15-17:30 Small group breakout sessions 17:30-19:00 Drinks reception in Holland Hall bar (kindly sponsored by Willis) 19:00- Participants go for dinner at restaurants in Exeter Wednesday 22 September: Overview talks and small group brainstorming 08:00-09:00 Breakfast in Holland Hall (for those staying there) Talks and discussions in the Queen’s building: 09:30-10:15 “Non-probabilistic frameworks” Arthur Dempster 10:15-11:00 “Probabilistic frameworks” Jonty Rougier 11:00-11:30 Morning coffee/tea 11:30-13:00 Small group breakout sessions on the 5 main themes 13:00-14:00 Buffet lunch 14:00-15:30 Small group breakout sessions 15:30-16:00 Afternoon tea/coffee 16:00-17:30 Small group breakout sessions 17:30-19:00 Drinks reception in Holland Hall bar (kindly sponsored by Willis) 19:00- Participants go for dinner at restaurants in Exeter Thursday 23 September: Plenary summary and departure 08:00-09:00 Breakfast in Holland Hall (for those staying there) Summary discussions in the Queen’s building 09:30-10:30 Summaries of breakout sessions (rapporteurs) 10:30-11:00 Final discussion and future plans 11:00-11:30 Morning coffee/tea