Theme D: Model Processes, Errors and Inadequacies Mat Collins, College of Engineering, Mathematics and Physical Sciences, University of Exeter and Met Office Hadley Centre

© Crown copyright Met Office

A Much Used Quote “All models are wrong, some are useful” - George Box Easy to test when a model is wrong, much harder to say when it is useful climate models

Uncertainties and Errors in Climate Models Uncertainties in parameters, sampled using Perturbed Physics Ensembles (PPEs) “Structural” uncertainties, at least partially sampled by Multi-Model Ensembles (MMEs) Coding errors! Errors common to all models (examples follow) Missing processes

Why do Errors Matter Predictions and projections Detection & attribution Understanding climate and climate change Risk assessment for natural events (cat modelling) …

Types of Errors & Inadequacies Incapabilities

Iconic Error 1: Blocking © Crown copyright Met Office Tim Hinton, Gill Martin

Iconic Error 2: Double ITCZ Source: IPCC Fourth Assessment Report observations models - MME

Boé J, Hall A, Qu X (2009), September Sea-Ice Cover in the Arctic Ocean Projected to Vanish by 2100, Nature Geosci, 2: Hall A, Qu X (2006) Using the current seasonal cycle to constrain snow albedo feedback in future climate change. Geophys. Res. Lett., 33, L03502 Errors known to be important for e.g. climate projections

Poorly Observed but Important Variables Source: IPCC Fourth Assessment Report Which model is right?

Known Missing Processes Land use change (in HadCM3Q) Soot/black carbon (in HadCM3Q) … Dynamic ice sheets Methane hydrate release Stratosphere …

Unknown Missing Processes

Dealing with Model Errors and Inadequacies

Do nothing Do nothing but at least discuss the implications of doing nothing Embark on an ambitious programme of metrics and intercomparisons Use a discrepancy term … (Continue to improve models, of course)

Metrics, metrics, metrics MME Error Characteristics Reichler and Kim, 2008 Relative model errors

HadGEM Traffic Lights © Crown copyright Met Office

Introduce a Discrepancy e.g. Rougier (2007) y = {y h,y f } climate variables (vector) f = Climate model e.g. HadCM3 x* = best point in HadCM3 parameter space – for observable and non-observable fields d = discrepancy – irreducible/”structural” model error (vector) How to determine d ?

Estimating Discrepancy in UKCP09 (David Sexton Talk) Use the multi-model ensemble from IPCC AR4 (CMIP3) and CFMIP (models from different centres) For each multi-model ensemble member, find point in HadCM3 parameter space that is closest to that member There is a distance between climates of this multi-model ensemble member and this point in parameter space i.e. effect of processes not explored by perturbed physics ensemble Pool these distances over all multi-model ensemble members Uses model data from the past and the future © Crown copyright Met Office

Scottish Snow Clear sky SW TOA flux over HadCM3 Scotland grid point Most extreme discrepancy found © Crown copyright Met Office

Theme D: Questions/Topics for Discussion Develop language/classification for errors and inadequacies Explore ways of specifying discrepancy Explore alternative strategies for dealing with errors and inadequacies Any more?

Blocking The frequency of blocking events in the perturbed physics HadCM3 ensemble (PPE_A1B, red lines) for winter (DJF, top) and summer (JJA, bottom) together with that estimated from ERA40 (thick black lines). The blocking index is calculated following Pelly and Hoskins (2003) and uses a variable latitude to track the location of the model storm track (in contrast to other indices which used a fixed latitude). Murphy et al UKCP09

Types of Errors and Inadequacies Incapabilities – e.g. due to resolution Known errors/biases in simulating mean climate and variability, leading to metrics, iconic errors Errors which are known to be important for projection, D&A, … Potential errors in poorly observed variables Known missing processes Unknown missing processes

Systematic Errors in All Models Collins et al. 2010