Assimilation of Observations and Bayesianity Mohamed Jardak and Olivier Talagrand Laboratoire de Météorologie Dynamique, École Normale Supérieure, Paris, France Fifth Meeting of THORPEX Workshop Group on Predictability and Dynamical Processes Reading University, UK 20 June 2012

Project PREVASSEMBLE Funded by French Agence Nationale pour la Recherche Duration : initially 3 years, extended to 4 (started 01/2009) ; Total funding : 780,000 euros Purpose: Study all aspects of ensemble methods (theory, practical implementation, validation, possible limitations, ….) for data assimilation and prediction in atmospheric sciences. Three partners 1. Institut Pierre-Simon Laplace pour les Sciences de l'Environnement Global, Paris (O. Talagrand, general coordinator of the project) 2. Institut de Recherche en Informatique et Systèmes Aléatoires, Rennes (F. Le Gland) 3. Météo-France, Toulouse (G. Desroziers) Positions funded : 4 post-doctoral positions (Paris, Toulouse), 1 scientist position (Paris), 1 graduate student position (Rennes) Contact:

Purpose of assimilation : reconstruct as accurately as possible the state of the atmospheric or oceanic flow, using all available appropriate information. The latter essentially consists of  The observations proper, which vary in nature, resolution and accuracy, and are distributed more or less regularly in space and time.  The physical laws governing the evolution of the flow, available in practice in the form of a discretized, and necessarily approximate, numerical model.  ‘Asymptotic’ properties of the flow, such as, e. g., geostrophic balance of middle latitudes. Although they basically are necessary consequences of the physical laws which govern the flow, these properties can usefully be explicitly introduced in the assimilation process.

Both observations and ‘model’ are affected with some uncertainty  uncertainty on the estimate. For some reason, uncertainty is conveniently described by probability distributions ( Jaynes, E. T., 2007, Probability Theory: The Logic of Science, Cambridge University Press ). Assimilation is a problem in bayesian estimation. Determine the conditional probability distribution for the state of the system, knowing everything we know (unambiguously defined if a prior probability distribution is defined; see Tarantola, 2005).

Present approaches to data assimilation  Obtain some ‘central’ estimate of the conditional probability distribution (expectation, mode, …), plus some estimate of the corresponding spread (standard deviations and a number of correlations).  Produce an ensemble of estimates which are meant to sample the conditional probability distribution (dimension N ≈ O( ) ).

A large part of presently existing algorithms for assimilation are still linear and gaussian, in the sense that they are empirical and heuristic extensions to weakly nonlinear and non-gaussian situations of algorithms which achieve bayesianity in linear and gaussian cases.

Two main classes of algorithms for ensemble assimilation exist at present - Ensemble Kalman filter, which is bayesian in its ‘forecast’ phase, but linear and gaussian in its updating phase. Abundantly used. - Particle filters, which are totally bayesian in their principle (P. J. van Leeuwen), but are still at research stage.

Data z =  x +  Then conditional posterior probability distribution P(x  z) = N [x a, P a ] with x a = (  T S -1  ) -1  T S -1 [z  ] P a = (  T S -1  ) -1 Ready recipe for producing sample of independent realizations of posterior probability distribution (resampling method) Perturb data vector additively according to error probability distribution N [0, S], and compute analysis x a for each perturbed data vector

Available data consist of - Background estimate at time 0 x 0 b = x 0 +  0 b  0 b  N [ , P 0 b ] - Observations at times k = 0, …, K y k = H k x k +  k  k  N [ , R k ] - Model (supposed to be exact) x k+1 = M k x k k = 0, …, K-1 Errors assumed to be unbiased and uncorrelated in time, H k and M k linear Then optimal state (mean of bayesian gaussian pdf) at initial time 0 minimizes objective function  0  S  J (  0 ) = (1/2) (x 0 b -  0 ) T [P 0 b ] -1 (x 0 b -  0 ) + (1/2)  k [y k - H k  k ] T R k -1 [y k - H k  k ] subject to  k+1 = M k  k,k = 0, …, K-1

Work done here. Apply ‘ensemble’ recipe described above to variational assimilation in nonlinear and non-gaussian cases, and look at what happens. Everything synthetic, Two one-dimensional toy models : Lorenz ’96 model and Kuramoto-Sivashinsky equation. Perfect model assumption

(Nonlinear) Lorenz’96 model

There is no (and there cannot be) a general objective test of bayesianity. We use here as a substitute the much weaker property of reliability. Reliability. Statistical consistency between predicted probability of occurrence and observed frequency of occurrence (it rains 40% of the time in circumstances when I predict 40%-probability for rain). Observed frequency of occurrence p‘(p) of event, given that it has been predicted to occur with probability p, must be equal to p. For any p, p‘(p) = p More generally, frequency distribution F‘(F) of reality, given that probability distribution F has been predicted for the state of the system, must be equal to F. Reliability can be objectively assessed, provided a large enough sample of realizations of the estimation process is available.

Reliability diagramme, NCEP, event T 850 > T c - 4C, 2-day range, Northern Atlantic Ocean, December February 1999

Rank histograms, T 850, Northern Atlantic, winter Top panels: ECMWF, bottom panels: NMC (from Candille, Doctoral Dissertation, 2003)

In both linear and nonlinear cases, size of ensembles N e =30 Number of realizations of the process M =3000 and 7000 for Lorenz and Kuramoto-Sivashinsky respectively.

Non-gaussianity of the error, if model is kept linear, has no significant impact on scores.

Brier Score (Brier, 1950), relative to binary event E B  E[(p - p o ) 2 ] where p is predicted probability of occurrence, p o = 1 or 0 depending on whether E has been observed to occur or not, and E denotes average over all realizations of the prediction system. Decomposes into B = E[(p-p’) 2 ] - E[(p’-p c ) 2 ] + p c (1-p c ) where p c  E(p o ) = E(p’) is observed frequency of occurrence of E. First term E[(p-p’) 2 ] measures reliability. Second term E[(p’-p c ) 2 ] measures dispersion of a posteriori calibrated probabilities p’. The larger that dispersion, the more discriminating, or resolving, and the more useful, the prediction system. That term measures the resolution of the system. Third term, called uncertainty term, depends only on event E, not on performance of prediction system.

Brier Skill Score B SS  1 - B/ p c (1-p c ) (positively oriented) and components B rel  E[(p-p’) 2 ] / p c (1-p c ) B res  1 - E[(p’-p c ) 2 ] / p c (1-p c ) (negatively oriented)

Very similar results obtained with the Kuramoto-Sivashinsky equation. u t + uu x + u xx + u xxxx = 0

Preliminary conclusions In the linear case, ensemble variational assimilation produces, in agreement with theory, reliable estimates of the state of the system. Nonlinearity significantly degrades reliability (and therefore bayesianity) of variational assimilation ensembles. Resolution, i. e., capability of ensembles to reliably estimate a broad range of different probabilities of occurrence, is also degraded.

Perspectives Perform further studies on physically more realistic dynamical models (shallow-water equations) Further comparison with performance Particle Filters, and with performance of Ensemble Kalman Filter.

Announcement International Conference on Ensemble Methods in Geophysical Sciences Supported by World Meteorological Organization, Météo-France, ANR, CNRS, INRIA Dates and location : November 2012, Toulouse, France Forum for study of all aspects of ensemble methods for estimation of state of geophysical systems (atmosphere, ocean, solid earth, …), in particular in assimilation of observations and prediction : theory, algorithmic implementation, practical applications, evaluation, … Deadline for submission of abstracts : 20 June 2012