1 Optimal solution error covariances in nonlinear problems of variational data assimilation Victor Shutyaev Institute of Numerical Mathematics, Russian Academy of Science, Moscow Igor Gejadze Department of Civil Engineering, University of Strathclyde, Glasgow, UK F.-X. Le Dimet, LJK, University of Grenoble, France

2 Problem statement Model of evolution process: Objective function (for the initial value control): Control problem: - nonlinear differential operator Unknown initial condition (analysis) Background Observations Inverse of the background covariance matrix Observation operator Inverse of the observation covariance matrix Optimal solution (analysis) error: True state

3 Optimal solution error via errors in input data In the nonlinear case the optimal solution error and input data errors are related via the nonlinear operator equation [1]: Background errorObservation error What interpretation for being of random nature? For example, for each sensor the observation error is a random time series. For the background error it can be seen as an error in expert guesses. Variational DA: Tikhonov regularization: Estimates in variational DA and Tikhonov’s method have different statistical properties: in particular, Tikhonov’s estimates are not consistent (biased)

4 Statistical properties of the optimal solution error Covariance matrix If we consider as random errors, then optimal solution error is a random error. Moreover we assume that it is subjected to the multivariate normal distribution and can be quantified by: Expectation What are reasons to believe that? Some classics from nonlinear regression: Estimate is consistent and asymptotically normal if is i.i.d. with and has certain regular properties, Gennrich (1969). This result follows from strong law of large numbers. Extended to multi-variate case and for certain classes of dependent observations, Amemiya, (1984). In reality the number of observation is always finite, thus the concept of ‘close-to-linear’ statistical behaviour, Ratkowski (1984). Are the results above valid for complete error equation ? It requires that and it must be normally distributed; A difficulty is that full equation might have many solutions; however, if among them we choose the one which corresponds to the global minimum of the cost functional, then we should also achieve consistency and asymptotic normality.

5 Covariance and the inverse Hessian Linear case and normal input data: H -Hessian, also: Fisher information matrix, grammian, … Nonlinear case and normal input data: With the following approximations one obtains The sufficient condition for these approximations to be valid is called the tangent linear hypothesis: This condition means that even though the dynamics is nonlinear, evolution of errors is well described by the tangent linear model. As most sufficient conditions the tangent linear hypothesis is overly restrictive. In practice, the above formula is valid if the linearization error is not cumulative in a probabilistic sense.

6 On errors 1. Consider function as the exact solution to the problem 3. End ensemble loop. 4. Compute the sample covariance 2. Start ensemble loop 2.1 Generate using Monte-Carlo 2.2 Compute 2.3 Solve the original nonlinear DA problem with perturbed data and find 2.4 Compute Fully nonlinear ensemble method (Monte Carlo) Two types of error are presented in the formula. Error due to approximations can be called the ‘linearization error’. However, the true state is not usually known (apart from the identical twin experiment setup) and one must use its best available estimate as the Hessian origin. Hence, another error called the ‘origin error’. This error cannot be eliminated, however its possible magnitude can be estimated.

7 Iterative methods for the inverse Hessian computation 1. Inverse Hessian by the Lanczos and Arnoldi methods The Lanczos and Arnoldi methods compute a set of leading Ritz values/vectors which approximate the eigen-pairs of the preconditioned Hessian using the Hessian-vector product: 2. Inverse Hessian by the BFGS method The BFGS forms the inverse Hessian in course of solving the auxiliary control problem: Iterative methods allow us to compute a limited-memory approximation of the inverse Hessian (at a limited computational cost) without a need to compute the Hessian matrix. These methods require efficient preconditioning (B).

8 Model (non-linear convection-diffusion): Example 1: Initialization problem Nonlinear diffusion coefficientField evolution and ensemble varianceand ensemble covariance

9 When the main result is not valid In a general nonlinear case one may not expect the inverse Hessian to always be a satisfactory approximation to the optimal solution error covariance. Model: 1D Burgers with strongly nonlinear dissipation term Field evolution: case A and case B and ensemble variance for initialization problem Case A: sensors atCase B: sensors at In Figures: inverse Hessian – solid line, ensemble estimate – marked line, background variance – dashed line

10 Effective Inverse Hessian method (EIH): main idea Exact nonlinear operator equation Exact optimal solution error covariance (by definition) Resulting from series of assumptions the equation above reduces to the form: I. Computing the expectation by Monte Carlo: l -th optimal solution II. Computing the expectation by definition: As we assume that - normal, then v is dummy argument ! Assumes nonlinear dynamics, but asymptotic normality and ‘close-to-linear’ statistical behavior (Ratkowski, 1983 )

11 EIH method: implementation Preconditioning 1-level preconditioning: 2-level preconditioning: Iterative process Monte Carlo (MC) for integration This integral is a matrix which can be presented in a compact form ! For integration instead of MC one can use quasi-MC or multi-pole method for faster convergence (smaller L) !

12 EIH method: example - 1 Relative error in the variance estimate by the ‘effective’ IH (asymptotic) and IH - reference covariance (sample covariance with large L=2500 ) Envelope for relative error in the sample variance estimate for L=25(black) and L=100 (white) - based on a set of optimal solutions - does not require optimal solutions Can be improved using ‘localization’, but requires optimal solutions! Envelope for by the ‘effective IH’, L=25(black) and L=100(red)

13 EIH method: example - 2 Relative error in the variance estimate by the ‘effective’ IH (asymptotic) and IH - reference covariance (sample covariance with large L and after ‘sampling error compensation’ procedure ) Envelope for relative error in the sample variance estimate for L=25(black) and L=100 (white) - based on a set of optimal solutions - does not require optimal solutions Can be improved using ‘localization’, but requires optimal solutions! Envelope for by the ‘effective IH’, L=25(black) and L=100(red)

14 EIH method: examples 1-2, correlation matrix Example 1 Example 2 Reference correlation matrix Error in the correlation matrix by IH method Error in the correlation matrix by EIH method

15 On a danger of the origin error Each is a likely optimal solution given. For each the likelihood region is defined by its covariance approximated by, which may significantly differ. Dependent on what optimal solution actually implemented (and considered as an origin), the covariance estimates may not approximate at all. Thus, solutions of such nonlinear systems cannot by verified in principle. Difference in mutual probabilities can be considered as an indicator of verifiability.

16 Conclusions For an exact origin: the inverse Hessian is expected to approximate well the optimal solution error covariance if the tangent linear hypothesis (TLH) is valid. In practice, this approximation can be sufficiently accurate even though the TLH breaks down; if the nonlinear DA problem tends to be at least asymptotically normal or (better) exhibits a ‘close-to-linear’ statistical behavior, then the optimal solution error covariance can be approximated by the ‘effective inverse Hessian’. For an approximate origin: the likely magnitude of the origin error can be revealed by a set of variance vectors generated around an optimal solution; based on this information the verifiability of the optimal solution can be analysed. the upper bound of the set can be chosen to achieve reliable (robust) state estimation In the linear case the optimal solution error covariance is equal to the inverse Hessian In the nonlinear case, one must distinguish the linearization error (originates from linearization of operators around the Hessian origin) and the origin error (originates from the difference between the best known and a true state) In an extremely nonlinear case the posterior covariance Does not represent the pDF (though locally!) reasonably nonlinear case

17 References Gejadze, I., Copeland, G.J.M., Le Dimet, F.-X., Shutyaev, V.P. Computation of the optimal solution errror covariance in variational data assimilation problems with nonlinear dynamics. J. Comp. Physics. (2011, in press) Gejadze, I., Le Dimet, F.-X., Shutyaev, V.P. On optimal solution error covariances in variational data assimilation problems. J. Comp. Physics. (2010), v.229, pp Gejadze, I., Le Dimet, F.-X., Shutyaev, V.P. On analysis error covariances in variational data assimilation. SIAM J. Sci. Comput. (2008), v.30, no.4,