Parameter estimation: To what extent can data assimilation techniques correctly uncover stochasticity? Jim Hansen MIT, EAPS (with lots of help from Cecile Penland and Greg Lawson)

Indistinguishable?

Accounting for vs. reducing model inadequacy Accounting for model inadequacy –“If you can show me how I can make better forecasts using chicken bones and voodoo dolls, then I’m going to use them!” »Harold Brooks, NSSL –initial conditions (Q-term) –forecasts (MM, stochastic, MOS, forecast 4d-Var) Reducing model inadequacy –Making changes to our model so that it becomes a better representation of the true system –parametric error –structural error

Accounting for vs. reducing model inadequacy Accounting for model inadequacy –“If you can show me how I can make better forecasts using chicken bones and voodoo dolls, then I’m going to use them!” »Harold Brooks, NSSL –initial conditions (Q-term) –forecasts (MM, stochastic, MOS, forecast 4d-Var) Reducing model inadequacy –Making changes to our model so that it becomes a better representation of the true system –parametric error –structural error

Reducing model inadequacy Reducing model inadequacy is best framed as an off-line, or “reanalysis” activity –The process of attempting to identify model inadequacy tends to make both initial conditions and forecasts worse –The aim is to quantify how the model is wrong, fix it, and then worry about data assimilation and forecasting

A proposed approach Use data assimilation tools to alter model parameters to better fit observations Identify relationships between fit parameters and prognostic variables (a parametric MOS) Change model to reflect relationships Repeat When all relationships have been uncovered, the history of fit parameter values provides a distribution from which to (carefully) draw for the purpose of stochastic parameterizations

Use data assimilation tools to alter model parameters to better fit observations Augment control vector with unknown parameters Augmentation removes the nonlinearity from the observation operator and inserts it into the specification of the control vector

Augmented control vector sample covariance

Parametric error example: L63 System equationsModel equations

Importance of a state-dependent background error covariance 4d-Var, static covarianceEnsemble 4d-Var time parameter

Structural error example: Lorenz ‘96 System: Model:

x(1) parameter Regressing parameter vs. prognostic variable gives:

Alter model equations with new information System: Original model: New model:

Example: Lorenz ’96 Model II

System: Model:

parameter x(1) Regressing parameter vs. prognostic variable gives:

System: Original model: New model:

SDE crash course The type of calculus used to integrate the stochastic bits of SDEs matters –Stratonovich calculus noise process is continuous (typical assumption for geophysical fluid flows) –Ito calculus noise process is discrete (like DA!) SDEs can be tricky (and expensive) to integrate –used stochastic RK4 (Hansen and Penland, 2005)

What if the system really is stochastic? System is an SDEModel is an ODE

Can DA uncover the correct form of the stochasticity? - NO Ensemble 4d-VarEnKF parameter time

Why can’t DA uncover the correct form of the stochasticity? Stochasticity operating at different time-scales –SDE has infinitesimal time-scale –ODE with DA has 6-hourly time-scale System is using Stratonovich calculus, DA is using Ito calculus All leads to a danger of misinterpretation Model error!

1.Deterministic model using constant, tuned parameter value ( ) 2.Stochastic model using mean and standard deviation of tuned parameter value ( ) 3.Deterministic, multi-model ensemble with parameters drawn from ( ) 4.Deterministic model where parameter varies in the same manner as it was estimated ( ) How should we use this information for forecasting? Tuned deterministic

1.Deterministic model using constant, tuned parameter value ( ) 2.Stochastic model using mean and standard deviation of tuned parameter value ( ) 3.Deterministic, multi-model ensemble with parameters drawn from ( ) 4.Deterministic model where parameter varies in the same manner as it was estimated ( ) How should we use this information for forecasting? Incorrect SDE

1.Deterministic model using constant, tuned parameter value ( ) 2.Stochastic model using mean and standard deviation of tuned parameter value ( ) 3.Deterministic, multi-model ensemble with parameters drawn from ( ) 4.Deterministic model where parameter varies in the same manner as it was estimated ( ) How should we use this information for forecasting? Multi-model where the draw from is held constant over the entire forecast period.

1.Deterministic model using constant, tuned parameter value ( ) 2.Stochastic model using mean and standard deviation of tuned parameter value ( ) 3.Deterministic, multi-model ensemble with parameters drawn from ( ) 4.Deterministic model where parameter varies in the same manner as it was estimated ( ) How should we use this information for forecasting? Hybrid where the draw from is made every 6 model hours.

Median of ensemble mean forecast distributions Tuned deterministic Incorrect SDE Multi-model Hybrid Perfect Normalized RMSE Forecast lead (model days)

std(err/ens_std) Must assess probabilistically! Tuned deterministic Incorrect SDE Multi-model Hybrid Perfect Forecast lead (model days)

relative entropy Relative (to perfect) entropy Multi-model Hybrid Forecast lead (model days)

What if we use a stochastic model? System is an SDEModel is an SDE

Now can DA uncover the correct form of the stochasticity? - NO parameter std time parameter mean time

What’s the problem this time? Wrong trajectory of random numbers Model error!

SDE forecast errors

What does it all mean? Deterministic model DA approaches alone are not enough to uncover the correct form of stochasticity –Implies that we cannot attach physical significance to tuned parameter values or distributions Our efforts to reduce model inadequacy ultimately lead to a sensible way to account for model inadequacy Synoptic time-scale, Ito-like stochasticity via parameter estimation does a great job accounting for model inadequacy during forecasting

The future(?) of data assimilation Model error issues Nonlinearity New disciplines: e.g. paleo, climate Improved image DA is part of a larger problem

The future(?) of data assimilation Nonlinearity –Implementing nonlinear approaches –Extend minimum error variance approaches a bit more into the nonlinear regime Feature-based non-Gaussianity

The future(?) of data assimilation Improved image –DA has a bad/boring reputation –Ensemble methods bringing DA to the masses University research can be quasi-operational Reasonable DA now where none before

ATMOS

COLLEGE

The future(?) of data assimilation Improved image –DA has a bad/boring reputation –Ensemble methods bringing DA to the masses University research can be quasi-operational Reasonable DA now where none before

The future(?) of data assimilation DA is part of a larger problem –The future of DA is not independent of the future of observations, ensemble forecasting, verification, calibration, etc.. Ensemble forecasting Targeting Increasing ensemble forecast size at low cost Ensemble synoptic analysis

Transformed Lag Ensemble Forecasting (TLEF) Ensemble size is increased by using ensemble-based data assimilation techniques to transform (scale and rotate) old forecasts using new observations. Time 

The future(?) of data assimilation DA is part of a larger problem –The future of DA is not independent of the future of observations, ensemble forecasting, verification, calibration, etc.. Ensemble forecasting Targeting Increasing ensemble size at low cost Ensemble synoptic analysis

Hakim and Torn WRF, 100 ensemble members, surface pressure obs

Hakim and Torn

Ensembles make PV inversion fun and easy! Note, no worries about balance assumptions or boundary conditions Approach defined by Hakim and Torn

The future(?) of data assimilation Model error issues Nonlinearity New disciplines: e.g. paleo, climate Sales/Marketing DA is part of a larger problem