Data Assimilation

Data assimilation in different tongues “Data assimilation” - GFD “State estimation” - nonlinear dynamics “Inverse modeling” - geophysics et al. “Signal processing” – engineering “Chaos synchronization” – physics ….Well, the main reason for that is that data assimilation is a body of techniques that many fields have relied upon and developed on their own in some fashion. In fact, the term “data assimilation” itself is largely synonymous with other terms, though practitioners in other fields will sometimes try to distinguish their meaning. Don’t be fooled. For all intents and purposes, DA is equivalent to “state estimation”, “inverse modeling”, and in some instances, “signal processing.” At root, one finds that all fields are essentially blending and combining multiple sources of information to get a “best estimate” or an “analysis.” DA is generally credited with Gauss’s invention of least squares in the early 1800s, and ever since then, all such uses and instantiations of DA have generally been in accordance with some sound theoretical underpinning…. Though it is often seen as such from the outside (often as a consequence of its complicated reputation), there is no “magic” in DA! When people use DA today, they are generally doing some form of regression at some point in their process. Part of the confusion is that this regression viewpoint can be arrived at from several different starting points. At root, it is blending/combining multiple sources of information to get a “best estimate.”

Goals of data assimilation Keep a numerical model “close” to a set of observations over time Provide appropriate initial conditions for a forecast Provide an estimate of analysis errors Propagate information from observations to unobserved locations Tell us something about how the model behaves

Review: main components Relating a gridded model state to observations Introducing them over some number of times (could be as few as one) (Initialization step)

Single-observation examples Single-time updates: Cressman Statistical Weather is 4D Sequential Continuous

Weighted-average of nearby observations based on the distance First steps: Cressman Weighted-average of nearby observations based on the distance squared

Why not Cressman? If we have a preliminary estimate of the analysis with a good quality, we do not want to replace it by values provided from poor quality observations.  When going away from an observation, it is not clear how to relax the analysis toward the arbitrary state, i.e. how to decide on the shape of the function.  An analysis should respect some basic known properties of the true system, like smoothness of the fields, or relationship between the variables (e.g. hydrostatic balance). This is not guaranteed by the Cressman method: random observation errors could generate unphysical features in the analysis. No background field!

A practical route to data assimilation…statistics What temperature is it here? To consider the more popular route to data assimilation, consider the very practical question of “what temperature is it here?” Suppose you want to perform a temperature-sensitive event, and you want to know if it is actually, say, 60 degrees F or 62. You have your Mom’s beautiful, antique Galilean thermometer, and that gives you one reading, but you know it has some resolution (i.e., accuracy) issues. You invite your friend over to help measure the temperature, and he brings over his outdoor thermometer, which you have to read the number off the dial, again with resolution issues. Also, of course, no measurement is perfect, and both sources are expected to have (random) instrumental errors involved….. So, what temperature is it? Both measurements are sources of information, so how do we come up with a best estimate of the temperature that utilizes the information content in both estimates?

Form a linear combination of estimates ? With two different estimates of the same quantity, we might think it reasonable to form a linear combination of them to come up with our best estimate of the temperature, here denoted xa for “analysis,” a specialized term in data assimilation for “best estimate.” Having written our analysis like this, it remains only to find alpha -- so that we can determine whether our analysis is like this or like this. We can write it as a linear combination in the “canonical” fashion, or with some foresight, we can alternately write it in a “correction” fashion, where we are correcting the “initial guess” of x2 by the information content within x1.

Include the background as one “obs” Generalize to many observations, including the background (first guess)

Statistical assimilation Observations have errors The forecast has errors The truth xt is unknown

Statistical assimilation Best linear estimate: combination between the background xf at observation locations and the observations yo themselves Think in terms of averages We do not know the truth, so we look for the maximum likelihood estimate, or the minimum-variance estimate

Statistical assimilation We don’t know truth, so we can’t know the errors. We have at least a chance of estimating the error variances Making these estimates is the heart of statistical data assimilation

Statistical assimilation The goal can be restated: find the best α that minimizes the analysis error on average The analysis is a combination between the observation and the forecast at the observation locations

Statistical assimilation It turns out that the best estimate is achieved when: Observation increment Analysis increment

RTFDDA How does it relate to statistical assimilation?

G includes it all Distance to the obs Time from the obs Expected obs error Quality control

Obs-nudging: Weighting Functions G = Wqf Whorizontal Wvertical Wtime

The fourth dimension: time Sequential assimilation Continuous assimilation Putting it all together

Variational methods 3DVAR 4DVAR Very complex, maybe not better.

Sequential assimilation This is RTFDDA!

Complexity