1. 2015 CIRA Training in Data Assimilation
Review on Data Assimilation and Introduction to Gridpoint Statistical Interpolation (GSI) Part I
Ting-Chi Wu

2. Outline Review on Data Assimilation Introduction to GSI Running GSI…
What? Why? How?
3DVar
4DVar and concept of Ensemble DA
Introduction to GSI
History and Background
Theory (Equation and Solution)
Additional Options
Observations
GSI code structure
Running GSI…
Today
Next Class

3. What is Data Assimilation?
Combines different sources of information to provide an optimal estimate of the state of a system:
Model
Observation, data
Background, a prior information
Statistics
Its objective is to produce a regular and physically consistent four dimensional representation of the state of a system from a heterogeneous array of in-situ and remote instruments which sample imperfectly and irregularly in space and time.

4. What is data assimilation for?
Initial conditions for predictions
Reanalysis (using past observation)
Calibration and validation
Observing system design (OSSE), monitoring and assessment
Better understanding model errors, data errors, physical process interactions, parameters, etc
Data Assimilation methods:
Successive corrections methods
Nudging methods
Optimal Interpolation (OI)
3DVar, 4DVar
Extended Kalman Filter, Ensemble KF
Hybrid Ens-Var
Empirical method
Constant statistical method
Adaptive statistical method

5. Review on 3DVar (1/2) Jb = Fit to background Jo + Fit to observationJc
+ Constraint terms
Jvar: cost function
x: analysis vector
xb: background vector
Bvar: background error covariance matrix (estimated offline)
h: observation operator
E+F = R: Instrument error + representative error
= observation error covariance (usually diagonal)
y: observation vector
Jc: constraint terms
Gaussian probability density function
Least-square (quadratic form): the most familiar minimization problem

6. Review on 3DVar (2/2) Jb Jo Jc
= Fit to background + Fit to observation + Constraint terms
Optimal xa is obtained by minimizing the cost function
Direct calculation of Jb and Jo terms for NWP problems (given that B and R are matrices of dimension 108) is almost impossible. Therefore, many practical simplifications are required
Incremental Var : (will discuss in intro to GSI)

7. Jb term – Bvar Background error covariance, B: Static B (or Bvar):
Contains multivariate (cross-variable) information
Observation distance of influence
Acts like a weighting to the correction to background
A huge matrix
x1, x2, and x3 can be different control variables (T, P, Q, U, V, etc) in 3D space (i, j, k).
Static B (or Bvar):
Computed offline (pre-estimated)
Does not evolve with the system
Hard to model cross-variable covariance with static B

8. Jo term – h
To assimilate observation, the model needs to be able to simulate the observation! Mapping the initial model state to the actual observation at given points in space and time.
Mapping can be as simple as spatial/temporal interpolation, or it can be as complicated as running a radiative transfer model.
Depends on the observation, h can be linear/nonlinear.
To introduce new observation, one needs to come up with a new operator h (and its tangent linear and adjoint (H & HT) for incremental Var).

9. Or Jo in this form where M is forecast model:
Review on 4DVar Jo
Over a time window !
Or Jo in this form where M is forecast model:
ECMWF
Model dynamics is the strong constraint
Minimization of the cost function involves multiple forward and backward model integration (M and MT)
Finds model trajectory that best fits observations over a time window.
where the MT is adjoint of model M
and innovation vector di :

10. Concept of Ensemble DA
The B: use ensemble forecasts to estimate flow-dependent uncertainty/error of the background forecast.
More accurate estimate, less error in the analysis
Improved initial conditions, improved ensemble forecasts.
Single-observation 1K greater than the first guess is assimilated.
The analysis increments are proportional to the background error covariances between model grid points and the observation location.

11. GSI History and Background (1/2)
Optimal Interpolation (OI) was the first statistic data analysis system.
The Spectral Statistical Interpolation (SSI), developed at NOAA NCEP in late 1980 – early 1990.
the first operational variational analysis system
directly use satellite radiance
SSI was a great improvement over the OI, but…
Spectral space made it less straightforward to use for regional systems
Spectral background error did not allow much spatial variation in the background
Was a prototype code to begin with; not well written for operational purpose

12. GSI History and Background (2/2)
Based on SSI, the Gridpoint Statistical Interpolation (GSI) is formulated in physical space.
Allows greater flexibility to background error statistics/covariance (geographically inhomogeneous and anisotropic)
Uses multiple recursive filters in physical space to construct latitude-dependent structure functions.
Straightforward to apply to regional domain.
GSI analysis code is an evolving system.
Monthly Weather Review

13. GSI Today (1/2) GSI system has become operational as the core of
NDAS (North American Data Assimilation)
NAM (North America Mesoscale)
GDAS (Global Data Assimilation for the GFS)
NASA Goddard Earth Observing System (GEOS)
Air Force Weather Agency (AFWA) mesoscale DA
NOAA Real-Time Mesoscale Analysis (RTMA)
NOAA Hurricane WRF (HWRF)
NOAA RAPid Refresh (RAP)
Groups involved in operational GSI development:
NASA GMAO / NOAA ESRL / NCAR MMM
Development Testbed Center (DTC) maintains the community GSI repository:
NASA Global Modeling and Assimilation Office
NOAA Earth System Research Laboratory
NCAR Mesoscale & Microscale Meteorology Laboratory

14. GSI Today (2/2)
Originally developed as 3DVar DA system; been evolving into Ensemble-Variational hybrid system in recent years.
Many options available or under development:
4DVar
Hybrid Assimilation
FGAT (First Guess at Appropriate Time)
FOTO (First Order Interpolation to Observations)
Observation sensitivity study
Additional observation types
SST retrieval
Focus on the standard operational 3DVar used in HWRF

15. GSI Theory Jb Jo Jc
= Fit to background + Fit to observation + Constraint terms
Analysis variables (background errors are defined)
Streamfunction (ψ)
Unbalanced velocity potential (χ)
Unbalanced temperature (T)
Unbalanced surface pressure (Ps)
Pseudo relative humidity or normalized relative humidity
Satellite bias correction coefficients
Ozone (global GSI only)
Cloud condensate mixing ratio (global GSI only)
Trace gases / aerosols / chemistry (for chemical DA)
Gust and visibility (for RTMA)

16. GSI Theory – Equations (1/2)
Assume linearity of the observation operator h, Incremental Var :
where
Define Δx = x – xb, analysis increment and o = y –h(xb), observation innovation.

17. Precondition* Physical space Preconditioned space
An ideal preconditioning can change the cost function so that the minimization is reached in less minimization iterations
Provide a balanced reduction of cost function; a change of all control variables that is in agreement with actual weather situation.

18. GSI Theory – Equations (2/2)
To improve convergence, GSI preconditions* Jvar by defining a new variable p = B-1Δx (or Bp = Δx)
Adjoing of H
Solve for p using conjugate gradient
Start from p = 0 and iterate over n:
where α is step size and
are descent directions

19. GSI Theory – Solution Strategy
Optimal xa is obtained by minimizing the cost function with respect to the control variables:
xa = Δxouter_iteration + Δxinner_inneration + xb
In outer iteration:
Quality control
More complete forward operator h
In inner iteration (can speed up by using coarser resolution):
Preconditioned conjugate gradient minimization
Simpler forward operator H
Variational quality control*
Solution used to start the next outer iteration
Remember: o = y –h(xb)
An ideal preconditioning can change the cost function so that the minimization is reached in less minimization iterations
Provide a balanced reduction of cost function; a change of all control variables that is in agreement with actual weather situation.
Operational HWRF-GSI: use 2 outer loops and 50 inner loops.

20. (need to have vis/gust/PBL height as control variable)
GSI Theory – Jc term
Jc terms are penalties for
Negative humidity constraint:
Excess moisture constraint:
where λ is weighting factor for negative moisture constraint
and λ2 is weighting factor for supersaturated moisture constraint
Negative visibility constraint
Negative gust constraint
Negative PBL height constraint
Conservation of global dry mass
(need to have vis/gust/PBL height as control variable)
(not applicable to regional problem)

21. Reference: Useful links: The materials of this presentation come from:
GSI Community Version 3.3 Advanced User’s Guide. Aug 2014
GSI Community Version 3.3 User’s Guide. June 2014
Derber J. C.: Overview of GSI Hurricane WRF Tutorial
Shao H. and M. Hu: Introduction to Data Assimilation and Community Gridpoint Statistical Interpolation System (GSI) Hurricane WRF Tutorial
Liu Z.: Hybrid Variational/Ensemble Data Assimilation WRFDA tutorial
Whitaker J.: GSI Hybrid EnVar Data Assimilation GSI Tutorial
Useful links:

22. Get ready for Python https://www.python.org/

23. Codecademy http://www.codecademy.com/en/tracks/python
ipython: Anaconda's Python distribution
Choose your platform
required

24. After successful installation…
Launcher
Terminal
GUI