1. NUMERICAL WEATHER PREDICTION (Data Assimilation Part) Dr Meral Demirtaş Turkish State Meteorological Service Weather Forecasting Department WMO, Training Course, 26-30 September 2011 Alanya, Turkey

2. Outline Introduction Basic concepts Observation resources Data assimilation techniques

3. Introduction NWP is an initial/boundary value problem. Having the following conditions, the model simulates or forecasts the evolution of the atmosphere. – an estimate of the present state of the atmosphere (initial conditions) – suitable surface and lateral boundary conditions The more accurate the estimate of the initial conditions, the better the quality of the forecasts. Operational NWP centres produce initial conditions through a statistical combination of observations and short-range forecasts. This approach is called data assimilation.

4. The purpose of a data assimilation is to blend in observations originating from different sources with information contained in a prior estimate of the state of the atmosphere (background). The background is modified to incorporate new observations by combining new and old information in a statistically optimal way. Then, the role of an NWP model is to carry the information gained from past observations forward in time.

5. Data Assimilation (DA) Processes  Ingesting the data  Decoding coded observations  Weeding out bad data  Comparing the data to first guess fields  Interpolating the data on to the model grid for making the forecast

6. Quality control (QC) First guess based rejections Variational QC rejections Compute increments Analysis Observational data pre-processing Preliminary work: Check out duplicate reports Hydrostatic check Black-listing: Data skipped due to systematic bad performance Thinning: Some data are not fully used to avoid over-sampling and correlated errors

7. Observation typesVariables usedNotes SYNOP (synoptic)u,v,T,ps (or z), rh u/v, used only over sea, in the tropics also over low terrain (<150 m). Orographic rejection limit 6hPa for rh,100hPa for z and 800m for ps. AIREPu,v,TUsed only below 50hPa. SATOBsu,vSelected areas and levels DRIBUu,v,psOrographic rejection limit 800 m for, ps. TEMPu,v,T, q, rhq only below 300hPa. 10m u/v used over land only in tropics (<150m. 10hPa orographic rejection limit for u/v, 100 hPa for z/T, 6hPa for rh and 4hPa for q. PILOTu,v10m u/v used over land only in tropics over low terrain (<150 m). Orographic rejection limit 10hPa for u/v SATEMTbSelected channels and areas. SCATTu,vUsed if SST warmer than 273K or if both observed and background wind less than 25m/s. Radaru,v,qCluttering and de-aliasing issues…

8. SYNOP/METAR/SHIP: MSLP, 10m-u/v, 2m-rh. TEMP: Wind, T, q Aircraft: Wind, Temperature Pilot/Profilers: Wind

9. DRIBU: MSL Pressure, Wind-10m

10. Remote sensing retrievals Atmospheric Motion Vectors (AMV) (geo/polar). SATEM thickness. Ground-based GPS Total Precipitable Water SSM/I oceanic surface wind speed and TPW. Scatterometer oceanic surface winds. Radar radial velocity and reflectivity Satellite temperature/humidity/thickness profiles. GPS refractivity (e.g. COSMIC). HIRS: NOAA-16, NOAA-17, NOAA-18, METOP-2 AMSU-A: NOAA-15, NOAA-16, NOAA-18, AMSU-B: NOAA-15, NOAA-16, NOAA-17 AIRS: EOS-Aqua SSMIS: DMSP-16 EOS-Aqua, METOP-2

14. Data coverage issues Primitive equation based models have a number of degrees of freedom in the order of 10 7. For a time window of +/-3 hours, there are typically10 to 100 thousand observations of the atmosphere. They are distributed non-uniformly in space and time. It is necessary to use additional information, called the background field, first guess or prior information. A short-range forecast is used as the first guess in operational data assimilation practices. Current operational systems typically use a 3/6-h cycle performed four times a day.

15. In the operational NWP practice, it is not sufficient to perform spatial interpolation of observations into regular grids. There are not enough data available to define the initial state. The number of degrees of freedom in an NWP model is of the order of 10 7, while the total number of conventional observations is of the order of 10 4 –10 5. There are many new types of data such as satellite and radar observations, but: they do not directly measure the variables used in the models their distribution in space and time is very non-uniform.

16. In addition to observations, it is necessary to use a first guess estimate of the state of the atmosphere at the grid points. The first guess (or background field) is our best estimate of the state of the atmosphere prior to the use of the observations. A short-range forecast is normally used as a first guess in operational systems in what is called an analysis cycle. Over data rich regions, the analysis is dominated by the information contained in the observations. In data-poor regions, the forecast benefits from the information coming from the upstream. For instance, 6-h forecasts over the North Atlantic Ocean are usually good, because of the information coming from the observation rich North America. The model is able to transport information from data-rich to data-poor areas.

17. Global analysis cycle.Regional analysis cycle.

18. Data Assimilation (DA) Techniques Empirical Assimilation Methods Successive Corrections-Iterative analysis (empirical) Newtonian Relaxation (nudging) Sequential Methods Optimal interpolation (OI) (statistical) 3-Dimentional “VARiational” DA (statistical) Non-Sequential Methods 4-Dimentional “VARiational” DA (statistical) Ensemble Kalman Filter (advanced) Hybrid methods: Combinations of ensemble DA techniques: ETKF-3DVar, EnKF-4DVar (advanced)

19. Suppose we have the pressure, p i and temperature, T i, every day for a year. Let n=365. The mean pressure is: and like-wise for mean temperature. The variance of pressure: and like-wise for temperature variance σ 2 T. The standard deviations, σ p and σ T are the square roots of the variances. They measure the root mean square deviation from the mean. Recalling Some Basics of Statistics

20. The covariance of p and T is defined as: The correlation between p and T is the normalized covariance: It is a dimensionless number, and bound between -1 and +1. If p and T tend to be greater than their mean values at the same time, or less than their mean values at the same time, they are positively correlated and ρpT > 0. If p tends to be greater than its mean value when T is less than its mean, and vice-versa, then p and T are negatively correlated and ρpT < 0.

21. An assimilation system deals with: Observations (y o ) Background field (x b ) Observation and forecast errors and statistics An assimilation system generates an analysis (synthesis). Analysis is used in a number of avenues: Initial conditions for NWP models Climatology and re-analyses Observing system design Fundementals of a DA System

22. Say the back ground field is a model 6-h forecast: x b If the observed quantities are not the same as the model variables, the model variables are converted to observed variables: y o The first guess of the observations is denoted as: H(x b ) where H is the observation operator. The difference between the observations and the background: y o − H(x b ) It is referred as the observational increment or innovation.

23. Observations (y o ) First guess at the grid points (X b ) First guess at the observation location H(X b ) The innovation vector (y o -H(X b )) The difference between observations and the first guess are taken at the observation location and referred as innovation vectors (yo-H(x)). A schematic illustration of innovation vectors

24. The analysis x a is obtained by adding the innovations to the background field with weights W that are determined based on the estimated statistical error covariances of the forecast and the observations: x a = x b +W[y o − H(x b )] Different analysis schemes (SCM, OI, 3D-Var,and KF) are based on this equation, but di ff er by the approach taken to combine the background and the observations to produce the analysis. Earlier methods such as the SCM used weights which were determined empirically. The weights were a function of the distance between the observation and the grid point, and the analysis was iterated several times.

25. In Optimal Interpolation (OI), the matrix of weights W is determined from the minimization of the analysis errors at each grid point. In the 3D-Var approach, one defines a cost function proportional to the square of the distance between the analysis and both the background and the observations. This cost function is minimized to obtain the analysis. Lorenc (1986) showed that OI and the 3D-Var approach are equivalent if the cost function is defined as: The cost function J measures: The distance of a field x to the observations (first term) The distance to the background x b (second term).

26. The distances are scaled by the observation error covariance R and by the background error covariance B respectively. The minimum of the cost function is obtained for x=x a, which is defined as the analysis. The analysis obtained by OI and 3D-Var is the same if the weight matrix is given by W=BHT(HBHT+R −1 ) −1 The difference between OI and the 3D-Var approach is in the method of solution: In OI, the weights W are obtained for each grid point or grid volume, using suitable simplifications. In 3D-Var, the minimization of J is performed directly, allowing for additional flexibility and a simultaneous global use of the data.

27. The variational approach has been extended to four dimensions, by including within the cost function the distance to observations over a time interval (assimilation window). This is called four-dimensional variational assimilation (4DVar). In the analysis cycle, the importance of the model may be stressed as: It transports information from data-rich to data- poor regions. It provides a complete estimation of the four- dimensional state of the atmosphere.

28. In variational assimilation, we minimize a cost function, J, which is normally a sum of two terms: J B is the distance between the analysis and the background field J O is the distance to the observations: Variational Data Assimilation Formulation

29. A Schematic illustration of variational DA

30. JbJb JoJo Where: x: model state variable (dimension N) x b : background field B: background-error covariance matrix H: forward interpolation (linear or non-linear) y: observation vector for the observations R: observation-error covariance matrix In practice, the solution is obtained through minimization algorithms for J(x) using iterative methods for minimization; the conjugate gradient or quasi-Newton methods. 3D-Var minimization cost function

31. A schematic representation of the cost function in a simple 1-D case. (Bouttier and Courtier, 1999)

32. Ensemble mean observed (idealized!) Ensemble members time Initial state Uncertainties in the initial state: Ensembles

33. Why do we need a hybrid DA system? A 3D-VAR system uses only climatological (static) background error covariances. Flow-dependent covariance through ensemble is needed. Hybrid combines climatological and flow-dependent background error covariances. It can be adapted to an existing 3D-VAR system. Hybrid can be robust for small size ensembles.

34. Ensemble Formulation Basics Assume the following ensemble forecasts: Ensemble mean: Ensemble perturbations: Ensemble perturbations in vector form:

35. 3D-VAR increment Total increment including hybrid Weighting coefficient for static 3D-VAR covariance Weighting coefficient for ensemble covariance Extended control variable C: correlation matrix for ensemble covariance localization Conserving total variance requires: The Hybrid Formulation… Ensemble covariance is implemented into the 3D-VAR cost function via extended control variables:

36. WRF Ensemble: M1 M2 M3 M4 M5 …. Mn Compute: ensemble mean Compute: Ensemble perturbations (ep2): u,v,t,ps,q, mean, and std_dev WRF-VAR (VERIFY): ob.etkf ensemble ETKF: Update ens perturbations Update: ens. initial conditions Update: ens. boundary conditions WRF-VAR (QC-OBS): filtered_ob.ascii WRF ensemble run for the next cycle cycling WRF-VAR (hybrid) Update: ensemble mean WRF deterministic forecast run (Demirtas et al. 2009) A hybrid (3DVAR –ETKF) system design and implementation

37. Data Assimilation Problems  Transferring information from the scattered locations and times of the observations to the model grid, while the same time.  Preserving the inter-related physical, dynamical and numerical consistency in the short-range forecast, since these are essential for making consistently good NWP forecasts.

38. Acknowledgements: Thanks to documents/images of ECMWF, P. Lynch (UoD) and various others that provided excellent starting point for this talk!

39. Thanks for attending….