ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov A very first introduction to data assimilation for NWP systems Joaquín Muñoz Sabater ECMWF

COSPAR Training Fortaleza, Brasil, 11-Nov ► The data assimilation concept, Some linear estimation theory, ► Data assimilation for Numerical Weather Prediction, ► Overview of the ECMWF Data Assimilation system, The observations, The physical processes, The observation operator (modelled variables), ► The operational configuration at ECMWF, ► The operational schedule at ECMWF Contents of lecture I

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov Why do we need data assimilation?  A crazy tool used by scientists?,  A fashion?  A magic mathematical formula which nobody understands but produces magic results?  …?

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov Why should we limit our speed?

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov How to ‘control’ our speed? Observables

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov RadarSpeedometer y r = 110 km/hy s = 95 km/h What is the best estimation of the speed x of the vehicle? Control variable : x  speed of the car, « Truth » : x t  real speed of the car (unknown), Observation 1 : y r  speed given by radar, Observation 2 : y s  speed given by speedometer Problem description; linear estimation

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov RadarSpeedometer y r = 110 km/hy s = 95 km/h Problem description; linear estimation

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov RadarSpeedometer y r = 110 km/hy s = 95 km/h Case 1) Police officer believe the radar measurement Problem description; linear estimation

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov RadarSpeedometer y r = 110 km/hy s = 95 km/h Case 1) Police officer believe the radar measurement x = y r = 110 Km/h  the driver will pay a traffic fine Problem description; linear estimation

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov RadarSpeedometer y r = 110 km/hy s = 95 km/h Case 1) Police officer believe the radar measurement x = y r = 110 Km/h  the driver will pay a traffic fine Case 2) Police officer calculates the mean between both observations Problem description; linear estimation

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov RadarSpeedometer y r = 110 km/hy s = 95 km/h Case 1) Police officer believe the radar measurement x = y r = 110 Km/h  the driver will pay a traffic fine Case 2) Police officer calculates the mean between both observations x = y r /2 + y s /2 = km/h  the driver will pay a traffic fine Problem description; linear estimation

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov RadarSpeedometer y r = 110 km/hy s = 95 km/h Case 3) Best Linear Unbiased Estimator with all the information, x = C 1 y r + C 2 y s Hypothesis (BLUE): 1) E[  ] = 0, 2) σ 2 x min  r = 10 km/h  s = 5 km/h Problem description; linear estimation

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov RadarSpeedometer y r = 110 km/hy s = 95 km/h Case 3) Best Linear Unbiased Estimator with all the information, x = C 1 y r + C 2 y s Hypothesis (BLUE): 1) E[  ] = 0, 2) σ 2 x min y r y s = 98 Km/h  r = 10 km/h  s = 5 km/h x = Problem description; linear estimation

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov RadarSpeedometer y r = 110 km/hy s = 95 km/h  r = 10 km/h  s = 5 km/h Generalization with p observations Chronometer y m = 85 km/h  m = 4 km /h [p] observations … x a = (H T R -1 H) -1 H T R -1 y x a  Control vector (analysed variables); [v a, d a, a a,…] y  vector of observations; [y r, y s, y m,…] R  variance-covariance matrix of observations; R ii =  i 2 ; R ij =  i  j H  non-linear observation operator; y = H (x) x a = K y y = H (x t ) + ε

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov Generalization with first-guess x a = x b + HB T (HBH T +R) -1 (y-Hx b ) x a  Control vector (analysed) x b  first-guess vector (in NWP forecast by meteorological model) y  vector of observations H  non-linear observation operator y o = H(x t )+  HB T (HBH T +R) -1  Gain K B  variance-covariance matrix of first-guess R  variance-covariance matrix of observations RadarSpeedometer y r = 110 km/hy s = 95 km/h  r = 10 km/h  s = 5 km/h Chronometer y m = 85 km/h  m = 4 km /h [p] observations … Physical model xbxb

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov But why data assimilation in NWP? 1.Improve model initial conditions for better model forecasts, 2.Better representation of  Observation errors (and their probabilistic distribution),  Model errors (and their probability distributions),  Correlations between Observations/Model, 3.Analysis of the role of different DA methodologies to improve weather forecast (minimizations, approximations, etc.), 4.Understanding of the interaction between different physical processes, 5.Others (cal/val, scalability in NWP, etc.)

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov Data assimilation for weather prediction Non-analysed fc temps Modelled variables (Temperature, humitidy,etc) 12h-forecast observations 00 UTC 12 UTC 12h-forecast Analysis 00 UTC (+ 24h) Sequential methods

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov Data assimilation for weather prediction Non-analysed fc temps Modelled variables (Temperature, humitidy,etc) 12h-forecast observations 00 UTC 12 UTC 12h-forecast Analysis 00 UTC (+ 24h) But…  Observations are irregularly distributed in time and space,  Matrices R and B contain millions of observations, inverting (HBH T +R) is very expensive.  Linear assumptions have to be done. Sequential methods

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov Data assimilation for weather prediction time Control variable Minimisation of a cost function: observations simulations Jo JbJb Assimilation window (12h) Jo JbJb First-guess trajetory Corrected trajectory Variational methods

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov Data assimilation for weather prediction A short-range forecast provides an estimate of the atmosphere that is compared with the observations. The two kinds of information are combined to form a corrected atmospheric state: the analysis. Corrections are computed and applied twice per day. This automatic process is called ‘Data Assimilation’. The FORECAST is computed on a grid over the globe. The meteorological OBSERVATIONS can be taken at any location in the grid. The computer model’s prediction of the atmosphere is compared against the available observations, in near real time

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov Overview of DA system at ECMWF  The observations are used to correct errors in the short forecast from the previous analysis time.  Every 12 hours we assimilate 10 – 11,000,000 observations to correct the 80,000,000 variables that define the model’s virtual atmosphere.  This is done by a careful 4-dimensional interpolation in space and time of the available observations; this operation takes as much computer power as the 10-day forecast.

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov Conventional observations used MSL Pressure, 10m-wind, 2m-Rel.Hum. DRIBU: MSL Pressure, Wind-10m Wind, Temperature, Spec. Humidity PILOT/Profilers: Wind Aircraft: Wind, Temperature SYNOP/METAR/SHIP: Radiosonde balloons (TEMP): Note: We only use a limited number of the observed variables; especially over land.

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov Satellite data very important  Satellite measurements are increasingly important:  Global coverage (often only source of observations over ocean and remote land)  High spatial and temporal resolution  Decrease in conventional observing networks (fewer radiosonde stations)  But satellite data are not easy to use:  Satellites do not measure the model variables (temperature, wind, humidity)  They measure radiances, so  either use derived products (e.g. cloud motion and scatterometer winds)  or calculate ‘model radiances’ and compare with observations  Recent developments in data assimilation are designed to improve the use of satellite data  Variational data assimilation: can use radiance data directly  Added model levels in upper stratosphere allows use of additional satellite data  4D-Var: use observations at appropriate time  Increased resolution – more in agreement with the resolution of measurements

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov Satellite data sources used in the operational ECMWF analysis Geostationary, 4 IR and 5 winds 5 imagers: 3xSSM/I, AMSR-E, TMI 4 ozone 13 Sounders: NOAA AMSU-A/B, HIRS, AIRS, IASI, MHS 2 Polar, winds: MODIS 3 Scatterometer sea winds: ERS, ASCAT, QuikSCAT 6 GPS radio occultation

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov Significant increase in number of observations assimilated Conventional and satellite data assimilated at ECMWF

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov The atmosphere does not evolve in isolation, interactions between the atmosphere and the underlying land and ocean are also important in determining the weather. Ocean ice processes, ocean surface waves, land surface, soil, hydrological and snow processes are all represented at ECMWF in the most advanced operational Earth-system model available anywhere. These physical processes have smaller scales than the model grid (16 km) and are therefore represented by so-called “Parametrization Schemes” which represent the effect of the small-scale processes on the large-scale flow. Physical processes in the ECMWF model An accurate model representation of the atmosphere is an important part of the assimilation system

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov The observation operator Observations are not made at model grid points Satellites often measure radiances, NOT temperature and humidity We calculate a model radiance estimate of the observation to enable comparison. This is done with the ‘observation operator’ H.  H may be a simple interpolation from model grid to observation location  H may possibly perform additional complex transformations of model variables to ‘radiance space’ for satellite data. Model T,u,v,q Observation Satellite Radiance compare H Model radiance

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov Model Radiance The variational method allows model radiances to be compared directly to observed radiances Enables use of advanced observation operators Model T and q H compare Observation Satellite Radiance

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov The operational configuration at ECMWF  Configuration:  Deterministic model: T1279L91 (~16km)  Outer loop of 4D-Var T1279L91 and inner loops T159/T255/T255 (~125km/80km/80km)  EPS target resolution T639L62 (to 10 days) and T319L62 thereafter  Wave model (25km and 36 directions)  Implemented in operations 26 January 2010

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov Extract data for 12h period UTC 70sec (min. 8x1PEs) Pre-process satellite data. Cloud clearing. Scatterometer winds. 340sec (min. 16x1PEs) Observation pre-processing for 0000UTC main cycle

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov Analysis and forecast for 0000UTC main cycle BUFR to ODB. 200sec 4x(8-16PEs) Fetch background forecast 275sec 2x(1PE) Analysis: trajectory, minimization and update 4320sec (3072PEs) 10 day forecast t-steps 3070sec (3072PEs) (or 15h fc for cycling: 260sec) Surface analysis. 1010sec 4x(1PEs) 430s 260s 880s 270s 820s 490s 1110s

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov T1279 (16km) Since January ,140,704 grid-points per level Outer loop model resolution is now T1279L91 Important for accurate comparison against observations L60 L91 T799 (25km) ,490 grid-points per level

ECMWF COSPAR Training Fortaleza, Brasil, 11-Nov Operational schedule Early delivery suite introduced June hFC 6h 4D-Var 21-03Z 00 UTC analysis (DA) T day forecast 51*T639/T399 EPS forecasts 03:40 04:00 04:40 06:05 05:00 Disseminate 06:35 Disseminate 02:00 12h 4D-Var, obs 09-21Z 18 UTC analysis 03:30 3hFC 6h 4D-Var 9-15Z 12 UTC analysis (DA) T day forecast 51*T639/T399 EPS forec. 15:40 16:00 16:40 18:05 17:00 Disseminate 14:00 12h 4D-Var, obs 21-09Z 06 UTC analysis 15:30