Maturation of Data Assimilation Over the Last Two Decades John C. Derber Environmental Modeling Center NCEP/NWS/NOAA

Roger Daley (1996) ECMWF seminar on data assimilation “Fifteen years ago, data assimilation was a minor and often neglected sub-discipline of numerical weather prediction. The situation is very different today. Data assimilation in now felt to be important for all climate/environmental monitoring and estimating the ocean state. There has been great advances in both modelling and instrumentation for a variety of atmospheric phenomena and variables, and data assimilation provides the bridge between them.”

Optimal Interpolation (1980s) With the advent of optimal interpolation, analysis schemes transitioned from empirical techniques to theoretically based techniques. With these techniques, one could begin to use information on: –Observations and observation errors –Short term forecasts and forecast errors However, for most applications of optimal interpolation, many approximations had to be made.

Variational assimilation (1990s and 2000s) J = J b + J o + J c J = (x-x b ) T B -1 (x-x b ) + (H(x)-y 0 ) T (E+F) -1 (H(x)-y 0 ) + J C J = Fit to background + Fit to observations + constraints x= Analysis x b = Background B= Background error covariance H= Forward model y 0 = Observations E+F= R = Instrument error + Representativeness error J C = Constraint term

Variational assimilation Inclusion of observation operator (H), transforming the analysis variable into the form of the observation operator, which in turn allowed; –Inclusion of radiances and other indirect observations –Definition of analysis variables different than the model variables or observations –Inclusion of forecast model in interpolation operator (4DVAR) Use of all observations at once, eliminating many approximations/complex codes which were prone to failure Allowed inclusion of additional constraint terms

Variational Assimilation Background error covariances –Background error variance now approximately ½ radiosonde error variance (ECMWF) –Fully non-separable covariance matrices –Inclusion of constraints within background error –Ongoing research in situation dependent background errors

Isotropic Error Correlation in Valley Plotted Over Utah Topography obs influence extends into mountains indiscriminately

Anisotropic Error Correlation in Valley Plotted Over Utah Topography obs influence restricted to areas of similar elevation

Anisotropic Error Correlation on Slope Plotted Over Utah Topography obs influence restricted to areas of similar elevation

Anisotropic Error Correlation on Mt Top Plotted Over Utah Topography obs influence restricted to areas of similar elevation

Observations No large scale data voids. Number of observations used in assimilation increasing rapidly (but not as rapidly as number of observations). Both operational and non-operational satellite data being used operationally. Increased use of data assimilation systems in calibration/validation activities for satellites. Expected data impact from data producers always overoptimistic.

Observations Adequate fast forward models for observations still major problem, e.g., –Precipitation (satellite/radar) –Clouds (IR and microwave) –Lightning Biases in forward models/observations greatly impact the usefulness of data

Variational Assimilation Surprises Computational cost for 3DVAR similar to OI (even with all observations used together). Much of need for Nonlinear Normal Mode Initialization came from data selection. Direct use of radiances produces significant impacts in both hemispheres. –In Southern Hemisphere, significantly stronger circulations were produced using radiances. –Southern Hemisphere forecast skill has become similar to Northern Hemisphere skill. Since NH much better observed, is SH easier? –Microwave instruments dominate impact.

Future? Extension of data assimilation techniques to: –Additional analysis variables (including cloud water/ice, etc.) –Smaller scales –Tropical disturbances –Land/ice/ocean surfaces Inclusion of improved bias correction for background and all types of observations Inclusion of observation specific observational/representativeness errors

Future ? Use of situation dependent background errors Trying to catch up with volume of data from new observing platforms Improved models and physics within analysis systems Ensembles? New systems judged on performance. With data assimilation you must do everything right!

Summary Assimilation is the integration of all knowledge of the atmosphere (observations, physics, statistics) to produce the best estimate of the real state of the atmosphere. Data assimilation systems have matured and become fairly good at large scales for the basic meteorological variables. Assimilation has advanced from a necessary evil to an essential scientifically-based component of numerical weather prediction. However, there are many “research groups” still using empirically based techniques and incomplete systems.

Summary Greatest potential improvement is in improved background error estimates (not increasing the number of observations). Unrealistic expectations from data providers in terms of data impacts. Operational community has lead the scientific development of modern data assimilation systems. In data assimilation, details are extremely important – you must do everything right!

Analysis variables Wind, temperature and surface pressure no longer sufficient. Additional variables: –Cloud ice and water –Ozone and other constituent gases –Surface variables (soil moisture, surface temperature, snow, etc. –Aerosols –Oceans –Etc.

Anisotropic Error Correlation on Mt Top Plotted Over Utah Topography ob’s influence restricted to areas of similar elevation x

Sample forecast error structure

Satellite observations Current Instruments (used) Infrared (IR) sounders (HIRS, GOES) Microwave sounders (MSU, AMSU-A/B) Microwave imager (SSM/I (wind speed, precipitation)) SBUV (ozone profiles) Winds inferred from imagery (GOES, GMS, Meteosat) Scatterometers (Quikscat)

Satellite observations Current Instruments (not used) Visible/IR imager (AVHRR, GOES, MODIS) Microwave sounders ( DMSP/T/T2, TMI) TRMM radar Total Ozone (TOMS) Etc.

Satellite observations Future Platforms EOS-PM (AIRS, AMSU-A, HSB, MODIS) GIFTS (IR sounder) DMSP (SSM/IS) NPP(CrIS, ATMS, VIIRS) NPOESS(CMIS, CrIS, OMPS, ATMS, VIIRS, ALT, SARSAT) METOP(AVHRR, AMSU, IASI, GOME, ASCATT) Cosmic (GPS radio-occultation) Etc.

Satellite observations Different observation and error characteristics –Type of data (cloud track winds, radiances, etc.) –Version of instrument type (e.g., IR sounders - AIRS, HIRS, IASI, GOES, GIFTS, etc.) –Different models of same instrument (e.g., NOAA-15 AMSU-A, NOAA-16 AMSU-A)