T HE D ATA A SSIMILATION S YSTEM IN THE ERA-20C R EANALYSIS ERA-20C: ERA-CLIM pilot reanalysis of the 20th-century using surface observations only Paul Poli, Hans Hersbach, David Tan, Dick Dee, Carole Peubey, Yannick Trémolet, Elias Holm, Massimo Bonavita, Lars Isaksen, and Mike Fisher

Outline Expectations and challenges ERA-20C system overview Assimilation method Evolution of background errors Post-assimilation diagnostics Issues Case studies (1899, 1987) Conclusions

How good are forecasts issued from analyses of Ps only? Poli, ERA-20C Data Assimilation System, EMS Day 6 >~ day 3 Day 6 ~ day 3 Day 6 fc error Day 3 fc error [K]

Challenge for any climate dataset based on observations: changing observing system Surface pressure Poli, ERA-20C Data Assimilation System, EMS

Challenge for any climate dataset based on observations: changing observing system (cont.) Wind above ocean surface Poli, ERA-20C Data Assimilation System, EMS

ERA-20C system overview Resolution as in ERA-20CM, except archive 3-hourly – 75 surface fields – 14 fields for each of the 91 model levels – 16 fields (+PV, +RH) for each of the 37 pressure levels Forcings: as in ERA-20CM Surface observations assimilated – Surface pressure from ISPD – Surface pressure and near-surface wind from ICOADS 2.5.1, ocean only 4DVAR analysis – Outer loop (short forecasts) at T159 or 125 km – Inner loop (analysis increments) at T95 or 210 km – 24-hour window 10 realizations or members, including a control 6 production streams Poli, ERA-20C Data Assimilation System, EMS

ERA-20C production streams Speed: ~30-40 days/day/stream. Completed in ~200 days. Missing Oct 2009-Dec 2010 During production: – 3.5 Tb/day, 350 million of meteorological fields. – DVAR assimilations daily A failure rate as low as 0.1% would imply already 2 manual interventions per day.  Home-grown solution to automatically detect model explosion, stop production, halve the model time-step, set the date back, resume production, record the problem, and resume to normal time-step once problematic date is recovered Poli, ERA-20C Data Assimilation System, EMS

Constructing a history of the past with (24-hour) 4DVAR data assimilation Poli, ERA-20C Data Assimilation System, EMS [Pa] Surface pressure at Montreal, Quebec Observations from ISPD 3.2.6, collection #3004 (Canadian Stations Environment Canada )

Ensemble of 4DVAR data assimilations: Discretization of the PDF of uncertainties Poli, ERA-20C Data Assimilation System, EMS Surface pressure at Montreal, Quebec Observations from ISPD 3.2.6, collection #3004 (Canadian Stations Environment Canada ) Background forecast, with uncertainties in the model and its forcings (HadISST ensemble) Observations with uncertainties (some could not be fitted – they are VARQC rejected) Analysis, with uncertainties Benefits: 1. Estimate automatically our background errors, and update them 2. Provide users with uncertainties estimates (not perfect, but better than … nothing) Forcing uncertainties Model uncertainties Observation uncertainties Reanalysis uncertainties

ERA analysis window configurations Poli, ERA-20C Data Assimilation System, EMS ERA-40 ERA-Interim ERA-20C

Observation diversity in ERA-20C Poli, ERA-20C Data Assimilation System, EMS Surface pressure Wind components

1-year ensemble spread, throughout the century Poli, ERA-20C Data Assimilation System, EMS h +27h [hPa]

From the ensemble spread, one can estimate background error variances Poli, ERA-20C Data Assimilation System, EMS Estimate of bkg. error stdev. for vorticity at model level 89, for the year 1900 [s**-1]

Evolution of background error (std. dev.) Zonal wind near the surface Poli, ERA-20C Data Assimilation System, EMS [m/s]

Self-updating background error covariances, throughout the century (updated every 10 days, based on past 90 days) Over the course of the century, more observations result in…  Smaller background errors, with sharper horizontal structures  Analysis increments that are smaller, over smaller areas = ERA-20C system adapts itself to the information available With satellites, radiosondes,… (for comparison) Poli, ERA-20C Data Assimilation System, EMS

Impact of using our own background errors, instead of those derived for NWP Poli, ERA-20C Data Assimilation System, EMS N. Hem. extratropics: 1 day of forecast gain S. Hem. extratropics: 1.5 day of forecast gain Tropics: brings 12h forecast skill above 60%

Background errors: stored also in the observation feedback Ortelius World map, circa 1570 ERA-20C 1900 weather world map of uncertainty, circa 2013 Poli, ERA-20C Data Assimilation System, EMS [hPa]

Fit to assimilated observations Southern mid-lat. Northern mid-lat. Poli, ERA-20C Data Assimilation System, EMS Before assimilation After assimilation

Assimilation error assumptions: budget closure AssumedActual Poli, ERA-20C Data Assimilation System, EMS Showing only observations in the first 90 minutes of the 24-h window

What about error growth within the 24-hour window? Poli, ERA-20C Data Assimilation System, EMS <+1h+23h [hPa] RMS (O-B) RMS (O-A) +12h <+1h+23h+12h

Estimated (and used) pressure observation error biases Poli, ERA-20C Data Assimilation System, EMS

Mean differences between consecutive streams Poli, ERA-20C Data Assimilation System, EMS

Upper-air temperatures Poli, ERA-20C Data Assimilation System, EMS Anomalies ( ) Analysis increments

Issues Model time-step – On the long (cheap) side, 1 hour instead of 30 minutes (would have doubled the cost of the run) Observation quality control – Too loose, let a few bad observations in Analysis increments far away from observations – Systematic and changing upper-air analysis increments, causing spurious signal interfering with trends Poli, ERA-20C Data Assimilation System, EMS

Poli, ERA-20C Data Assimilation System, EMS Analyses Forecasts, from 96 hours ahead to 12 hours ahead Great Storm 16 October 1987, 00 UTC NWP ERA-15 ERA-40 ERA-Int ERA-20C “It was the worst storm since 1703 and was analysed as being a one in 200 year storm for southern Britain” (Met Office)

U.S. East Coast Great Blizzard February 1899 One of the most intense blizzards in US history Subject of earlier research, e.g. Kocin, Paul J., Alan D. Weiss, Joseph J. Wagner, 1988: The Great Arctic Outbreak and East Coast Blizzard of February Wea. Forecasting, 3, 305–318. Maps used for such studies usually based on measurements over the continental US and Canada Results from ERA-20C show global picture, with a wave-2 planetary pattern Embedded in this system, an extraordinary powerful low, nearly stationary, battered the Atlantic for several days Poli, ERA-20C Data Assimilation System, EMS

Comparison of surface pressure reanalyses for 1-15 February 1899 Poli, ERA-20C Data Assimilation System, EMS ERA-20C NOAA/CIRES 20CR

11 February 1899 Kocin et al., WAF 1987 Poli, ERA-20C Data Assimilation System, EMS ERA-20C NOAA/CIRES 20CR

Application of ERA-20C for comparing with independent observational data records: e.g. temperatures from ships Temperatures from ships biased warm during day- time (measurements contaminated by the ship structures, heated by sun) Some data problems in 1980? Can be traced to 3 individual collections from the feedback archive Poli, ERA-20C Data Assimilation System, EMS

Conclusions Innovative components in ERA-20C DAS – Ensemble of SST conditions (HadISST ) – Variational bias correction of surface pressure observations – 24-hour 4DVAR – Self-updating background error global covariances from ensemble, and cycling local variances ERA-20C ensemble production essentially done (missing last few months). A ~700Tb meteorological dataset produced in ~200 days. Trends are contaminated by systematic analysis increments Preliminary assessment suggests some capacity at representing interesting known extreme events, provided they were observed, in spite of low horizontal resolution, very likely thanks to the ensemble, flow- and time- dependent background errors, and 24-hour 4DVAR The automatic/self-update of the background errors approach developed and tested in ERA-20C is expected to be extended to ECMWF NWP operations soon Poli, ERA-20C Data Assimilation System, EMS

For more details… Poli, ERA-20C Data Assimilation System, EMS ERA Report 14 available from the ECMWF website >> Publications >> ERA Reports >> ERA Report Series