1. Hybrid 4D EnVar for the NCEP GFS (with comments on initialization) Daryl Kleist 1,3, Jeff Whitaker 2, Kayo Ide 3, and John Derber 1 Lili Lei 2,4, Rahul Mahajan 1,5, Catherine Thomas 1,3,5 1 NOAA/NWS/NCEP/EMC 2 NOAA/OAR/ESRL/PSD 3 University of Maryland 4 CIRES 5 IMSG With acknowledgements to Dave Parrish (EMC), Ricardo Todling (NASA/GMAO), Xuguang Wang (OU), Ting Lei (OU) and many others 16 th EnKF Workshop

2. Ensemble-Var methods: nomenclature En-4DVar: Propagate ensemble P b from one assimilation window to the next (updated using EnKF for example), replace static P b with ensemble estimate of P b at start of 4DVar window, P b propagated with tangent linear model within window. 4D-EnVar: P b at every time in the assim. window comes from ensemble estimate (TLM no longer used). As above, with hybrid in name: P b is a linear combination of static and ensemble components. 3D-EnVar: same as 4D ensemble Var, but P b is assumed to be constant through the assim. window (current NCEP implementation). 2

3. 3 EnKF member update member 2 analysis high res forecast GSI Hybrid Ens/Var high res analysis member 1 analysis member 2 forecast member 1 forecast recenter analysis ensemble Dual-Res Coupled Hybrid Var/EnKF Cycling member 3 forecast member 3 analysis T254L64 T574L64 Generate new ensemble perturbations given the latest set of observations and first-guess ensemble Ensemble contribution to background error covariance Replace the EnKF ensemble mean analysis and inflate Previous CycleCurrent Update Cycle

4. 4D EnVar: The Way Forward ? Natural extension to operational 3D EnVar –Uses variational approach with already available 4D ensemble perts No need for development of maintenance of TLM and ADJ models –Makes use of 4D ensemble to perform 4D analysis –Modular, usable across a wide variety of models Highly scalable –Aligns with technological/computing advances Computationally inexpensive relative to 4DVAR (with TL/AD) –Estimates of improved efficiency by 10x or more, e.g. at Env. Canada (6x faster than 4DVAR on half as many cpus) Compromises to gain best aspects of (4D) variational and ensemble DA algorithms Other centers exploring similar path forward for deterministic NWP –Canada (potentially replace 4DVAR), UKMO (potentially replace En4DVar)

5. Single Observation (-3h) Example for 4D Variants 5 4DVAR H-4DVAR_AD  f -1 =0.25 H-4DENSV  f -1 =0.25 4DENSVTLMADJ ENSONLY

6. 6 Observing System Simulation Experiments (OSSE) Joint OSSE –International, collaborative effort between ECMWF, NASA/GMAO, NOAA (NCEP/EMC, NESDIS, JCSDA), NOAA/ESRL, others –ECMWF-generated nature run (c31r1) T511L91, 13 month free run, prescribed SST, snow, ice Observations from (operational) 2005/2006 observing system developed –NCEP: ‘conventional’, sbuv ozone retrievals, GOES sounder radiances –NASA/GMAO: all other radiances (AMSUA/B, HIRS, AIRS, MSU) Older version of the CRTM Simulated observation errors developed by Ron Errico (GMAO/UMBC) –Horizontally correlated errors for radiances –Vertically correlated errors for conventional soundings Synthetic observations used in this study were calibrated by Nikki Prive (GMAO) –Attempt to match impact of various observation types with results from data denial experiments

7. 3D to 4D hybrid 7 Model/Assimilation NCEP Global Forecast System (GFS) (T382L64; post May 2011 version – v9.0.1) 80 member EnKF, 25% static hybrid, additive/mult. inflation Test Period 01 July 2005-31 August 2005 (3 weeks ignored for spin-up) Analysis Error Difference (August) H-4DEnVar (With TLNMC) minus 3D hybrid

8. 500 hPa AC (against EC NR) 8

9. Constraint Options 9 Tangent Linear Normal Mode Constraint –Based on past experience and tests with 3D hybrid, default configuration includes TLNMC over all time levels (quite expensive) Weak Constraint “Digital Filter” –Construct filtered/initialized state as weighted sum of 4D states Combination of the two –Apply TLNMC to center of assimilation window only in combination with JcDFI (Cost effective alternative?)

10. Constraint impact (single case) Impact on tendencies Dashed: Total tendencies Solid: Gravity mode tendencies All constraints reduce incremental tendencies Impact on ratio of gravity mode/total tendencies JcDFI increases ratio of gravity mode to total tendencies TLNMC most effective (but most expensive) Combined constraint potential (cost effective alternative)

11. Analysis Error (cycled OSSE) Time mean (August) change in analysis error (total energy) relative to 4D hybrid EnVar experiment that utilized no constraints at all TLNMC universally better Combined constraint mixed JcDFI increases analysis error

12. Low Resolution GFS/GDAS Experiments with real observations 12 Basic configuration –T254L64 GFS, opnl obs, GFS/GDAS cycles 20120701-20121001 PR3LOEX0 –3DVAR PRHLOEX1 –Hybrid 3D EnVar, 80 member T126L64 ensemble with fully coupled (two-way) EnKF update, slightly re-tuned localization and inflation for lower resolution, TLNMC on total increment, 75% ensemble & 25% static PRH4DEX1 –Hybrid 4D EnVar, TLNMC on all time levels, only 1x150 iterations –Hourly TC relocation, O-G, binning of observations

13. 500 hPa Die Off Curves 13 Northern Hemisphere Southern Hemisphere 4DHYB-3DHYB 3DVAR-3DHYB Move from 3D Hybrid (current operations) to Hybrid 4D- EnVar yields improvement that is about 75% in amplitude in comparison from going to 3D Hybrid from 3DVAR. 4DHYB ---- 3DHYB ---- 3DVAR ---- 4DHYB ---- 3DHYB ---- 3DVAR ----

14. 14

15. (Preliminary) Results and Comments 15 4D extension has positive impact in OSSE and real observation (low resolution) framework 4D EnVar does have slower convergence As configured, 4D EnVar was 40% more expensive than 3D hybrid (caveats being different iteration count, low resolution and machine variability) –TLNMC (balance constraint) over all time levels quite expensive –I/O potential issue, optimization will be necessary prior to implementation. Option to post-process ensemble prior to use in assimilation in subsequent cycle

16. (4D) Incremental Analysis Update 16 In addition to TLNMC, we currently utilize full-field digital filter initialization in the GFS –Can have undesirable impacts being a full field filter IAU (Bloom et al.) has been used at GMAO and elsewhere as alternative to DFI –Increment is passed to model throughout the window as a forcing term –However, this has typically been done using a 3D increment (rescaled) through the window 4D EnVar lends itself to a minor modification of the IAU procedure to use the 4D increment –This also provides a mechanism for passing the 4D solution to the model –Perhaps a way to help spin up/spin down of clouds, precipitation, etc. Promising preliminary results in EnKF context, work underway in 4D hybrid EnVar context

17. EnKF-IAU and EnKF-DFI EnKF-IAU noticeably better 17 Courtesy of Lili Lei

18. 18 Temperature Compare 6-h forecasts to EC analysis EnKF-IAU forecasts closer to EC analysis, esp for winds Uwind (Vwind is similar to Uwind) Courtesy of Lili Lei

19. Next Steps/Future Development 19 Configure update and outer loop –Continue to investigate use of IAU to force 4D increment into model Replace all other initialization options? Get rid of TLNMC as well? –Quasi-outer loop (rerun of nonlinear model as in 4D Var) –Lengthen assimilation window (backward) in catch-up cycle May require temporal localization (code in place courtesy of OU collaboration) Test role of stochastic physics as replacement for additive inflation in 4D EnVar context Improved (or at least tuned) localization Optimize static B for 4D hybrid –Computationally (current static B is bottleneck in code) –Add temporal information (FOTO) –Weights, including scale-dependence –Increase role of ensemble (to 100%?) 4D EnVar tentatively scheduled to be part of GDAS/GFS upgrade in 2015

20. Backup Slides 20

21. Time Evolution of Increment 21 t=-3h t=0h t=+3h H-4DVAR_AD H-4DENSV Solution at beginning of window same to within round-off (because observation is taken at that time, and same weighting parameters used) Evolution of increment qualitatively similar between dynamic and ensemble specification ** Current linear and adjoint models in GSI are computationally impractical for use in 4DVAR other than simple single observation testing at low resolution TLMADJENSONLY

22. Accounting for under-represented source of uncertainty 22 Current operations –Multiplicative inflation – Helps compensate for finite-sized ensemble –Additive perturbations – Helps compensate for lack of consideration of model uncertainty Proposed configuration for T1534 SL (3072 x 1536) GFS –Keep multiplicative inflation –Reduce additive perturbations from 32 to 5% –Add stochastic physics in model SPPT -- Stochastic Physics Perturbation Tendency SKEB – Stochastic Energy Backscatter VC – Vorticity Confinement SHUM – Boundary Layer Humidity perturbations See talk by Jeff Whitaker in Session 9

23. 23 Better spread behavior (2014042400) Current operations –Spread too large –Spread decays and recovers Stochastic Physics –Spread decreased overall (consistent with error estimates) –Spread grows through assimilation window 3HR 6HR 9HR

24. Hybrid ensemble-4DVAR [H-4DVAR_AD] 24 Incremental 4DVAR: bin observations throughout window and solve for increment at beginning of window (x 0 ’). Requires linear (M) and adjoint (M T ) models Can be expanded to include hybrid just as in the 3DHYB case With a static and ensemble contribution to the increment at the beginning of the window ADJ TLM

25. 4D-Ensemble-Var [4DENSV] 25 As in Buehner (2010), the H-4DVAR_AD cost function can be modified to solve for the ensemble control variable (without static contribution) Where the 4D increment is prescribed exclusively through linear combinations of the 4D ensemble perturbations Here, the control variables (ensemble weights) are assumed to be valid throughout the assimilation window (analogous to the 4D-LETKF without temporal localization). Need for computationally expensive linear and adjoint models in the minimization is conveniently avoided.

26. Hybrid 4D-Ensemble-Var [H-4DENSV] 26 The 4DENSV cost function can be easily expanded to include a static contribution Where the 4D increment is prescribed exclusively through linear combinations of the 4D ensemble perturbations plus static contribution Here, the static contribution is considered time-invariant (i.e. from 3DVAR- FGAT). Weighting parameters exist just as in the other hybrid variants. Again, no TLM or ADJ.