1. 2.11 Assimilation methods, including use of data in cloudy regions Daryl Kleist 1 Data Assimilation and Observing Systems Working Group Beijing, China 27-28 October 2015 Univ. of Maryland-College Park, Dept. of Atmos. & Oceanic Science With sincere thanks to everyone that provided material. I have tried to condense as much as possible, but was left with much more than the time allotted. With apologies for leaving anything out…..

2. Hybrid-4DEnVar (Courtesy UKMO) Hybrid-4DVar uses PF model to propagate B implicitly 4DEnVar uses ensemble instead of PF, but much more IO required. – Each iteration ~8 times faster with 200 members. Analysis consists of two parts: – A 3DVar-like analysis based on the climatological covariance B c – A 4D analysis consisting of a linear combination of the ensemble perturbations. Localisation is currently in space only: same linear combination of ensemble perturbations at all times. The same executable can do an ensemble of analyses: En-4DEnVar: – (For each member, analyse increments relative to its own background trajectory.) Additional routines required to deal with perturbed observations, inflation, etc..

3. ECMWF Testing of Hybrid Gain Figure 5. Root mean square background forecast observation departures for radiosonde zonal wind (left column) and radiosonde temperature (right column) for the northern hemisphere (top row), tropics (middle row) and southern hemisphere (bottom row). Curves refer to TL399 EnKF (continuous line), TL399 static 4DVar (dashed line), TL399 Hybrid Gain EnDA (dash-dot line) and present statistics computed over the 2012-06-01 to 2012-08-31 period. Triangle symbols indicate statistically significant difference at the 95% confidence level between the Hybrid Gain EnDA departures and the 4DVar departures. Bonavita, M., M. Hamrud, and L. Isaksen, 2015: EnKF and Hybrid Gain Ensemble Data Assimilation Part II: EnKF and Hybrid Gain Results. Mon. Wea. Rev. doi:10.1175/MWR-D-15-0071.1, in press.

4. Page 4 – December 15, 2015 Alternative ensemble DA approaches (Mark Buehner, Environment Canada) Our EnKF provides high-quality 256-member ensembles for both data assimilation (deterministic and ensemble) and initializing the ensemble forecasts Due to computational cost, current EnKF algorithm limits the volume of observations (maybe 10-20% of obs used in EnVar?) Only a limited amount of fortran code is shared between the EnKF and EnVar systems – significant work required to increase the amount of shared code Theoretically better method for applying localization and ability to use hybrid covariances are advantages of variational approach vs. current approach in EnKF These ideas motivate the testing of variational approaches for ensemble data assimilation (MetOffice and MeteoFrance are also pursuing similar approaches)

5. Page 5 – 15 décembre 2015 Scale-dependent covariance localization Forecast impact – Comparison against ERA-Interim T+24h Zonal mean  Control is better  Scale- Dependent is better Std Dev difference for U North Pole South Pole

6. NCEP (Planned) 2016 Implementation 6 Hybrid 4D EnVar to become operational for GFS/GDAS in early 2016 (tentative) – Tests at low resolution helped design configuration – Real time and retrospectives underway, operational package quasi- frozen Package Configuration – T1534 deterministic GFS with 80 member T574L46 ensemble with fully coupled (two-way) EnKF update (87.5% ensemble & 12.5% static), same localization as current operations – Incremental normal mode initialization (TLNMC) on total increment – Multiplicative inflation and stochastic physics for EnKF perturbations – Full field digital filter – All-sky MW radiance assimilation, aircraft temperature bias correction – Minor model changes

7. Full Resolution (T1534/T574) Trials 500 hPa AC for the Operational GFS (Black, 3D Hybrid) and Test 4D configuration (Red) for the period covering 02-01-2015 through 10-03- 2015 (242 cases!). AtlanticE-Pac C-Pac W-Pac Mean TC Track Errors for the operational GFS (Green, 3D Hybrid) and Test 4D configuration (Red) for the period covering 05-01-2015 through 10-03-2015 ().

8. 4DEnVar (Courtesy UKMO)

9. Increasing model resolution produces smaller error than increasing ensemble size in NCEP GFS EnVar Exp. NameHoriz. localizationVert. localiza1onWeight of static B T254T670Operational localization table-0.50.125 T254T670Ens320Operational localization table‐0.50.0 T670T670 75% of the operational localiza4on table ‐0.3750.125 c/o Lili Lei and Jeff Whitaker

10. Use of data in cloudy regions

11. Progression of All-sky Microwave Radiance in NCEP’s GFS System  Previous work  Preference given to clear-sky in the data thinning was removed;  Cloud signal removal for radiances with thin cloud was turned off in bias correction;  Thick cloud filtering was turned off;  AMSU-A observation error: symmetric observation error (Geer et al. 2011).  Major upgrade in 2012: provided basic framework for all-sky radiance assimilation study for GFS, NAM and HWRF  Introduced individual hydrometeors ql,qi,qr,qs,qg,qh into GSI as state variables;  Passed Jacobians w.r.t. hydrometeors into the GSI inner loop;  Control variable(s): cloud water (cw) or individual hydrometeors.  Observation operator revision and bug fixes  To present: in the hybrid 3D EnKF-VAR data assimilation system  Situation-dependent observation error inflation; AMSUA-A observation error re-tuned;  All-sky radiance bias correction strategy (Zhu et al. 2014)  Additional quality control: cloud effect (Geer et al. 2013) and emissivity sensitivity screening;  Normalized cloud water control variable; New static background error variance and correlation lengths for cloud water; Non-zero Jacobian for locations of clear-sky or small amount clouds;  Validation and improvement of CRTM under all-sky (see posters from Emily Liu 8p.01 & Paul van Delst 2p.03)  Other changes and bug fixes.  Parallel test in the 4D EnVar system for Q1FY16 implementation  Included in the pre-implementation package (see poster from Andrew Collard 6.01) 11

12. Clear-sky OmF vs. All-sky OmF CLW  More data coverage: Thick clouds that are excluded from clear-sky assimilation are now assimilated under all-sky condition  Rainy spots are excluded from both conditions AMSUA NOAA19 CH1 00Z 20131029 12

13. Two approaches to the problem: 1- the “complete” physical approach to clouds, where the analysis control variable is augmented to include microphysical variables. Not much progress on this side since I left NCAR, 2- the “dynamical” approach, where the analysis control variable is limited to 3D cloud fraction and the model is reduced to dynamical advection (the physics being switched off). This approach is useful for nowcasting. We are further developing this for the US Air Force. Worth mentioning, the current testing of: - the use of particle filters to retrieve cloud fraction profiles for every Infrared satellite sounder field-of-view, - the Adaptive Advection Adjustment via Realigned Grid mesH (AAARGH). JCSDA Perspective

14. AAARGH: Components of the Adjustment Technique Observations: GOES-imager radiances Obs operator + TL/AD: simple operator using CRTM (clear-sky) + cloud fractions Grid warping: compute & store 2D displacement field Following cycle: read previous displacement field and apply to WRF forecast using dWRF limited to cloud fractions

15. The following cycle: WRF WRF + AAARGH Difference

16. RMSE of forecast vs. GOES radiances WRF + AAARGH WRF Difference

17. ECMWF Progress on All-Sky Recent progress in all-sky assimilation: - Consolidation of the all-sky technique as the way to assimilate microwave water vapour and imaging channels: - Activation of GMI and AMSR-2 in operations (August 2015) - Transfer of 4 * MHS to the all-sky framework in operations (May 2015) - All-sky water-vapour channels are in 2nd/3rd place in FSOI measures now (at around 15%) - Better understanding of forecast model cloud issues (which give biases in allsky O-B): - Maritime stratocumulus diurnal cycle problems - Lack of supercooled liquid water in high-latitude cold-air outbreaks (CAO) Plans for next year: - Improved all-sky framework: - Add coverage over snow-covered land surfaces - Improve all-sky observation error modelling in clear and light-cloud situations - Address forecast model cloud biases: - Changes to the shallow convection scheme to produce supercooled liquid water at lower temperatures. - Move HIRS (and possibly IASI) infrared water vapour channels to the all-sky approach Further challenges: - Try again to transfer microwave temperature sounding channels to the all-sky approach - Observation error correlations in cloudy situations - Low-frequency rain-focused channels (e.g. 10 GHz) - Simplified 3-D radiative transfer modelling - Further improve simulation of optical properties of frozen particles - Support ICI, the future mm-wave instrument targeting ice clouds 17

18. Status of tropospheric microwave at ECMWF: Operations at 41r2 configuration for early 2016 Slide 18 InstrumentOceanSea-iceLandSnowy land High land SSMIS-F17 imager  GMI  AMSR2  SSMIS-F17 sounder  SSMIS-F18 sounder  MHS on Metop-A/B and NOAA-18/19  MWHS  MWHS-2  ATMS  AMSU-A on Metop-A/B and NOAA-15/18/19  Imager channels 19-90 GHz 183 GHz WV 50 GHz temperature  =clear-sky  =cloud and precipitation

19. Towards an Advanced Cloudy IR Radiance Assimilation (T,Q,V)(T,Q,V) model physics (M) (T,Q,V,ciw,clw,cc) Cloudy (RT) (R cal ) Jo=(R obs -R cal ) Cloudy (RT) * model physics (M*) (T,Q,V,ciw,clw,cc)* (T,Q,V)* (R cal )* Cloudy radiances R cal are simulated via a chain of forward operators (M,RT). The fit of the analysis to the observations is computed (Jo) Jo is minimized by perturbing the analysis variables according to gradients from a chain of adjoint operators (RT*,M*)

20. ECMWF Summary and Plans All Sky IR The simplified assimilation of overcast HIRS, AIRS and IASI works very well and became operational at ECMWF in 2009. We are in the process of developing the advanced cloudy assimilation system (drawing on experience from a similar scheme for rain affected microwave radiances) Initially, cloud parameters (clw,ciw,cc) will be diagnostic and driven by the model physics via adjustments in the analysis control vector variables (T,Q and V). Later the cloud parameters will be tested as full variables in the analysis control vector (but this will require accurate background error observations

21. Backup and Reference Slides