1. Data Assimilation for High Impact Weather Forecast Yuanfu Xie NOAA/OAR/ESRL OAR/ESRL/GSD/Forecast Applications Branch

2. Outline Review data assimilation techniques; Improving variational data assimilation; Limitations of ensemble Kalman filter (EnKF); Highly nonlinear and non-Gaussian data assimilation; Some case studies of hurricane and tornado; Summary. OAR/ESRL/GSD/Forecast Applications Branch

3. History and modern techniques Objective analysis in the 70s; Variational analysis in the 90s; Kalman filter including EnKF, EnSRF, became hot topics in the late 90s til now; Hybrid of variational analysis (3DVAR/4DVAR) with EnKF covariance matrix now; What is next? OAR/ESRL/GSD/Forecast Applications Branch

4. Improving variational analysis Major reason for the recent interests in EnKF is due to the unsatisfactory variational analysis even it is so successful at ECMWF. Of course, the easy implementation of EnKF or any Monte Carlo methods is another reason. Variational analysis, particularly, 4DVAR, is too complicated and difficult to maintain. OAR/ESRL/GSD/Forecast Applications Branch

5. Issues for the unsatisfactory story It is known that the error covariance is difficult to compute as it is flow, location and time dependent; Adjoint system is complicated, 4DVAR; Control variables, e.g. wind; Constraints for 3DVAR and 4DVAR; Forward operators. OAR/ESRL/GSD/Forecast Applications Branch

6. Control Variable Issues Xie et al 2002 has studied the analysis impact by selecting different control variables for wind, ψ-χ, u-v, or ζ-δ VARS. Conclusions: ζ-δ is preferable (long waves on OBS; short on background); u-v is neutral; ψ-χ is not preferred: short on OBS; long on background; OAR/ESRL/GSD/Forecast Applications Branch

7. Non-physical error by ψ-χ VARS Ψ and χ are some type of integral of u-v. A correction to the background toward the obs changes the integral of u. The background term adds opposite increment to the analysis for keeping the same integral as the background. OAR/ESRL/GSD/Forecast Applications Branch

8. Responses of three type of VARS For a single obs of u, e.g., ψ-χ, u-v, or ζ-δ VARS have different responses as shown here. A ψ-χ VAR may not show this clearly as many filters applied. OAR/ESRL/GSD/Forecast Applications Branch

9. Limitations of Ensemble DA Ensemble DA is relatively easy to implement and It has been considered to improve VARs. Limitations: Major challenge is the limited ensemble members for any applications (more needed, Julier et al 1997); It may not be appropriate for nonlinear and non- Gaussian applications (e,g., severe weather); For some nonlinear observation data, it may have problems. OAR/ESRL/GSD/Forecast Applications Branch

10. Hybrid technique Ensemble filter to provide an error covariance matrix for variational techniques; At ESRL, STMAS uses a multigrid technique to combine EnKF or particle filter with a sequence of variational analysis; at coarse grids, it can use any estimated error covariance matrix, e.g. EnKF… OAR/ESRL/GSD/Forecast Applications Branch

11. Multigrid Technique Using the number of gridpoints to control the base functions. For irresolvable scales, EnKF or particle filter is used.

12. Nonlinear and Non-Gaussian DA Particle filter has been investigated for nonlinear and non-Gaussian DA. It requires more ensemble members; STMAS considers a DA in two steps, variational retrieval and data assimilation. With modern observation networks, more accurate information is available for retrieving. STMAS uses a variational retrieval to mimic objective analysis with simultaneous dynamic constraints. OAR/ESRL/GSD/Forecast Applications Branch

13. A Lognormal 1DVAR PDF: Cost function:

14. Recent development STMAS has been developed and under testing; Its adjoint system is under development; Radar radial wind and reflectivity DA has been added into STMAS; STMAS uses a u-v as control variable in its multigrid analysis; NCAR WRF DA group is working with ESRL for testing ζ-δ WRF DA as seen some advantages in cloud scale analysis. OAR/ESRL/GSD/Forecast Applications Branch

15. Storm Boundary Detection ESRL and MIT have been using STMAS surface analysis of high resolution (2-5km) and high temporal (5-15 minutes); It uses high frequency observation data, e.g., 5 minute ASOS (1 minute ASOS planned); Multi-variate analysis with certain dynamic constraints is under development; STMAS technique will be applied to fire weather, wind energy and other applications. OAR/ESRL/GSD/Forecast Applications Branch

16. Status of STMAS Surface Analysis for Storm/Gust Front Detection Space and Time Multiscale Analysis System (STMAS) is a 4-DVAR generalization of LAPS and modified LAPS is running at terminal scale for FAA for wind analysis. STMAS real time runs with 15 minute latency due to the observation data: STMAS surface analysis is running in real time over CONUS with 5-km resolution and 15-minute analysis cycle. It is so efficient that it runs on a single processor desktop; Real time 5-minute cycle run of STMAS assimilating 5-minute ASOS data also on single processor. Note: Modified LAPS is running at 40+ sites at terminal scales for FAA in real time. STMAS surface analysis with 5 minute latency with (MPI/SMS) and targeting at 2-km resolution; 5-minute cycle; over CONUS domain; assimilating 1-minute ASOS; using HRRR as background. CONUS 15-min cycle 5-min ASOS, 5-min cycle OAR/ESRL/GSD/Forecast Applications Branch

17. FAA/MIT Boundary Detection Application

18. STMAS for frontal boundary detection

19. STMAS vs. HPC

20. Hurricane Forecast Improvement Project (HFIP/NOAA) NOAA HFIP is a major effort for improving hurricane forecast (both tracks and intensity); NOAA 5 year plan: Accuracy of hurricane intensity forecast is improved by 50%; Track lead time is up to 5 days with 50% accuracy improvement. OAR/ESRL/GSD/Forecast Applications Branch

21. 21 Hurricane: costly natural disaster Overview of 2008 hurricane season: 8 hurricanes (5 major) Fatalities: 855 Damage: $42 billions

22. 22 Increased National Needs

23. 23 — 48-Hour Hurricane Track Forecast Error Hurricane Forecast Enhancements

24. 24 — 48 hr Hurricane Intensity Forecast Error Trend Hurricane Forecast Enhancements

25. ESRL in HFIP Global modeling: FIM (finite volume Icosahedra grid modeling); Statistical post-processing; OSSE etc; DA: 1.Regional EnSRF (square root EnKF). 2.Testing STMAS for hurricanes and typhoon (supported by CWB/Taiwan) comparing to LAPS OAR/ESRL/GSD/Forecast Applications Branch

26. Katrina Pre-balanced 950-mb wind speed and height. Balanced 950-mb wind speed and height. LAPS analysis

27. KATRINA LAPS / HRS(2005)

28. Track: WRF 20km Katrina forecast by STMAS Best track: every 6 hours WRF-ARW 72 hour fcst w/ Ferrier physics: every 3 hours

29. Intensity: WRF Katrina forecast by STMAS Wind Barb, Windspeed image, Pressure contour at 950mb Surface pressure j

30. Testing Cycling Scheme for STMAS The previous analysis is done by STMAS using GFS 1 degree forecast as background. The analysis does not contain detailed small scales; The observation data for Katrina at Aug. 27, 2005 at 18Z is sparse even though there is some dropsonde data into the hurricane eyes. Cycling scheme could provide detailed information through assimilation of early hours before STMAS 4DVAR is used. OAR/ESRL/GSD/Forecast Applications Branch

31. Cycling Impact on STMAS analysis

32. STMAS-WRF ARW cycling Impact OAR/ESRL/GSD/Forecast Applications Branch

33. Rapid Intensification and Rapid Weakening (RIRW) Cycling certainly helps with the rapid intensification; For the Kartina case, the WRF ARW keeps the intensification for a longer period of time, which is not realistic; Further study will be on the rapid weakening. OAR/ESRL/GSD/Forecast Applications Branch

34. Severe Storms and Tornado NOAA long term goal (20 years) is on tornado forecasts, for longer lead time forecasts (1-3 hour forecast) ESRL/GSD/FAB is investigating the possibility of tornado forecast, what is needed and how to improve. Radar data assimilation is critical to severe storms and tornado forecasts.

35. Examples of LAPS/STMAS analysis impact LAPS Forecast: Jun. 16, 2002 LAPS hot-start & WRF-ARW Forecast (2km with Lin Micro-physics ): IHOP cases LAPS Forecast: Jun. 13, 2002 (Steve Albers and Isidora will talk these cases in more details at their presentations A well-balanced initial condition is the key for improving very short range forecast for tornados.

36. Windsor tornado case, 22 May 2008 Tornado touched down at Windsor, Colorado around 17:40 UTC, 22 May 2008 STMAS initialization 1.67 km 301 x 313 background model: RUC 13km, 17 UTC hot start (cloud analysis) Boundary conditions: RUC 13km, 3-h RUC forecast (initialized at 15 UTC) WRF-ARW 1.67 km, 1-h forecast Thompson microphysics Postprocessing: Reflectivity

37. 00-01hr 800mb wind initialized at 17 UTC 22 May 2005, STMAS analysis vs. WRF forecast (STMAS)

38. 00-01hr wind cross-section initialized at 17 UTC 22 May 2005, STMAS analysis vs. WRF forecast (STMAS)

39. 00-01hr 800mb reflectivity initialized at 17 UTC 22 May 2005, mosiac radar vs. WRF forecast (STMAS)

40. 00-01hr reflectivity cross-section initialized at 17 UTC 22 May 2005, mosiac radar vs. WRF forecast (STMAS)

41. 00-01hr reflectivity cross-section initialized at 17 UTC 22 May 2005, WRF forecast, RUC vs. STMAS

42. Future Study Investigating the initialization of STMAS for WRF forecast model; Assimilating radar reflectivity data into the well-balance analysis of STMAS; STMAS rapid update cycle; Satellite radiance and GPS impact on severe storm and tornado forecasts; Possible OSSE system for severe storms.

43. Summary Variational data assimilation requires further improvement, particularly the adjoint systems; Effectively combining variational data assimilation techniques with EnKF or particle filter for nonlinear and non-Gaussian data assimilation for severe weather forecasts; Improving radar, satellite data assimilation, forward operators and efficiency.

44. OAR/ESRL/GSD/Forecast Applications Branch

45. 45 First Assumption in a 3DVAR In most cases, DA has two information sources: Background and observation; A standard 3DVAR treats the background ( x b ) and observation ( x o ) as random representations of the true atmosphere ( x t ); Assumption I: x b -x t and x o -x t are random variables.

46. 46 The Bayesian Theorem A simplified version: P a (x)  P(x=x t | x=x o ) = P(x=x o | x=x t ) P(x=x t ); P(x=x o | x=x t ) = P(x o =x t ) = P o (x-x o ), i.e., observational error probability. Assumption II. P(x=x t ) = P b (x-x b ), as this prior probability can depend our knowledge of the background on the true atmosphere, see Lorenc 86. Then we have: P a (x)= P o (x-x o ) P b (x-x b ).

47. 47 Derivation of a 3DVAR I Assumption III. The background and observation error probabilities follow a Gaussian distribution: P o (x-x o )  exp[-(x-x o ) T O -1 (x-x o )] P b (x-x b )  exp[-(x-x b ) T B -1 (x-x b )] Assumption IV. Covariances O and B are known!

48. 48 To maximize the probability for estimation of the To best approximate the true atmosphere x t, we maximize exp[-(x-x b ) T B -1 (x-x b )]+exp[-(x-x o ) T O -1 (x-x o )]. This is equivalent to minimize: (x-x o ) T B -1 (x-x o )+(x-x o ) T O -1 (x-x o ), the standard 3DVAR cost function. Optimality = maximized probability Derivation of a 3DVAR II

49. It is derived from a statistical analysis assuming the error’s distribution is Gaussian. It solves a variational problem: subject to a model constraint for 4DVAR Single 3-4DVAR approach 南京大学

50. Background Observation Longer wave Background Observation Longer wave Resolvable Information for a Given Observation Network Difference on longer waveDifference on shorter wave 南京大学

51. STMAS STMAS is implemented in two steps. 1.It retrieves the resolvable observation information. 2.After the resolvable information retrieved, STMAS is reduced to a standard statistical variational analysis With long waves retrieved, STMAS deals with a localized error covariance, a banded matrix, at its last phase of analysis. 南京大学