1. Takemasa Miyoshi Numerical Prediction Division, JMA
2007/10/8
Research activities of the local ensemble transform Kalman filter (LETKF) at JMA
Takemasa Miyoshi
Numerical Prediction Division, JMA

2. Role of data assimilation
Major Methodologies in Atmos & Oceanic Sciences
Observing studies
©Vaisala
Modeling studies
Data Assimilation
Initialize model predictions; essential in NWP
Observing scientists get dynamically consistent 4-D analysis
Modeling scientists get information of real nature
Synergy between two major fields helps scientific advancement

3. Data assimilation in NWP
Numerical Weather Prediction
FCST
FCST
ANL
ANL
ANL
Model Simulation
OBS
OBS
FCST-ANALYSIS Cycle accumulates past observations
True state (unknown)
time

4. Data Assimilation = Analysis
FCST
FCST
FCST Error
ANL
ANL Error
True state (unknown)
The error in the initial condition (ANL) grows in a chaotic system
OBS Error
OBS

5. Probabilistic view OBS w/ errors ANL w/ errors FCST w/ errors P
Problem：
d.o.f. of the system: ~O(106)
d.o.f. of the error: even Gaussian distribution has d.o.f. of the covariance ~O(1012)
Too large to express explicitly
ANL w/ errors
T=t0
T=t1

6. EnKF = ensemble fcst. + ensemble update
A schematic of EnKF
Obs.
Analysis Ens. mean
FCST Ens. mean P
Generate ensemble members best representing the analysis errors
An initial condition with errors
T=t0
T=t1
T=t2
EnKF = ensemble fcst. + ensemble update

7. EnKF - summary
EnKF considers flow-dependent error structures, or the “errors of the day”
“advanced” data assimilation method
4D-Var is also an “advanced” method. How different?
EnKF analyzes the analysis errors in addition to analysis itself
“ideal” ensemble perturbations

8. Research activities MISSION STATEMENT
Develop a next-generation data assimilation system to improve operational NWP at JMA
Path to operations
Develop and test LETKF (Hunt 2005; Hunt et al. 2007; Ott et al. 2004, Hunt et al. 2004) with the Earth Simulator
Develop LETKF with the JMA nonhydrostatic model
Develop LETKF with the JMA global model
Assess LETKF under the quasi-operational setup
Researches using the Earth Simulator LETKF system
Experimental 1.5-yr reanalysis (ALERA)
Collaborative work with observing scientists
LETKF with the Earth Simulator coupled atmos-ocean GCM

9. Outline Developments of LETKF toward operations
Recent improvements
Quasi-operational comparison with 4D-Var
Probabilistic forecast skills
Research projects with the Earth Simulator
Experimental Ensemble Reanalysis: ALERA
Collaboration with observing scientists

10. Developments toward operations

11. LETKF Two categories of the EnKF (Ensemble Kalman Filter)
LETKF (Local Ensemble Transform Kalman Filter)
is a kind of ensemble square root filter (SRF)
is efficient with the parallel architecture
Perturbed observation (PO) method
Square root filter (SRF)
Classical
Relatively new
Already in operations (Canadian EPS)
Not in operations yet
Additional sampling errors by PO
No such additional sampling errors

12. LETKF developments at JMA
LETKF (Local Ensemble Transform Kalman Filter) has been applied to 3 models
AFES (AGCM for the Earth Simulator)
NHM (JMA nonhydrostatic model)
GSM (JMA global spectral model)
Miyoshi and Yamane, 2007: Mon. Wea. Rev., in press.
Miyoshi, Yamane, and Enomoto, 2007: SOLA,
Miyoshi and Aranami, 2006: SOLA,
Miyoshi and Sato, 2007: SOLA,

13. Quasi-operational Experimental System
4D-Var
LETKF
Ensemble forecast
9hr TL159/L40
Deterministic forecast
9hr TL319/L40 QC QC
4D-Var

14. Recent improvements Assimilation of satellite radiances
greatly improves the analysis accuracy
Removing local patches
solves the discontinuity problem near the Poles
Efficient MPI parallel implementation
solves the load imbalance problem
accelerates by a factor of 3
Adaptive satellite bias correction
a new idea analogous to the variational bias correction
showing great positive impact
Miyoshi and Sato, 2007: SOLA,
Miyoshi et al., 2007: SOLA,
about 30% faster than operational 4D-Var with similar settings

15. Impact by satellite radiances
RMSE and bias against radiosondes
Reduced RMSE of Z in mid-upper troposphere ( hPa), especially in the SH and Tropics
Blue: w/o satellite radiances
Red: w/ satellite radiances
Reduced negative bias of Z and T
20 members

16. Typhoon track ensemble prediction
Bred Vectors w/ 4D-Var
(Currently in operations)
Typhoon #13, 2004

17. Typhoon track ensemble prediction
SV w/ 4D-Var
(Next operational system)
Typhoon #13, 2004

18. Typhoon track ensemble prediction
LETKF (Developing)
Typhoon #13, 2004

19. This was a very difficult typhoon case
Best track
Operational Systems as of Aug 2004
LETKF

20. Statistical typhoon track errors
Typhoons in August 2004

21. Satellite radiance bias correction
Observation has a bias
Air mass bias (dependent on atmospheric state)
Scan bias (constant)
Coefficients of predictors are estimated statistically
Statistically estimated offline
Zenith angle
Surface temperature
Constant
etc.

22. Adaptive bias correction
Coefficients would change partly due to the deterioration of sensors
Allow temporal variation of the coefficients
using data assimilation
Variational bias correction (e.g., Dee 2003; Sato 2007)
Find the minimizer b of the cost function J through the variational procedure

23. Adaptive bias correction with LETKF
Analytical solution of the variational problem: minimizer (x, b)
Adaptive bias correction with LETKF
1. Solve the LETKF data assimilation problem first
difference
2. Solve the equation for b explicitly
This coincides with the variational BC

24. Time series of bias coefficients
AMSU-A 4ch (sensitive to middle-lower troposphere) indicates significant drift from those estimated by
4D-Var
AMSU-A 6ch (sensitive to upper troposhere) and other sensors/channels indicate no significant drift

25. Impact by adaptive bias correction
24hr temperature forecast error improvements relative to 4D-Var NH
RED: LETKF is better
BLUE: 4D-Var is better
Tropics
Apply Adaptive BC SH

26. T850 forecast bias against initial condition
Bias reduction
T850 forecast bias against initial condition
Global
Red: LETKF
Blue: 4D-Var NH
Apply Adaptive BC
Tropics
The improvements would be due to the bias reduction SH

27. Look at other variablesZ U
Daily variations of 24-hr forecast error improvements relative to 4D-Var
Horizontal axis: Date
Red: LETKF is better
Blue: 4D-Var is better
Global NH
Almost everything in the SH
Temperature below 800hPa
Temperature above 700 hPa except SH
Wind except SH
Tropics SH

28. With adaptive satellite bias correctionZ U
Daily variations of 24-hr forecast error improvements relative to 4D-Var
Horizontal axis: Date
Red: LETKF is better
Blue: 4D-Var is better
Global NH
Tropics SH

29. Improvement (%) relative to 4D-Var
Apply adaptive bias correction

30. Improvement (%) relative to 4D-Var
August 2004
December 2005
LETKF is advantageous in the summer hemisphere

31. Comparison with 3D-Var AC Initial conditions Radiosondes NH Tropics SH
RMSE
BIAS
RMSE
BIAS NH
Tropics SH
Red: LETKF
Blue: 3D-Var
Z500

32. Computation of LETKF is reasonably fast, good for the operational use.
Computational time
LETKF
4D-Var
11 min x 60 nodes
17 min x 60 nodes
5 min for LETKF
6 min for 9-hr ensemble forecasts
TL319/L60/M50
Inner: T159/L60
Outer: TL959/L60
Estimated for a proposed next generation operational condition
6 min (measured) x 8 nodes for LETKF with TL159/L40/M50
Computation of LETKF is reasonably fast, good for the operational use.

33. Probabilistic forecast skills
High T > TCLM + 2K
LETKF is advantageous up to 3-day forecasts

34. Probabilistic forecast skills
Low T < TCLM - 2K
LETKF is more sensitively affected by the negative temperature model bias

35. Economic value
High T > TCLM + 2K

36. Economic value
Low T < TCLM - 2K

37. Summary
LETKF indicated identical performance in the NH to the operational 4D-Var
However, LETKF is clearly worse than 4D-Var in the SH
Larger bias in LETKF; why?
LETKF is advantageous in Typhoon prediction
Probabilistic forecast skill is generally improved
Still bias problem

38. Researches with the Earth Simulator

39. Ensemble Reanalysis: ALERA
T159/L48 AFES (AGCM for the ES)
LETKF w/ 40 members
Assimilate real observations used in the JMA operational global analysis, except for satellite radiances
Reanalysis from May 2005 through February 2007
ALERA
(AFES-LETKF Experimental Ensemble Reanalysis)

40. Stable performance Very Stable! (compared to NCEP/NCAR)
System change: vertical localization of ps obs.

41. Snapshot (SLP)
ALERA
NNR
ALERA analysis is almost identical to NNR.

42. Zonal mean winds in JJA
ALERA
NNR

43. Comparison with observations
Radiosondes
AIRS retrievals

44. Compared with radiosondesU Z
Radiosondes
Radiosondes

45. 予報スコア
Adapted from Miyoshi and Yamane (2007)

46. (AFES-LETKF Experimental Ensemble Reanalysis)
ALERA dataset
ALERA
(AFES-LETKF Experimental Ensemble Reanalysis)
data are now available online for free!!
Contents
Ensemble reanalysis dataset for over 1.5 years since May 1, 2005
40 ensemble members
ensemble mean
ensemble spread
Available ‘AS-IS’ for free ONLY for research purposes
Any feedback is greatly appreciated.

47. Analyzing the analysis errors
EnKF provides not only analysis itself but also analysis errors (or uncertainties of the analysis)
What is dynamical meaning of the analysis errors?
GOES-9 Image
Courtesy of T. Enomoto

48. QBO and ensemble spread
Large spread at the initial stage of phase change
Courtesy of T. Enomoto

49. Stratospheric sudden warming
SPREAD
ALERA
NCEP/NCAR
Courtesy of T. Enomoto

50. Large spread in tropical lower wind
Courtesy of T. Enomoto

51. Tropical lower wind and the spread
Courtesy of T. Enomoto
SPREAD

52. Courtesy of T. Enomoto

53. Collaboration with observing scientists
There was an intensive observing project in the tropical western Pacific (a.k.a. PALAU-2005); the dropsonde obs during June 12-17, 2005 have been assimilated with the AFES-LETKF system.
Satellite image and dropsonde locations
Moteki et al., 2007: SOLA, accepted.

54. Collaboration with observing scientists
Impact by dropsonde observations
Ensemble spreads of the ensemble analysis by LETKF
NO OBS
DROPSONDES
Moteki et al., 2007: SOLA, accepted.

55. Collaboration with observing scientists
Propagation of observing signals
Faster propagation than the advection speed, the faster speed (~12 m/s) corresponds to Rossby wave propagation
Moteki et al., 2007: SOLA, accepted.

56. Summary

57. Development plans Quasi-operational comparison in winter months
Try to replace SV EPS with LETKF first
Developments for high-resolution deterministic forecasts
TL959 (about 20-km resolution) global system
Incremental method (analogous to incremental 4D-Var)
Improve skills in the SH (bias problem!)

58. Research plans with the ES
ALERA-2
5-yr reanalysis (21st century reanalysis)
AFES-MATSIRO (SiB) (atmos-land coupled)
More diagnostics (OLR, Precipitation, land variables, etc.; any requests?)
CFES-LETKF
LETKF with coupled atmos-land-ocean model
More observing projects (e.g., PALAU-2008)

59. Other research activities with LETKF
Collaboration with chemical-transport modeling scientists at the Meteorological Research Institute
LETKF has begun to work recently

60. Collaborators AFES-LETKF NHM-LETKF GSM-LETKF Chemical Transport
Prof. Shozo Yamane (Chiba Institute of Science and FRCGC/JAMSTEC, AFES)
Dr. Takeshi Enomoto (ESC/JAMSTEC, AFES)
Dr. Qoosaku Moteki (IORGC/JAMSTEC, Observing scientist)
NHM-LETKF
Kohei Aranami (NPD/JMA, NHM)
Dr. Hiromu Seko (MRI/JMA, DA with NHM)
GSM-LETKF
Yoshiaki Sato (NPD/JMA, staying at NCEP)
Takashi Kadowaki (NPD/JMA, 4D-Var)
Ryouta Sakai (NPD/JMA, EPS)
Munehiko Yamaguchi (NPD/JMA, Typhoon EPS)
Chemical Transport
Dr. Thomas Sekiyama (MRI/JMA, Chemical model)

61. Related publications AFES-LETKF
Miyoshi and Yamane, 2007: MWR, in press. (AFES-LETKF system)
Miyoshi et al., 2007: SOLA, 3, (LETKF without local patches)
Miyoshi et al., 2007: SOLA, 3, (ALERA)
Moteki et al., 2007: SOLA, accepted. (Observing project)
NHM-LETKF
Miyoshi and Aranami, 2006: SOLA, 2, (perfect model experiments)
GSM-LETKF
Miyoshi and Sato, 2007: SOLA, 3, (satellite radiance assimilation)

62. Thank you
for your attention!

63. This cycle process = EnKF
A core concept of EnKF
Complementary relationship between data assimilation and ensemble forecasting
FCST error
Data Assimilation
ANL Error
Ensemble Forecasting
This cycle process = EnKF
Analyze w/ the flow-dependent forecast error, ensemble forecast w/ initial ensemble reflecting the analysis error

64. Difference between EnKF and 3D-Var
Flow-dependent errors expand in low-dimensional subspace
Uniform error structure
“Errors of the day”
Analysis without flow-dependent error structure (e.g., 3D-Var)

65. An example of EnKF analysis accuracy
EnKF is advantageous to traditional data assimilation methods including 3D-Var, currently in operations at several NWP centers.
Many centers (ECMWF, JMA, UK MetOffice, MeteorFrance, Canada, etc.) switched to 4D-Var which also considers flow-dependent error structures.
30 ensemble members
3DVAR (31m)
LEKF (8m)
Serial EnSRF (5m)

66. EnKF vs. 4D-Var EnKF 4D-Var “advanced” method? Y Simple to code?
N (e.g., Minimizer)
Adjoint model? N
Observation operator
Only forward
(e.g., TC center)
Adjoint required
Asynchronous obs?
Y (4D-EnKF)
Y (intrinsic)
Initialization after analysis?
Analysis errors?
Y (ensemble ptb)
Limitation
ensemble size
Assim. window
EnKF with infinite ensemble size and 4D-Var with infinite window are equivalent.

67. KF and EnKF Kalman Filter Ensemble Kalman Filter Forecast equations
Ensemble forecasts
Approximated by
Kalman gain
[pxp] matrix inverse
Analysis equations
Solve for the ensemble mean
Ensemble perturbations

68. LETKF algorithm (Hunt, 2005, et al., 2007)
In the space spanned by
Eigenvalue decomposition:
Analysis equations
LETKF analysis
Ensemble analysis increments

69. Ensemble size 20  50 RMSE and bias against radiosondes
Blue: Operational 4D-Var
Red: 20-member LETKF
Green: 50-member LETKF
50 members > 20 members
Generally 4D-Var > LETKF
Exception: mid-upper tropospheric temperature in the SH
w/ satellite radiances

70. 500 hPa height forecast verifications
Red: LETKF Blue: 4D-Var
Global NH
Tropics SH

71. With adaptive satellite bias correction
Red: LETKF Blue: 4D-Var
Global NH
Tropics SH

72. QBO (time series of tropical wind)
ALERA
NNR

73. Stratospheric sudden warming
ALERA
NNR

74. Stratospheric sudden warming