Takemasa Miyoshi Numerical Prediction Division, JMA 2007/10/8 Research activities of the local ensemble transform Kalman filter (LETKF) at JMA Takemasa Miyoshi Numerical Prediction Division, JMA

Role of data assimilation Major Methodologies in Atmos & Oceanic Sciences Observing studies ©Vaisala Modeling studies Data Assimilation Initialize model predictions; essential in NWP Observing scientists get dynamically consistent 4-D analysis Modeling scientists get information of real nature Synergy between two major fields helps scientific advancement

Data assimilation in NWP Numerical Weather Prediction FCST FCST ANL ANL ANL Model Simulation OBS OBS FCST-ANALYSIS Cycle accumulates past observations True state (unknown) time

Data Assimilation = Analysis FCST FCST FCST Error ANL ANL Error True state (unknown) The error in the initial condition (ANL) grows in a chaotic system OBS Error OBS

Probabilistic view OBS w/ errors ANL w/ errors FCST w/ errors P Problem： d.o.f. of the system: ~O(106) d.o.f. of the error: even Gaussian distribution has d.o.f. of the covariance ~O(1012) Too large to express explicitly ANL w/ errors T=t0 T=t1

EnKF = ensemble fcst. + ensemble update A schematic of EnKF Obs. Analysis Ens. mean FCST Ens. mean P Generate ensemble members best representing the analysis errors An initial condition with errors T=t0 T=t1 T=t2 EnKF = ensemble fcst. + ensemble update

EnKF - summary EnKF considers flow-dependent error structures, or the “errors of the day” “advanced” data assimilation method 4D-Var is also an “advanced” method. How different? EnKF analyzes the analysis errors in addition to analysis itself “ideal” ensemble perturbations

Research activities MISSION STATEMENT Develop a next-generation data assimilation system to improve operational NWP at JMA Path to operations Develop and test LETKF (Hunt 2005; Hunt et al. 2007; Ott et al. 2004, Hunt et al. 2004) with the Earth Simulator Develop LETKF with the JMA nonhydrostatic model Develop LETKF with the JMA global model Assess LETKF under the quasi-operational setup Researches using the Earth Simulator LETKF system Experimental 1.5-yr reanalysis (ALERA) Collaborative work with observing scientists LETKF with the Earth Simulator coupled atmos-ocean GCM

Outline Developments of LETKF toward operations Recent improvements Quasi-operational comparison with 4D-Var Probabilistic forecast skills Research projects with the Earth Simulator Experimental Ensemble Reanalysis: ALERA Collaboration with observing scientists

Developments toward operations

LETKF Two categories of the EnKF (Ensemble Kalman Filter) LETKF (Local Ensemble Transform Kalman Filter) is a kind of ensemble square root filter (SRF) is efficient with the parallel architecture Perturbed observation (PO) method Square root filter (SRF) Classical Relatively new Already in operations (Canadian EPS) Not in operations yet Additional sampling errors by PO No such additional sampling errors

LETKF developments at JMA LETKF (Local Ensemble Transform Kalman Filter) has been applied to 3 models AFES (AGCM for the Earth Simulator) NHM (JMA nonhydrostatic model) GSM (JMA global spectral model) Miyoshi and Yamane, 2007: Mon. Wea. Rev., in press. Miyoshi, Yamane, and Enomoto, 2007: SOLA, 45-48. Miyoshi and Aranami, 2006: SOLA, 128-131. Miyoshi and Sato, 2007: SOLA, 37-40.

Quasi-operational Experimental System 4D-Var LETKF Ensemble forecast 9hr TL159/L40 Deterministic forecast 9hr TL319/L40 QC QC 4D-Var

Recent improvements Assimilation of satellite radiances greatly improves the analysis accuracy Removing local patches solves the discontinuity problem near the Poles Efficient MPI parallel implementation solves the load imbalance problem accelerates by a factor of 3 Adaptive satellite bias correction a new idea analogous to the variational bias correction showing great positive impact Miyoshi and Sato, 2007: SOLA, 37-40. Miyoshi et al., 2007: SOLA, 89-92. about 30% faster than operational 4D-Var with similar settings

Impact by satellite radiances RMSE and bias against radiosondes Reduced RMSE of Z in mid-upper troposphere (500-100hPa), especially in the SH and Tropics Blue: w/o satellite radiances Red: w/ satellite radiances Reduced negative bias of Z and T 20 members

Typhoon track ensemble prediction Bred Vectors w/ 4D-Var (Currently in operations) Typhoon #13, 2004

Typhoon track ensemble prediction SV w/ 4D-Var (Next operational system) Typhoon #13, 2004

Typhoon track ensemble prediction LETKF (Developing) Typhoon #13, 2004

This was a very difficult typhoon case Best track Operational Systems as of Aug 2004 LETKF

Statistical typhoon track errors Typhoons in August 2004

Satellite radiance bias correction Observation has a bias Air mass bias (dependent on atmospheric state) Scan bias (constant) Coefficients of predictors are estimated statistically Statistically estimated offline Zenith angle Surface temperature Constant etc.

Adaptive bias correction Coefficients would change partly due to the deterioration of sensors Allow temporal variation of the coefficients using data assimilation Variational bias correction (e.g., Dee 2003; Sato 2007) Find the minimizer b of the cost function J through the variational procedure

Adaptive bias correction with LETKF Analytical solution of the variational problem: minimizer (x, b) Adaptive bias correction with LETKF 1. Solve the LETKF data assimilation problem first difference 2. Solve the equation for b explicitly This coincides with the variational BC

Time series of bias coefficients AMSU-A 4ch (sensitive to middle-lower troposphere) indicates significant drift from those estimated by 4D-Var AMSU-A 6ch (sensitive to upper troposhere) and other sensors/channels indicate no significant drift

Impact by adaptive bias correction 24hr temperature forecast error improvements relative to 4D-Var NH RED: LETKF is better BLUE: 4D-Var is better Tropics Apply Adaptive BC SH

T850 forecast bias against initial condition Bias reduction T850 forecast bias against initial condition Global Red: LETKF Blue: 4D-Var NH Apply Adaptive BC Tropics The improvements would be due to the bias reduction SH

Look at other variables Z U Daily variations of 24-hr forecast error improvements relative to 4D-Var Horizontal axis: Date Red: LETKF is better Blue: 4D-Var is better Global NH Almost everything in the SH Temperature below 800hPa Temperature above 700 hPa except SH Wind except SH Tropics SH

With adaptive satellite bias correction Z U Daily variations of 24-hr forecast error improvements relative to 4D-Var Horizontal axis: Date Red: LETKF is better Blue: 4D-Var is better Global NH Tropics SH

Improvement (%) relative to 4D-Var Apply adaptive bias correction

Improvement (%) relative to 4D-Var August 2004 December 2005 LETKF is advantageous in the summer hemisphere

Comparison with 3D-Var AC Initial conditions Radiosondes NH Tropics SH RMSE BIAS RMSE BIAS NH Tropics SH Red: LETKF Blue: 3D-Var Z500

Computation of LETKF is reasonably fast, good for the operational use. Computational time LETKF 4D-Var 11 min x 60 nodes 17 min x 60 nodes 5 min for LETKF 6 min for 9-hr ensemble forecasts TL319/L60/M50 Inner: T159/L60 Outer: TL959/L60 Estimated for a proposed next generation operational condition 6 min (measured) x 8 nodes for LETKF with TL159/L40/M50 Computation of LETKF is reasonably fast, good for the operational use.

Probabilistic forecast skills High T > TCLM + 2K LETKF is advantageous up to 3-day forecasts

Probabilistic forecast skills Low T < TCLM - 2K LETKF is more sensitively affected by the negative temperature model bias

Economic value High T > TCLM + 2K

Economic value Low T < TCLM - 2K

Summary LETKF indicated identical performance in the NH to the operational 4D-Var However, LETKF is clearly worse than 4D-Var in the SH Larger bias in LETKF; why? LETKF is advantageous in Typhoon prediction Probabilistic forecast skill is generally improved Still bias problem

Researches with the Earth Simulator

Ensemble Reanalysis: ALERA T159/L48 AFES (AGCM for the ES) LETKF w/ 40 members Assimilate real observations used in the JMA operational global analysis, except for satellite radiances Reanalysis from May 2005 through February 2007 ALERA (AFES-LETKF Experimental Ensemble Reanalysis)

Stable performance Very Stable! (compared to NCEP/NCAR) System change: vertical localization of ps obs.

Snapshot (SLP) ALERA NNR ALERA analysis is almost identical to NNR.

Zonal mean winds in JJA ALERA NNR

Comparison with observations Radiosondes AIRS retrievals

Compared with radiosondes U Z Radiosondes Radiosondes

予報スコア Adapted from Miyoshi and Yamane (2007)

(AFES-LETKF Experimental Ensemble Reanalysis) ALERA dataset ALERA (AFES-LETKF Experimental Ensemble Reanalysis) data are now available online for free!! http://www3.es.jamstec.go.jp/ Contents Ensemble reanalysis dataset for over 1.5 years since May 1, 2005 40 ensemble members ensemble mean ensemble spread Available ‘AS-IS’ for free ONLY for research purposes Any feedback is greatly appreciated.

Analyzing the analysis errors EnKF provides not only analysis itself but also analysis errors (or uncertainties of the analysis) What is dynamical meaning of the analysis errors? GOES-9 Image Courtesy of T. Enomoto

QBO and ensemble spread Large spread at the initial stage of phase change Courtesy of T. Enomoto

Stratospheric sudden warming SPREAD ALERA NCEP/NCAR Courtesy of T. Enomoto

Large spread in tropical lower wind Courtesy of T. Enomoto

Tropical lower wind and the spread Courtesy of T. Enomoto SPREAD

Courtesy of T. Enomoto

Collaboration with observing scientists There was an intensive observing project in the tropical western Pacific (a.k.a. PALAU-2005); the dropsonde obs during June 12-17, 2005 have been assimilated with the AFES-LETKF system. Satellite image and dropsonde locations Moteki et al., 2007: SOLA, accepted.

Collaboration with observing scientists Impact by dropsonde observations Ensemble spreads of the ensemble analysis by LETKF NO OBS DROPSONDES Moteki et al., 2007: SOLA, accepted.

Collaboration with observing scientists Propagation of observing signals Faster propagation than the advection speed, the faster speed (~12 m/s) corresponds to Rossby wave propagation Moteki et al., 2007: SOLA, accepted.

Summary

Development plans Quasi-operational comparison in winter months Try to replace SV EPS with LETKF first Developments for high-resolution deterministic forecasts TL959 (about 20-km resolution) global system Incremental method (analogous to incremental 4D-Var) Improve skills in the SH (bias problem!)

Research plans with the ES ALERA-2 5-yr reanalysis (21st century reanalysis) AFES-MATSIRO (SiB) (atmos-land coupled) More diagnostics (OLR, Precipitation, land variables, etc.; any requests?) CFES-LETKF LETKF with coupled atmos-land-ocean model More observing projects (e.g., PALAU-2008)

Other research activities with LETKF Collaboration with chemical-transport modeling scientists at the Meteorological Research Institute LETKF has begun to work recently

Collaborators AFES-LETKF NHM-LETKF GSM-LETKF Chemical Transport Prof. Shozo Yamane (Chiba Institute of Science and FRCGC/JAMSTEC, AFES) Dr. Takeshi Enomoto (ESC/JAMSTEC, AFES) Dr. Qoosaku Moteki (IORGC/JAMSTEC, Observing scientist) NHM-LETKF Kohei Aranami (NPD/JMA, NHM) Dr. Hiromu Seko (MRI/JMA, DA with NHM) GSM-LETKF Yoshiaki Sato (NPD/JMA, staying at NCEP) Takashi Kadowaki (NPD/JMA, 4D-Var) Ryouta Sakai (NPD/JMA, EPS) Munehiko Yamaguchi (NPD/JMA, Typhoon EPS) Chemical Transport Dr. Thomas Sekiyama (MRI/JMA, Chemical model)

Related publications AFES-LETKF Miyoshi and Yamane, 2007: MWR, in press. (AFES-LETKF system) Miyoshi et al., 2007: SOLA, 3, 89-92. (LETKF without local patches) Miyoshi et al., 2007: SOLA, 3, 45-48. (ALERA) Moteki et al., 2007: SOLA, accepted. (Observing project) NHM-LETKF Miyoshi and Aranami, 2006: SOLA, 2, 128-131. (perfect model experiments) GSM-LETKF Miyoshi and Sato, 2007: SOLA, 3, 37-40. (satellite radiance assimilation)

Thank you for your attention!

This cycle process = EnKF A core concept of EnKF Complementary relationship between data assimilation and ensemble forecasting FCST error Data Assimilation ANL Error Ensemble Forecasting This cycle process = EnKF Analyze w/ the flow-dependent forecast error, ensemble forecast w/ initial ensemble reflecting the analysis error

Difference between EnKF and 3D-Var Flow-dependent errors expand in low-dimensional subspace Uniform error structure “Errors of the day” Analysis without flow-dependent error structure (e.g., 3D-Var)

An example of EnKF analysis accuracy EnKF is advantageous to traditional data assimilation methods including 3D-Var, currently in operations at several NWP centers. Many centers (ECMWF, JMA, UK MetOffice, MeteorFrance, Canada, etc.) switched to 4D-Var which also considers flow-dependent error structures. 30 ensemble members 3DVAR (31m) LEKF (8m) Serial EnSRF (5m)

EnKF vs. 4D-Var EnKF 4D-Var “advanced” method? Y Simple to code? N (e.g., Minimizer) Adjoint model? N Observation operator Only forward (e.g., TC center) Adjoint required Asynchronous obs? Y (4D-EnKF) Y (intrinsic) Initialization after analysis? Analysis errors? Y (ensemble ptb) Limitation ensemble size Assim. window EnKF with infinite ensemble size and 4D-Var with infinite window are equivalent.

KF and EnKF Kalman Filter Ensemble Kalman Filter Forecast equations Ensemble forecasts Approximated by Kalman gain [pxp] matrix inverse Analysis equations Solve for the ensemble mean Ensemble perturbations

LETKF algorithm (Hunt, 2005, et al., 2007) In the space spanned by Eigenvalue decomposition: Analysis equations LETKF analysis Ensemble analysis increments

Ensemble size 20  50 RMSE and bias against radiosondes Blue: Operational 4D-Var Red: 20-member LETKF Green: 50-member LETKF 50 members > 20 members Generally 4D-Var > LETKF Exception: mid-upper tropospheric temperature in the SH w/ satellite radiances

500 hPa height forecast verifications Red: LETKF Blue: 4D-Var Global NH Tropics SH

With adaptive satellite bias correction Red: LETKF Blue: 4D-Var Global NH Tropics SH

QBO (time series of tropical wind) ALERA NNR

Stratospheric sudden warming ALERA NNR

Stratospheric sudden warming