Verification of Global Ensemble Forecasts Fanglin Yang Yuejian Zhu, Glenn White, John Derber Environmental Modeling Center National Centers for Environmental Prediction College Park, Maryland, USA GCWMB Bi-weekly Briefing, September 19,

A new Web Page: Verification of Ensemble Mean Forecasts The site displays verification statistics for global ensemble mean forecasts. It does not contain any verification of ensemble probabilistic forecast. For a complete list of ensemble verification products, please visit Ensemble Team Web and its Global Ensemble Verification Products.Ensemble Team Web Global Ensemble Verification Products The NWP models evaluated are: GFS: NCEP Global Deterministic Forecast; GEFSM: Mean of NCEP Global Ensemble Forecasts; CMCEM: Mean of Canadian Global Ensemble Forecasts; FENSM: Mean of US Navy Global Ensemble Forecasts; NAEFSM: Biased Corrected Mean of GEFS and CMCE; ECMWF ensemble is yet to be included in this verification page. 2

 Grid-to-Grid Verification Metrics: verified against each model’s own high-res deterministic analyses, including NH, SH, Tropics, Globe and PNA regions, up to 16 days of forecasts Anomaly correlation Bias RMS Murphy’s MSE Skill Scores Ratio of Standard Deviation Regional means of surface variables (T2m, RH2m, clm water, SLP etc)  Grid-to-Obs Verification Metrics: verified against surface and Rawinsonde observations, including bias and RMS, up to 7 days of forecasts Surface T2m, RH2m, 10 Wind, SLP against station obs, over the CONUS Upper air T, Q, RH, and Wind against RAOBS, over NH, SH, Tropics, the Globe and PNA.  Weather forecast maps 3

500-hPa Height Anomaly Correlation NH SH Ensembles GEFS, CMCE, and NAEFS have higher AC scores than GFS GEFS is better than CMCE and FENS 4

SLP Anomaly Correlation NH SH Biased corrected NAEFS is the best It seems the initial conditions of CMC ensembles are very different from its deterministic high-res analysis (why) 5

WIND RMSE vs Variance All ensembles have smaller RMSE than the deterministic GFS However, GFS maintains the best its variance. All ensembles lose variances with forecast leading time. Fields become much smoothers probably due to lower resolution and multi-member averaging. Note that for ensemble forecast, ensemble spread is a more meaningful metric. NH, 200hPa 6

WIND RMSE vs Variance Note that FENS performed quite well in the tropics GEFS tends to lose its variance quicker than others in the tropics Tropics, 850hPa 7

Comparison of A few near-surface forecast variables Tropics 10m U FENS has the strongest wind CMCE is the weakest NH T2m FENS is much warmer than others for daily high CMCE has the lowest daily low 8

Comparison of A few near-surface forecast variables Does CMCE has SSTs in the forecast hours set to its initial conditions? It seems there is no SST relaxation to its climatology ! SH T2m 9

Verification of T2m to Surface Observations over the CONUS West 10 GFS is too warm in the afternoon, while ECMWF (deterministic, 12-hourly) is too warm in the morning GEFS performed the best FENS is too warm in both day and night, while CMCE is too cold July28 – Sept 14, 2013

Verification of T2m to Surface Observations over the CONUS East 11 Again, FENS is too warm in both dayt and night Both GEFS and GFS are too cold in the afternoon While GFS is slightly too warm for daily high, GEFS is slightly too cold. Both CMCE and ECMWF are slightly too warm July28 – Sept 14, 2013

Verification of 10-m Wind to Surface Observations over the CONUS West 12 All models have weaker 10-m wind than the observed in the CONUS west, although they all captured the observed diurnal variation. CMCE has the slowest wind speed. July28 – Sept 14, 2013

Verification of Forecasts Against Rawinsonde Observations, d-6 Wind 13 Winds from all ensembles are too weak compared to the RAOBS Deterministic GFS and EMWF are also slightly weaker than the observed in the troposphere. Tropics NH

Forecast of Hurricane Humberto, IC

Colorado Flooding, 6-d forecasts from high-res deterministic models h rainfall, valid at h rainfall, valid at GFS too far north

Colorado Flooding, 3-d forecasts from high-res deterministic models h rainfall, valid at h rainfall, valid at good forecast

Colorado Flooding, 3-d ENS FCST, IC

Colorado Flooding, 7-d ENS FCST, IC