NCEP CMC ECMWF MEAN ANA BRIAN A COLLE MINGHUA ZHENG │ EDMUND K. CHANG Applying Fuzzy Clustering Analysis to Assess Uncertainty and Ensemble System Performance to Cool Season High-Impact Weather 2015 Northeast Winter Storm Verifying at 0000 UTC 25 Jan Day forecast

Should forecasters utilize ensembles beyond ensemble mean/spread, probs, and anomalies? Some Key issues: 1. Forecasters need ensemble tools to condense useful information from ensembles to understand predictability issues in the forecast. 2. Forecasters need more guidance about potential storm scenarios, biases, and outliers, especially for high impact weather events 3. Important, since ensembles can fail for major weather events, and we need to quantify more when and how this happens. Tools and analysis: Ensemble sensitivity analysis (see Zheng et al. 2014). Apply an efficient ensemble tool (fuzzy clustering analysis) to quickly separate forecast scenarios among the large ensemble set. Evaluate different ensemble models’ performance in forecasting winter storms over East Coast for medium-range forecast and find out on average which model is more reliable when expecting a severe storm. Motivation

ECMWF (50-member) mean (shaded in mm) and spread (contoured) for the day-7 24-h Precipitation Initialized 1200 UTC 16 December 2013

EOF1 Pattern for the day 7 Precipitation and Analyzed Precipitation (in mm) (See Zheng et al for Ensemble Sensi details)

Ensemble Sensitivity Based on EOF1 Pattern

Forecasters likely want to know the various forecast scenarios: Our attempt using Fuzzy Clustering Data: - TIGGE Ensemble forecast data: NCEP (20 mem) + CMC (20 mem) + ECMWF (50 mem) - Analysis: NCEP operational analysis - Variables: MSLP and Z500 - Historical cases selections: 124 (114 for region 2 – East coast) cyclone cases (minimum pressure <996 hPa) from 2007 to 2014 cool seasons (NDJFM) using Hodges cyclone tracker. Method: - Empirical Orthogonal Function (EOF) analysis - To quantify ensemble forecast variance patterns - Fuzzy clustering analysis - To group ensemble members based on EOF PCs (Harr et al. 2008)

Ensemble Means (Green = CMC, Red = NCEP, and Blue = EC: Valid at 1200 UTC 27 UTC January 2015

CPC 24-h Precipitation and day 3 ensemble mean precipitation for forecasts initialized at 1200 UTC 24 January 2015.

Case study: 3-Day forecast, IT: 1200 UTC 24 Jan 2015 UTC 00; VT: 1200 UTC 27 Jan 2015 MSLP spaghetti plot for 996 hPa at Jan 27 h Z Ensemble mean (contours) and spread (shades) of MSLP, [hPa]

STEP1: EOF analysis of MSLP on 90 members of forecasts at valid time (VT) EOF1 MSLP anomaly pattern, [hPa] EOF2 MSLP anomaly pattern, [hPa] 57.1% 23.8%

STEP2: group ensemble members into 5 clusters based on PCs using Fuzzy clustering scatter plots +PC1: Deeper +slight N -PC1: Weaker +slight S +PC2: West +slight deeper -PC2: East +slight weaker Group 5 Deeper+SW Group 2 Weaker+NE Group 3/ANA Shift to NE Group 4 Weaker+SW

STEP3: Pick up a contour line and plot group mean summary based on the partitions of clusters EM NE/Weaker(2) NE Shift (3) SW/Weaker(4) SW/Deeper(5) Analysis

STEP4: Look at ensemble sensitivity to know where upstream to look for any important differences between cluster solutions

STEP5: Look at the evolution of the clusters upstream and how they compare to analysis as they become available… EM NE/Weaker(2) NE Shift (3) SW/Weaker(4) SW/Deeper(5) Analysis Day 1 ForecastDay 1.5 Forecast

Historical evaluations using 124 (114) cyclone cases NCEP/CMC/EC: 7/4/9 Day 3 forecast 124 cyclone cases for region cyclone cases for region 2 8 out-of-envelope or outlier cases are not included. Day 6 forecast Region 1Region 2

Percentage of each model ensemble’s members w.r.t. its total ensemble member falling into the Group ANA for region 2 Day 3 forecast Day 9 forecast

Percentage of each ensemble's members in Group ANA

RMSE and Spread relation (RMSE/SPRD) for PC1 and PC2 for regions 1 and 2 Over-dispersed Under-dispersed Over-dispersed Under-dispersed 1 is the best

Percentage of cases each model misses Group ANA

The ratio of cases with Group ANA the same as Group EM to the average cases ANA more likely than average to be in Group EM ANA less likely than average to be in Group EM

Conclusions Ensembles have difficulties with major cyclone events. We still need forecasters in the loop, but they need more tools. Case study of “2015 Northeast Winter Storm” shows that fuzzy clustering can quickly separate different forecast scenarios. For day 6 (9) forecasts, in about 7% (13%) of the cases, the analysis falls outside the envelope of the multi-model ensemble. ECMWF members have the highest probability to be included in the analysis group for 3 and 6 days forecast. The analysis is not more likely to be in the Ensemble mean group than other groups (except for day 3 forecast), suggesting that focusing on the ensemble mean may be misleading in many cases The RMSE/SPREAD relations show that ECMWF ensemble is less under- dispersed than NCEP and CMC for day 3 and day 6. For day 9, all individual models are severely under-dispersed. The multi-model ensemble is less under-dispersed than any individual models for day 6 and day 9.  This research is supported by NOAA CSTAR program.  Fuzzy clustering website: