1. Exploring Multi-Model Ensemble Performance in Extratropical Cyclones over Eastern North America and the Western Atlantic Ocean Nathan Korfe and Brian A. Colle 1 School of Marine and Atmospheric Sciences, Stony Brook University, Stony Brook, NY

2. Difficulty Forecasting East Coast Cyclones Extratropical cyclones during the cool season have large societal impacts with heavy snow, mixed precipitation, damaging winds, and large storm surge Conveying confidence in weather forecasts at the medium range is challenging, especially for high impact weather Shift in the ensemble envelope causes high error for medium range (day 4-6) and even short range (day 1-3) forecasts Courtesy: Dan Petersen

3. Why Verify East Coast Cyclones? Many of the ensemble prediction systems (EPSs) used to forecast conditions along the East Coast and Western Atlantic have not been extensively verified for the medium range (72-144 h). – Charles and Colle (2009) focus on SREF – Froude et al. (2010) focus on full Northern Hemisphere Regional cyclone verification will be used to address the following motivational questions: 1.How well do the operational ensembles (NCEP, CMC, ECMWF) predict cool season extratropical cyclones with lead times from days 1-6? 2.What are the intensity biases in the ensembles? Where are the largest intensity errors located within domain? 3.What flow patterns are associated with the high error cyclone cases in the ensemble? http://wavy.somas.stonybrook.edu/cyclonetracks/

4. Cyclone Tracking Process / Data TIGGE: THORPEX Interactive Grand Global Ensemble (2007-2015) 20 member NCEP - GEFS: Global Ensemble Forecast System 50 member ECMWF: European Center for Medium-Range Weather Forecasts 20 member CMC: Canadian Meteorological Center Ensemble Control Members are included for statistical comparison Track ensemble, control, and reanalysis cyclones using Hodges (1995) surface cyclone tracking scheme – ECMWF ERA-Interim Re-Analysis is used to verify cyclone properties from October to March – All MSLP data is 1˚ x 1˚ resolution – Data is filtered to remove planetary scale effects (small wavenumber) and small scale effects (large wavenumber) – Cyclone tracking conditions: 24 h lifetime and 1000 km distance traveled (Colle et al. 2013) Preprocess Data: Bandpass Filter Hodges Cyclone Tracking Calculate Cyclone Intensity Cyclone Verification/ Matching Process Displacement/Intensity Error Calculations Mean Error Calculations Subset Tracks For Active Cyclone Regions

5. Cyclone Verification Criteria Two criteria need to be satisfied to successfully match the forecast and reanalysis cyclone (modified criteria from Froude et al. 2007) 1.The pairing distance d of a point in an individual forecast track to a point in the analysis track, which coincides in time with the analysis track, is less than the maximum pairing distance d max. d ≤ d max (d max = 1500 km) 2.At least T% of the points in the forecast track coincided in time with the analysis track and satisfied d ≤ d max. 100 × [2n M /(n A + n F )] ≥ T (T = 60%) Storm #1 Storm #2

6. Cyclone Displacement and Intensity MAE East Coast Cyclones ECMWF shows least amount of error across the East Coast with NCEP MAE’s increasing from hour 48-72 more significantly than ECMWF East Coast cyclones account for ~40% of cyclones in the domain (350 cases)

7. Brier Skill Scores – Day 4-6 Only events where model or observed values are below SLP threshold The BSS is calculated using the GEFS Control member as reference 1 indicates perfect probabilistic forecast compared with the reference score 0 indicates no improvement over the reference score East Coast CyclonesAll Domain Cyclones

8. Yearly Cyclone Intensity MAE ECMWF’s most recent winter seasons are below 8-year average MAE from short to medium range ECMWF NCEP Day 1-3 Day 4-6 CMC NCEP’s most recent winter seasons are near 8-year average MAE in medium range

9. Day 4-6 East Coast Cyclones NCEP Ensemble Mean Errors Much than EC NCEP MAE (1.3 stdev/7.2 mb) > ECMWF MAE (82 cases)ECMWF MAE (1.3 stdev/7.2 mb) > NCEP MAE (30 cases) Day 0

10. Average Cyclone Intensity Errors Day 4-6 CMC shows the highest domain wide error during the medium range ECMWF has lower intensity error magnitude than other EPSs – North Atlantic region – Hudson’s Bay CMC NCEP ECMWF

11. Cyclone Intensity ME – East Coast Cyclones ECMWF and NCEP mean shows the least intensity bias during the day 1-6 period with less variability than the CMC mean CMC mean has a slight overdeepening bias cyclones past hour 96 The number of data points decreases due to cyclone matching in medium/long range

12. Spatial ME Intensity and Direction – Day 4-6 NCEP and ECMWF show an overdeepening bias and a downstream directional bias in the eastern US CMC shows more underdeepening bias over the Mid-Atlantic region with notably less overdeepening bias near Hudson’s Bay than NCEP CMC NCEP ECMWF

13. Cyclone MAE/ME – Deep Cyclones ECMWF/NCEP cyclone intensity MAE is ~1 mb greater for deep cyclones compared to East Coast cyclones Intensity bias in the medium range (hour 96-144) is >1 mb for all EPSs past hour 96 Cyclone Intensity ME

14. ECMWF and NCEP Control Cyclone Density Difference Difference in track density amounts to 5-10% error where cyclones are underpredicted off the East Coast and near Hudson’s Bay Both NCEP and ECMWF tend to overpredict over northern New England ECMWF Control Day 4-6 NCEP Control Day 4-6

15. NCEP Large Error Events – Hour 96 Mean Positive SLP Error: 1.5 std dev (+7.6 mb) > mean Mean Negative SLP Error: 1.5 std dev (-7.9 mb) < mean

16. Underdeepened Cases Overdeepened Cases Day -4 Day -2 Day 0

17. Conclusions The ECMWF ensemble mean has the lowest ME and MAE scores and probabilistic skill (track and intensity) for cyclones in the US East Coast for days 3-7. – Short range MAE scores for NCEP are comparable to ECMWF The EC shows a steady improvement in cyclone forecast accuracy for days 4-6 over the last 8 years, but not NCEP and CMC. For days 4-6, all ensembles have similar intensity biases, such as overdeepening bias and a northeastward directional bias in the eastern US, while there are too few cyclones over the western Atlantic, and the relatively deep cyclones are more underdeepened. NCEP large error overdeepened cases show a pattern of a blocking high near Greenland, while underdeepened cases show a less amplified smaller-scale wave pattern at 500 hPa Some future work will focus on subsetting cyclone cases where the analysis lies outside the ensemble envelope and identifying flow regimes associated with these cases.

18. EXTRA SLIDES

19. ERA-Interim Track Density (2007-2015)

20. Average Cyclone Displacement Errors Day 4-6 CMC shows the highest domain wide error during the medium range ECMWF has lower intensity error magnitude than other EPSs – North Atlantic region – Hudson’s Bay CMC NCEP ECMWF

21. Cyclone Displacement/Intensity MAE All Cyclones ECMWF shows smallest errors across the entire domain with NCEP/CMC showing similar error growth from hour 24 to hour 96 Shorter lived, weak cyclones outnumber the more intense cyclones

22. Miller A Cyclone Intensity ME NCEP mean underdeepens cyclones by 0.75 hPa to 1.25 hPa from hour 72 to 120 CMC mean underdeepens cyclones in short range and overdeepens cyclones in the medium range ECMWF mean closely follows the multi- model mean from hour 48 to 120 The range of data points varies from 1,600 in the short range to 1,000 in medium range