1. 1 Presented at the 36 th Climate Prediction and Diagnostics Workshop Fort Worth, TX, 03–06 Oct 2011 15W (Talas) approaches Japan 01 Sep 2011 Improvements to, and Verification of, Intraseasonal to Seasonal Prediction of Tropical Cyclogenesis David Meyer and Tom Murphree Naval Postgraduate School dwmeyer@nps.edudwmeyer@nps.edu and murphree@nps.edumurphree@nps.edu 36 th CDPW, Oct11, murphree@nps.edu

2. 2 ** LSEFs = large scale environmental factors: SST, ζ 850, shear 200-850, div 200, f Inputs and outputs are all on a daily, 2.5° scale. Statistical model is a logistic regression model. Each LSEF influences TC formation at over a 99% confidence level Extensive ensembling using multiple initial conditions and lead times (e.g., 184 members for 90 day leads)  equivalent to Monte Carlo simulation. Long Lead Forecasting of Tropical Cyclone Formation in the Western North Pacific Produce statistical- dynamical model output: ensemble- based long lead forecasts of TC formation probabilities (NPS TC LLFs, 4-90 day lead times) Force statistical model with dynamical, ensemble-based, long lead forecasts of LSEFs** (use NCEP Climate Forecast System v1) Apply statistical model of TC formation probability (NPS logistic regression model) Build statistical model based on relationships between TC formations and LSEFs** (based on JTWC best track and NCEP R2 reanalysis data) NPS Statistical-Dynamical Forecast System TC LLF Brief, 13Sep11, murphree@nps.edumurphree@nps.edu

3. 3 Statistical Model Performance Verification 26 years of zero lead hindcasts for western North Pacific (WNP) for 1982-2007 show statistical model has skill: 1.Hindcasts evaluated at daily, 2.5° resolution 2.Proportion correct: 89% (671 of 759 TCs formed within a forecasted region) 3.Positive Brier skill score 4.Positive ROC skill score 5.Modeled probabilities match well with observed formations for: a.El Nino and La Nina b.Eight MJO phases 36 th CDPW, Oct11, murphree@nps.edu

4. 4 LTM TC Formation Probability LTM TC Formation Probability and Formations 36 th CDPW, Oct11, murphree@nps.edu 1.Probabilities from NPS statistical model forced with R1 LTM LSEFs. 2.Probabilities not calculated directly from observed TC formations. 3.LTM probabilities correspond well to LTM formations. Qualitative Statistical Model Verification: JASO LTM TC Formation Probability and Actual Formations

5. 5 EN 36 th CDPW, Oct11, murphree@nps.edu 1.Statistical model based probabilities using EN and LN LSEFs. 2.EN probabilities higher in central – southeastern WNP. 3.LN probabilities higher in northwestern, northern, and southwestern WNP. 4.Probabilities correspond well to observed formations. Qualitative Statistical Model Verification: JASO ENLN TC Formation Probability and Actual Formations El NinoLa Nina EN LN

6. 6 1 2 4 37 8 5 6 LTM LTM TC Formation Probability MJO TC Formation Probabilities, Phases 1-8 36 th CDPW, Oct11, murphree@nps.edu Qualitative Statistical Model Verification: JASO MJO TC Formation Probability and Actual Formations 1.Statistical model based probabilities using LSEFs for phases 1-8. 2.Large variations in probability patterns and magnitudes by phase.

7. 7 LTM LTM TC Formation Probability and Formations 1 2 4 37 8 5 6 MJO TC Formation Probabilities and Actual Formations, Phases 1-8 36 th CDPW, Oct11, murphree@nps.edu Qualitative Statistical Model Verification: JASO ENLN TC Formation Probability and Actual Formations Good agreement between statistically modeled probabilities and actual formations for phases 1-8.

8. Sample NPS TC Formation Forecast System Output: Probability of TC Formation at Intraseasonal Leads 36 th CDPW, Oct11, murphree@nps.edu 8 TC Formation Daily Probability Forecast Issued: 25 Aug 2011. Valid: 07-13 Sep 2011. 1.Weekly forecasts: average of 7 consecutive daily 21 day lead forecasts. 2.Forecast shown is an example of NPS inputs to CPC GTHB technical teleconference (but with subsequent addition of verifying formations in red). 3.Preliminary verification: from first position in JTWC TCFAs. 4.Updated verification: from JTWC best track data (released in following Mar-Apr).

9. Sample NPS TC Formation Forecast System Output: Probability of TC Formation at Intraseasonal Leads 36 th CDPW, Oct11, murphree@nps.edu 9 TC Formation Daily Probability Forecast Issued: 25 Aug 2011. Valid: 07-13 Sep 2011. 18W formed 10 Sep 11

10. Sample NPS TC Formation Forecast System Output: Probability of TC Formation at Intraseasonal Leads 36 th CDPW, Oct11, murphree@nps.edu 10 TC Formation Daily Probability Forecast Issued: 25 Aug 2011. Valid: 07-13 Sep 2011. 18W formed 10 Sep 1119W formed 12 Sep 11

11. 36 th CDPW, Oct11, murphree@nps.edu 11 Sample NPS TC Formation Forecast System Output: Probability of TC Formation at Zero Lead Storm that became a TD over land and quickly dissipated TC Formation Daily Probability Forecast Issued: 10 Sep 2011. Valid: 10 Sep 2011. 18W formed 10 Sep 1119W formed 12 Sep 11

12. Sample NPS TC Formation Forecast System Output: Probability of TC Formation at Two Month Leads 36 th CDPW, Oct11, murphree@nps.edu 12 TC Formation Daily Probability Forecast Issued: 17 July 2011. Valid: 07-13 Sep 2011. 18W formed 10 Sep 1119W formed 12 Sep 11

13. Forecast Verification Methods: Graphic Contingency Table YesNo Yes a = total no. of hitsb = total no. of weekly false alarms No c = total no. of weekly missesd = total weekly correct rejections Observed Forecasted 13 TC LLF Brief, 13Sep11, murphree@nps.edumurphree@nps.edu       

14. 14 Quantitative Forecasting System Skill Assessment, Week One and Week Two Outlooks, Jun-Nov 2010 PC: Proportion Correct. POD: Probability of Detection. FAR: False Alarm Ratio. HSS: Heidke skill score. ETS: Equitable Threat Score. PCA: proportion of WNP within forecast contours. FARN: False Alarm Ratio for forecasted non-formation 1.Good overall performance but FAR is higher than we would like. 2.Additional indicator of good skill: positive Brier skill scores for all leads. 3.Stable performance: similar performance in 2009 and 2011 (preliminary). 4.Similar skill at longer leads (21-90 days), but at a cost of reduced resolution and sharpness. 1.One and two week outlooks: same as weekly forecasts we now provide to CPC. 2.Verification done on weekly basis. 3.Neighborhood radius of 2.5° used on observations. 36 th CDPW, Oct11, murphree@nps.edu

15. 15 Quantitative Forecasting System Skill Assessment: Total Forecasted Area 1.Total marine area of the WNP is ~25 million square kilometers. 2.Almost all of this marine area is capable of TC formation during JASO. 36 th CDPW, Oct11, murphree@nps.edu

16. 16 Quantitative Forecasting System Skill Assessment: Total Forecasted Area 36 th CDPW, Oct11, murphree@nps.edu Red box represents: Total forecasted area in an average daily probability forecast 10% of the WNP marine area

17. 17 Qualitative Forecasting System Skill Assessment: Intraseasonal Forecasts - Peak Season 2010 vs. 2011 1.Highest formation probabilities and most TCs occurred west of 125E. 2.This probability pattern typical for much of 2010. 3.Formations typical for La Niña periods. 4.5 of 7 TCs successfully forecasted (2.5° neighborhood). 5.Monthly average probability locations and magnitudes similar to those in corresponding weekly forecasts. 36 th CDPW, Oct11, murphree@nps.edu Monthly Average of Daily Probability Forecasts 21 Day Lead. Valid: 16 Aug - 15 Sep 2010.

18. 18 36 th CDPW, Oct11, murphree@nps.edu Qualitative Forecasting System Skill Assessment: Intraseasonal Forecasts - Peak Season 2010 vs. 2011 Monthly Average of Daily Probability Forecasts 21 Day Lead. Valid: 16 Aug - 15 Sep 2011. 1.Highest formation probabilities and all TCs occurred east of 130E. 2.This probability pattern typical for much of 2011. 3.Northern formations may be indication of La Nina impacts. 4.7 of 7 TCs successfully forecasted (2.5° neighborhood). 5.Monthly average probability locations and magnitudes similar to those in corresponding weekly forecasts.

19. 19 NPS TC Forecasting System: Summary and Plans 1.Skill assessment: a.Skill evaluated over three years of forecasts, 2009-2011, with: 1.EN, LN, neutral, and MJO conditions 2.A total of 69 TCs b.Quantitative: System has skill over chance and/or climatology at all leads (out to 90 days) and at daily and weekly resolution c.Qualitative: System represents EN, LN, and MJO formation probability patterns 2.Statistical model being rebuilt based on CFSR and additional years 3.Reforecast 2011 and forecast 2012 with rebuilt statistical model and CFSv2 as dynamical component. 4.Expand forecast regions to N Atlantic and other basins (promising results from preliminary work). 36 th CDPW, Oct11, murphree@nps.edu

20. 20 NPS TC Forecasting System: Summary and Plans 5.Begin issuing experimental forecasts from corresponding TC intensity forecasting system. Preliminary WNP results are very promising. 6.Begin work on corresponding TC track forecasting system. 7.Generate new statistical model for off season forecasting 8.Continue coordination / collaboration with: CPC, FNMOC, JTWC, NHC 9.Continuing and expanding discussions with potential forecast customers. 36 th CDPW, Oct11, murphree@nps.edu

21. Tom Murphree, Ph.D. Research Associate Professor Department of Meteorology Naval Postgraduate School 254 Root Hall 589 Dyer Road Monterey, CA 93943-5114 831-656-2723 office 831-402-9603 cell 831-656-3061 fax murphree@nps.edu Long Lead Forecasting of Tropical Cyclones: Contact Information 21 David Meyer Operations Research Analyst Department of Meteorology Naval Postgraduate School Monterey, CA 93943-5114 831-233-8438 cell/office dwmeyer@nps.edu 36 th CDPW, Oct11, murphree@nps.edu

22. 22

23. 23 SST Relative Vorticity 850 hPa Divergence 200 hPa Vertical Wind Shear, Zonal, 200-850 hPa LTM TC Formation Probability LTM LSEFs LTM TC Formation Probability and Formations 36 th CDPW, Oct11, murphree@nps.edu Vertical Wind Shear, Meridional, 200-850 hPa 1.Probabilities from NPS statistical model forced with R1 LTM LSEFs, not directly from observed TC formations. 2.LTM probabilities consistent with LTM LSEFs. 3.LTM probabilities correspond well to LTM formations. Qualitative Statistical Model Verification: JASO LTM TC Formation Probability and Actual Formations

24. 24 ENLN TC Formation Probabilities and Formations LTM TC Formation Probabilities and Formations 36 th CDPW, Oct11, murphree@nps.edu LTM LN EN EN, LN, and LTM probabilities correspond well to observed formations. Qualitative Statistical Model Verification: JASO ENLN TC Formation Probability and Actual Formations

25. 25 Modeled TC Formation Probability Anomalies for EN and LN, Jul-Oct EN LN Regression model based probability anomalies for EN and LN: 1.Consistent with prior observational studies  model verification. 2.Forecast system has potential to represent EN-LN impacts. 3.Results useful in education/training and seasonal scale situational awareness for operational forecasters. 4.Useful products for long range planning. Qualitative Statistical Model Verification: JASO ENLN TC Formation Probability and Actual Formations

26. 26 Modeled TC Formation Probability Anomalies for MJO Phases 1-8, Jul-Oct 1 2 4 37 8 5 6 Regression model based probability anomalies for MJO phases 1-8: 1.Consistent with prior observational studies  model verification. 2.Forecast system has potential to represent MJO impacts. 3.Results useful in education and intraseasonal scale situational awareness for forecasters. 4.Useful products for long range planning. Qualitative Statistical Model Verification: JASO MJO TC Formation Probability and Actual Formations

27. Quantitative Forecast Verification Methods 1.Standard 2 x 2 contingency table verification metrics used to allow ready interpretation and comparison with other forecasts (e.g., CPC one and two week forecasts). 2.Verification done for both formation and non-formation forecasts. YesNo Yesab Nocd Observed Forecasted Proportion correct (PC) = (a+d)/(a+b+c+d) Basic proportion correct (BPC) = a/(a+b) Hit rate (HR) = probability of detection (POD) = a/(a+c) False alarm ratio (FAR) = b/(a+b) False alarm rate (FAR) = b/(b+d) Threat score (TS) = critical success index (CSI) = a/(a+b+c) Heidke skill score (HSS) = [2(ad-bc)] / [(a+c)(c+d) + (a+b)(b+d)] Cf. Wilks (2006). 27 36 th CDPW, Oct11, murphree@nps.edu

28. 28 Quantitative Forecasting System Skill Assessment, 30 Day Lead Forecasts, Jun-Nov 2010 PC: Proportion Correct, POD: Probability of Detection, FAR: False Alarm Ratio, HSS: Heidke Skill Score, ETS: Equitable Threat Score, PCA: proportion of WNP contoured by forecasts, FARN: False Alarm Ratio for forecasted non-formation 1.System performance stability: 2010 performance comparable to 2009 and preliminary 2011 results 2.Additional indicator of good skill: positive Brier Skill Scores 3.No comparable forecasts at these leads available from other sources 1.Performance metrics calculated on a weekly basis for consistency with CPC’s weekly GTH forecast 2.A 2.5° neighborhood was used on observations (radius) 3.Weekly outlooks created by averaging 7 days worth of daily 30 day lead forecasts

29. 29 Quantitative Forecasting System Skill Assessment, 60 Day Lead Forecasts, Jun-Nov 2010 PC: Proportion Correct, POD: Probability of Detection, FAR: False Alarm Ratio, HSS: Heidke Skill Score, ETS: Equitable Threat Score, PCA: proportion of WNP contoured by forecasts, FARN: False Alarm Ratio for forecasted non-formation 1.System performance stability: 2010 performance comparable to 2009 and preliminary 2011 results 2.Additional indicator of good skill: positive Brier Skill Scores 3.No comparable forecasts at these leads available from other sources 1.Performance metrics calculated on a weekly basis for consistency with CPC’s weekly GTH forecast 2.A 2.5° neighborhood was used on observations (radius) 3.Weekly outlooks created by averaging 7 days worth of daily 6- day lead forecasts

30. 30 Quantitative Forecasting System Skill Assessment, 90 Day Lead Forecasts, Jun-Nov 2010 PC: Proportion Correct, POD: Probability of Detection, FAR: False Alarm Ratio, HSS: Heidke Skill Score, ETS: Equitable Threat Score, PCA: proportion of WNP contoured by forecasts, FARN: False Alarm Ratio for forecasted non-formation 1.System performance stability: 2010 performance comparable to 2009 and preliminary 2011 results 2.Additional indicator of good skill: positive Brier Skill Scores 3.No comparable forecasts at these leads available from other sources 1.Performance metrics calculated on a weekly basis for consistency with CPC’s weekly GTH forecast 2.A 2.5° neighborhood was used on observations (radius) 3.Weekly outlooks created by averaging 7 days worth of daily 90 day lead forecasts

31. 31 Comparisons of Forecasts to TC Formations and TC Invests 31 TC LLF Brief, 13Sep11, murphree@nps.edumurphree@nps.edu 14W formed 19 Aug 2011 15W formed 22 Aug 2011  Invest 95W formed 18 Aug 2011 TC Formation Daily Probability Forecast Issued: 04 Aug 2011. Valid: 17-23 Aug 2011.

32. 32 TC LLF Brief, 13Sep11, murphree@nps.edumurphree@nps.edu 18W formed 10 Sep 2011 TC Formation Daily Probability Forecast Issued: 25 Aug 2011. Valid: 07-13 Sep 2011. 19W formed 11 Sep 2011  Invest 95W formed 12 Sep 2011   Unnamed TD / Invest 93W formed 09 Sep 2011 Invest 92W formed 08 Sep 2011 Comparisons of Forecasts to TC Formations and TC Invests