1 Presented at the 36 th Climate Prediction and Diagnostics Workshop Fort Worth, TX, 03–06 Oct W (Talas) approaches Japan 01 Sep 2011 Improvements to, and Verification of, Intraseasonal to Seasonal Prediction of Tropical Cyclogenesis David Meyer and Tom Murphree Naval Postgraduate School and 36 th CDPW, Oct11,

2 ** LSEFs = large scale environmental factors: SST, ζ 850, shear , div 200, f Inputs and outputs are all on a daily, 2.5° scale. Statistical model is a logistic regression model. Each LSEF influences TC formation at over a 99% confidence level Extensive ensembling using multiple initial conditions and lead times (e.g., 184 members for 90 day leads)  equivalent to Monte Carlo simulation. Long Lead Forecasting of Tropical Cyclone Formation in the Western North Pacific Produce statistical- dynamical model output: ensemble- based long lead forecasts of TC formation probabilities (NPS TC LLFs, 4-90 day lead times) Force statistical model with dynamical, ensemble-based, long lead forecasts of LSEFs** (use NCEP Climate Forecast System v1) Apply statistical model of TC formation probability (NPS logistic regression model) Build statistical model based on relationships between TC formations and LSEFs** (based on JTWC best track and NCEP R2 reanalysis data) NPS Statistical-Dynamical Forecast System TC LLF Brief, 13Sep11,

3 Statistical Model Performance Verification 26 years of zero lead hindcasts for western North Pacific (WNP) for show statistical model has skill: 1.Hindcasts evaluated at daily, 2.5° resolution 2.Proportion correct: 89% (671 of 759 TCs formed within a forecasted region) 3.Positive Brier skill score 4.Positive ROC skill score 5.Modeled probabilities match well with observed formations for: a.El Nino and La Nina b.Eight MJO phases 36 th CDPW, Oct11,

4 LTM TC Formation Probability LTM TC Formation Probability and Formations 36 th CDPW, Oct11, 1.Probabilities from NPS statistical model forced with R1 LTM LSEFs. 2.Probabilities not calculated directly from observed TC formations. 3.LTM probabilities correspond well to LTM formations. Qualitative Statistical Model Verification: JASO LTM TC Formation Probability and Actual Formations

5 EN 36 th CDPW, Oct11, 1.Statistical model based probabilities using EN and LN LSEFs. 2.EN probabilities higher in central – southeastern WNP. 3.LN probabilities higher in northwestern, northern, and southwestern WNP. 4.Probabilities correspond well to observed formations. Qualitative Statistical Model Verification: JASO ENLN TC Formation Probability and Actual Formations El NinoLa Nina EN LN

LTM LTM TC Formation Probability MJO TC Formation Probabilities, Phases th CDPW, Oct11, Qualitative Statistical Model Verification: JASO MJO TC Formation Probability and Actual Formations 1.Statistical model based probabilities using LSEFs for phases Large variations in probability patterns and magnitudes by phase.

7 LTM LTM TC Formation Probability and Formations MJO TC Formation Probabilities and Actual Formations, Phases th CDPW, Oct11, Qualitative Statistical Model Verification: JASO ENLN TC Formation Probability and Actual Formations Good agreement between statistically modeled probabilities and actual formations for phases 1-8.

Sample NPS TC Formation Forecast System Output: Probability of TC Formation at Intraseasonal Leads 36 th CDPW, Oct11, 8 TC Formation Daily Probability Forecast Issued: 25 Aug Valid: Sep Weekly forecasts: average of 7 consecutive daily 21 day lead forecasts. 2.Forecast shown is an example of NPS inputs to CPC GTHB technical teleconference (but with subsequent addition of verifying formations in red). 3.Preliminary verification: from first position in JTWC TCFAs. 4.Updated verification: from JTWC best track data (released in following Mar-Apr).

Sample NPS TC Formation Forecast System Output: Probability of TC Formation at Intraseasonal Leads 36 th CDPW, Oct11, 9 TC Formation Daily Probability Forecast Issued: 25 Aug Valid: Sep W formed 10 Sep 11

Sample NPS TC Formation Forecast System Output: Probability of TC Formation at Intraseasonal Leads 36 th CDPW, Oct11, 10 TC Formation Daily Probability Forecast Issued: 25 Aug Valid: Sep W formed 10 Sep 1119W formed 12 Sep 11

36 th CDPW, Oct11, 11 Sample NPS TC Formation Forecast System Output: Probability of TC Formation at Zero Lead Storm that became a TD over land and quickly dissipated TC Formation Daily Probability Forecast Issued: 10 Sep Valid: 10 Sep W formed 10 Sep 1119W formed 12 Sep 11

Sample NPS TC Formation Forecast System Output: Probability of TC Formation at Two Month Leads 36 th CDPW, Oct11, 12 TC Formation Daily Probability Forecast Issued: 17 July Valid: Sep W formed 10 Sep 1119W formed 12 Sep 11

Forecast Verification Methods: Graphic Contingency Table YesNo Yes a = total no. of hitsb = total no. of weekly false alarms No c = total no. of weekly missesd = total weekly correct rejections Observed Forecasted 13 TC LLF Brief, 13Sep11,       

14 Quantitative Forecasting System Skill Assessment, Week One and Week Two Outlooks, Jun-Nov 2010 PC: Proportion Correct. POD: Probability of Detection. FAR: False Alarm Ratio. HSS: Heidke skill score. ETS: Equitable Threat Score. PCA: proportion of WNP within forecast contours. FARN: False Alarm Ratio for forecasted non-formation 1.Good overall performance but FAR is higher than we would like. 2.Additional indicator of good skill: positive Brier skill scores for all leads. 3.Stable performance: similar performance in 2009 and 2011 (preliminary). 4.Similar skill at longer leads (21-90 days), but at a cost of reduced resolution and sharpness. 1.One and two week outlooks: same as weekly forecasts we now provide to CPC. 2.Verification done on weekly basis. 3.Neighborhood radius of 2.5° used on observations. 36 th CDPW, Oct11,

15 Quantitative Forecasting System Skill Assessment: Total Forecasted Area 1.Total marine area of the WNP is ~25 million square kilometers. 2.Almost all of this marine area is capable of TC formation during JASO. 36 th CDPW, Oct11,

16 Quantitative Forecasting System Skill Assessment: Total Forecasted Area 36 th CDPW, Oct11, Red box represents: Total forecasted area in an average daily probability forecast 10% of the WNP marine area

17 Qualitative Forecasting System Skill Assessment: Intraseasonal Forecasts - Peak Season 2010 vs Highest formation probabilities and most TCs occurred west of 125E. 2.This probability pattern typical for much of Formations typical for La Niña periods. 4.5 of 7 TCs successfully forecasted (2.5° neighborhood). 5.Monthly average probability locations and magnitudes similar to those in corresponding weekly forecasts. 36 th CDPW, Oct11, Monthly Average of Daily Probability Forecasts 21 Day Lead. Valid: 16 Aug - 15 Sep 2010.

18 36 th CDPW, Oct11, Qualitative Forecasting System Skill Assessment: Intraseasonal Forecasts - Peak Season 2010 vs Monthly Average of Daily Probability Forecasts 21 Day Lead. Valid: 16 Aug - 15 Sep Highest formation probabilities and all TCs occurred east of 130E. 2.This probability pattern typical for much of Northern formations may be indication of La Nina impacts. 4.7 of 7 TCs successfully forecasted (2.5° neighborhood). 5.Monthly average probability locations and magnitudes similar to those in corresponding weekly forecasts.

19 NPS TC Forecasting System: Summary and Plans 1.Skill assessment: a.Skill evaluated over three years of forecasts, , with: 1.EN, LN, neutral, and MJO conditions 2.A total of 69 TCs b.Quantitative: System has skill over chance and/or climatology at all leads (out to 90 days) and at daily and weekly resolution c.Qualitative: System represents EN, LN, and MJO formation probability patterns 2.Statistical model being rebuilt based on CFSR and additional years 3.Reforecast 2011 and forecast 2012 with rebuilt statistical model and CFSv2 as dynamical component. 4.Expand forecast regions to N Atlantic and other basins (promising results from preliminary work). 36 th CDPW, Oct11,

20 NPS TC Forecasting System: Summary and Plans 5.Begin issuing experimental forecasts from corresponding TC intensity forecasting system. Preliminary WNP results are very promising. 6.Begin work on corresponding TC track forecasting system. 7.Generate new statistical model for off season forecasting 8.Continue coordination / collaboration with: CPC, FNMOC, JTWC, NHC 9.Continuing and expanding discussions with potential forecast customers. 36 th CDPW, Oct11,

Tom Murphree, Ph.D. Research Associate Professor Department of Meteorology Naval Postgraduate School 254 Root Hall 589 Dyer Road Monterey, CA office cell fax Long Lead Forecasting of Tropical Cyclones: Contact Information 21 David Meyer Operations Research Analyst Department of Meteorology Naval Postgraduate School Monterey, CA cell/office 36 th CDPW, Oct11,

22

23 SST Relative Vorticity 850 hPa Divergence 200 hPa Vertical Wind Shear, Zonal, hPa LTM TC Formation Probability LTM LSEFs LTM TC Formation Probability and Formations 36 th CDPW, Oct11, Vertical Wind Shear, Meridional, hPa 1.Probabilities from NPS statistical model forced with R1 LTM LSEFs, not directly from observed TC formations. 2.LTM probabilities consistent with LTM LSEFs. 3.LTM probabilities correspond well to LTM formations. Qualitative Statistical Model Verification: JASO LTM TC Formation Probability and Actual Formations

24 ENLN TC Formation Probabilities and Formations LTM TC Formation Probabilities and Formations 36 th CDPW, Oct11, LTM LN EN EN, LN, and LTM probabilities correspond well to observed formations. Qualitative Statistical Model Verification: JASO ENLN TC Formation Probability and Actual Formations

25 Modeled TC Formation Probability Anomalies for EN and LN, Jul-Oct EN LN Regression model based probability anomalies for EN and LN: 1.Consistent with prior observational studies  model verification. 2.Forecast system has potential to represent EN-LN impacts. 3.Results useful in education/training and seasonal scale situational awareness for operational forecasters. 4.Useful products for long range planning. Qualitative Statistical Model Verification: JASO ENLN TC Formation Probability and Actual Formations

26 Modeled TC Formation Probability Anomalies for MJO Phases 1-8, Jul-Oct Regression model based probability anomalies for MJO phases 1-8: 1.Consistent with prior observational studies  model verification. 2.Forecast system has potential to represent MJO impacts. 3.Results useful in education and intraseasonal scale situational awareness for forecasters. 4.Useful products for long range planning. Qualitative Statistical Model Verification: JASO MJO TC Formation Probability and Actual Formations

Quantitative Forecast Verification Methods 1.Standard 2 x 2 contingency table verification metrics used to allow ready interpretation and comparison with other forecasts (e.g., CPC one and two week forecasts). 2.Verification done for both formation and non-formation forecasts. YesNo Yesab Nocd Observed Forecasted Proportion correct (PC) = (a+d)/(a+b+c+d) Basic proportion correct (BPC) = a/(a+b) Hit rate (HR) = probability of detection (POD) = a/(a+c) False alarm ratio (FAR) = b/(a+b) False alarm rate (FAR) = b/(b+d) Threat score (TS) = critical success index (CSI) = a/(a+b+c) Heidke skill score (HSS) = [2(ad-bc)] / [(a+c)(c+d) + (a+b)(b+d)] Cf. Wilks (2006) th CDPW, Oct11,

28 Quantitative Forecasting System Skill Assessment, 30 Day Lead Forecasts, Jun-Nov 2010 PC: Proportion Correct, POD: Probability of Detection, FAR: False Alarm Ratio, HSS: Heidke Skill Score, ETS: Equitable Threat Score, PCA: proportion of WNP contoured by forecasts, FARN: False Alarm Ratio for forecasted non-formation 1.System performance stability: 2010 performance comparable to 2009 and preliminary 2011 results 2.Additional indicator of good skill: positive Brier Skill Scores 3.No comparable forecasts at these leads available from other sources 1.Performance metrics calculated on a weekly basis for consistency with CPC’s weekly GTH forecast 2.A 2.5° neighborhood was used on observations (radius) 3.Weekly outlooks created by averaging 7 days worth of daily 30 day lead forecasts

29 Quantitative Forecasting System Skill Assessment, 60 Day Lead Forecasts, Jun-Nov 2010 PC: Proportion Correct, POD: Probability of Detection, FAR: False Alarm Ratio, HSS: Heidke Skill Score, ETS: Equitable Threat Score, PCA: proportion of WNP contoured by forecasts, FARN: False Alarm Ratio for forecasted non-formation 1.System performance stability: 2010 performance comparable to 2009 and preliminary 2011 results 2.Additional indicator of good skill: positive Brier Skill Scores 3.No comparable forecasts at these leads available from other sources 1.Performance metrics calculated on a weekly basis for consistency with CPC’s weekly GTH forecast 2.A 2.5° neighborhood was used on observations (radius) 3.Weekly outlooks created by averaging 7 days worth of daily 6- day lead forecasts

30 Quantitative Forecasting System Skill Assessment, 90 Day Lead Forecasts, Jun-Nov 2010 PC: Proportion Correct, POD: Probability of Detection, FAR: False Alarm Ratio, HSS: Heidke Skill Score, ETS: Equitable Threat Score, PCA: proportion of WNP contoured by forecasts, FARN: False Alarm Ratio for forecasted non-formation 1.System performance stability: 2010 performance comparable to 2009 and preliminary 2011 results 2.Additional indicator of good skill: positive Brier Skill Scores 3.No comparable forecasts at these leads available from other sources 1.Performance metrics calculated on a weekly basis for consistency with CPC’s weekly GTH forecast 2.A 2.5° neighborhood was used on observations (radius) 3.Weekly outlooks created by averaging 7 days worth of daily 90 day lead forecasts

31 Comparisons of Forecasts to TC Formations and TC Invests 31 TC LLF Brief, 13Sep11, 14W formed 19 Aug W formed 22 Aug 2011  Invest 95W formed 18 Aug 2011 TC Formation Daily Probability Forecast Issued: 04 Aug Valid: Aug 2011.

32 TC LLF Brief, 13Sep11, 18W formed 10 Sep 2011 TC Formation Daily Probability Forecast Issued: 25 Aug Valid: Sep W formed 11 Sep 2011  Invest 95W formed 12 Sep 2011   Unnamed TD / Invest 93W formed 09 Sep 2011 Invest 92W formed 08 Sep 2011 Comparisons of Forecasts to TC Formations and TC Invests