1. Updated real-time website and new products for the 2015 hurricane season
Kate Musgrave1, Christopher Slocum2,
and Andrea Schumacher1
1CIRA/CSU, Fort Collins, CO
2CSU Dept of Atmospheric Science, Fort Collins, CO
HFIP bi-weekly teleconference
09/09/2015
Acknowledgements: John Knaff (NOAA/NESDIS/StAR/RAMMB),
Brian McNoldy (UMiami)

2. Outline CIRA/NESDIS HFIP Products Large-Scale Model Diagnostic Files
Updated TC Real-time Site
Large-Scale Model Diagnostic Files
Hybrid Wind Speed Probabilities
Satellite-based Model Evaluation
Total Precipitable Water

3. CIRA/NESDIS HFIP Featured Products
Large-Scale Model Diagnostic Files
SPICE
RII
ECMWF
SHIPS
Real-time
Graphics
Real-time
Verification
Seasonal
Verification
SPICE 
Hybrid Wind Speed Probabilities
Real-time
Graphics
Seasonal
Verification
Satellite-based Model Evaluation
Synthetic
Infrared
Real-time
Graphics
Real-time
Verification
Seasonal
Verification
Total Precipitable
Water
Synthetic
Water-vapor

4. CIRA/NESDIS HFIP Real-time Product Delivery
SPICE
delivered to NCAR, and to CIRA ftp site
Diagnostic text files
delivered to HFIP Product Page
Hybrid Wind Speed Probability Plots
Model Inter-comparison Plots (GFS, HWRF, GFDL)
delivered to HFIP Product Page, and to TC Real-time Page
Diagnostic File Skew-T Log-p Plots (GFS, HWRF, GFDL)
delivered to TC Real-time Page
Diagnostic Storm-based Verification Plots (GFS, HWRF, GFDL)
Model Field Animations (GFS, HWRF, GFDL)
Blue: HFIP official repository
Orange: CIRA public repository

5. CIRA/RAMMB TC real-time website

6. CIRA/RAMMB TC real-time website …

7. CIRA/RAMMB TC real-time website
Multi-model Diagnostic Plots
Single-model Plots
GFS, HWRF, GFDL
Animation of innermost grid
(10° x 10° TC-following box for GFS)
Precipitation and SST
850 hPa Vorticity and Vertical Motion
10 m Wind Speed and SLP
Diagnostic Skew-T Log-p Animation
Diagnostic Verification

8. Future work with Website Updates
Continue real-time product support for 2015
Add synthetic satellite imagery to TC real-time page
Synthetic infrared and water-vapor imagery
Total precipitable water imagery
Evaluate transition of products to HFIP products page

9. Large-Scale Model Diagnostics

10. Large-Scale Model Diagnostic Files
Purpose
TC environment plays large role in intensity, structure
Basis of SHIPS, LGEM intensity guidance
Input required
Model grib files
u, v, T, RH, Z at mandatory levels 1000 to 100 hPa
SST field if available
Model storm track (ATCF format)
Output
Text file with SHIPS model predictors to 126 hr
Code available from CIRA
ftp://rammftp.cira.colostate.edu/musgravek/diagcode/
New version released May 2015
Verification
GFS, HWRF and GFDL diagnostic files (against GFS analysis)
Key parameters are calculated in prescribed areas...
This is already done with GFS output to create SHIPS “predictor” files available on NHC's FTP server
Sea surface temp (RSST)
mb shear (SHDC); 200 mb zonal wind (U20C)
200 mb temp (T200); mb RH (RHLO)
mb RH (RHMD); mb RH (RHHI)
200 mb divergence (D200); 850 mb vorticity (Z850)

11. Large-Scale Model Diagnostic Files
Diagnostic files available from
Three sections: Storm Data, Sounding Data, and Custom Data
Storm Data section contains: LAT, LON, VMAX, RMW, MSLP, shear magnitude and direction,
TC speed and heading, SST, OHC, TPW, distance to land,
850 mb tangential winds and vorticity, and 200 hPa divergence
Sounding Data section contains: U, V, T, RH, and Z at specified pressure levels,
and surface U, V, T, RH, and P

12. Multi-Model Diagnostic Comparison Plots
Model inter-comparison available from
Panel Design
5 panels:
Intensity (top left)
Track (bottom left)
Deep-Layer Shear ( hPa , top right)
SST (middle right)
Mid-Level RH ( hPa , bottom right)
Non-track panels show previous and next 5 days, centered at current time
Vertical lines indicate the initial time of the most recent available forecast, color-coded by model
Previous times are analysis values
Track panel shows five day forecasts and recent best track
Models Selected
Intensity: GFS, HWRF, GFDL, DSHP, LGEM, OFCL, BEST
Upcoming: SPC3 will be re-introduced
Track: GFS, HWRF, GFDL, OFCL, BEST
Deep-Layer Shear: GFS, HWRF, GFDL
SST: GFS, HWRF, GFDL
Mid-Level RH: GFS, HWRF, GFDL
Purpose
Provides overview of TC environment
Comparison of model track, intensity, and basic dynamic and thermodynamic environment

13. Diagnostic File Skew-T Log-p Plots
Purpose
Provide vertical view of model TC environment
Design
Uses “Sounding Data” section of diagnostic files
Highlight features not captured in multi-model comparison plots:
Vertical wind shear not shown in hPa shear calculation
Dry layers not included in hPa relative humidity
Contains “Storm Data” variables in text box, for compact view of the diagnostics for an individual forecast time
Animation of all forecast hours for each forecast cycle

14. Diagnostic Verification Plots
Panels (CW from top left):
Intensity, Deep-Layer Shear, Mid-Level RH, SST
Dashed Line: RMSE; Solid Line: MAE (left-hand axis)
Avg. track error indicated in MAE line shading [<100, 100 to 200, >200 nmi]
Gray Shaded Bars: Bias (right-hand axis)

15. Future work with Model Diagnostic Files
Implement upgraded diagnostic code at EMC
Assess utility of current and new diagnostic-based products
Provide Python code upon request
Investigate verification metrics and guidance-on-guidance
Update seasonal verification in late 2015
The seasonal verification code for the diagnostic files is being re-written to be consistent with the updated diagnostic file code
includes support for all global basins
includes an expanded parameter set
incorporates the histogram code provided by Ryan Torn
The updated verification code will be made available to the HFIP community

16. Hybrid wind speed probabilities

17. Hybrid statistical-dynamical wind speed probabilities
Overview
The Monte Carlo wind speed model estimates the probability of 34-, 50-, and 64-kt winds out to 5 days
Generates 1000 realizations based on the official forecast by sampling from past official track and intensity errors and using a persistence and climatology radii model.
Version of Monte Carlo wind speed probability model where the statistical track realizations are replaced with global model ensemble tracks
Intensity and radii come from same method as statistical MC model
Motivation: global model track models have been found to be well-dispersed, can they improve WSPs? (internal guidance, not meant to replace public product)
Up to 133 tracks used: GFS (20), CMC (20), EMCWF (50), FNMOC (20), and UKMET (23)

18. Hybrid statistical-dynamical wind speed probabilities
Status
Available since 21 Aug 2012 for Atlantic, NE Pacific, and NW Pacific TCs
Runs at 0 and 12 Z: Ensemble data latency differs (6-12 hrs), so provisional versions of hybrid WSPs are run using all data available
Displayed in real-time on HFIP experimental products web page: “Ensemble Model Output” tab

19. Hybrid vs. Statistical - Example
Hurricane Isaac (2012) made landfall along the Louisiana coast at 0000 UTC 29 August
Statistical: Largest coastal probabilities focused along FL panhandle
Hybrid: Largest coastal probabilities smaller, spread from FL to LA

20. Hybrid vs. Statistical - Verification
Sample: (only full seasons hybrid WSPs have run)
Overall, hybrid WSPs had improved Brier scores over the statistical version in all basins (although NE Pacific is marginal)

21. Future work with Hybrid WSPs
Continue running in real-time for 2015
Display will remain on HFIP products webpage
Update verification in early 2016
Improvements to statistical WSPs
Update serial correlation method
Incorporate intensity GPCE
Investigate other hybrid approaches
Preliminary work has begun on developing model-based WSPs that use forecasts and error statistics from global models

22. Total Precipitable Water (TPW)

23. Verification of large-scale HWRF synthetic total precipitable water
Purpose:
Environmental moisture plays a critical role in TC evolution
Validate HWRF synthetic total precipitable water using the NESDIS operational blended product (Kidder & Jones 2007)
Metrics:
Mean absolute error (MAE)
Mean bias
Mean square error (MSE) skill score (SS) with climatology as a reference
Potential extensions to work:
Correlate MSE SS from the analysis to 24/48 hour intensity error
Perform outlier analysis
Comparison and difference plots and normalized histograms for each forecast.
Kidder, S. Q., and A. S. Jones, 2007: A Blended Satellite Total Precipitable Water Product for Operational Forecasting. J. Atmos. Oceanic Technol., 24, 74–81.

24. Mean absolute error and bias for 2015
Atlantic and East Pacific HWRF model runs (12Z June 9 onward) calculated on a 40°x40° storm-relative box
Relatively constant positive bias
MAE grows with time
Similar trend to 2014 season for parent grid and inner nest
Atlantic
E. Pacific

25. Shifting to a Mean Square Error Skill Score
MAE and bias do not provide insight into the divergence in observations and forecast
Correlation is unbiased by construct but forecasts are not
Adding climatology tames spurious skill given to a forecast predicting climatology
Phase association:
Measure of forecast feature location, shape, and relative magnitude
Conditional bias:
Amplitude error
Unconditional bias:
Map error (bias)
MSE SS: 1 = perfect; 0 = no skill;  – ∞ 2 2
y = forecast
o = observation
s = std dev
[ ]’ = [ ] - clim
Livezey, R. E., 1995a. Field intercomparison. In: H. von Storch and A. Navarra, eds., Analysis of Climate Variability. Springer,
Livezey, R. E., 1995b. The evaluation of forecasts. In: H. von Storch and A. Navarra, eds., Analysis of Climate Variability. Springer,
Murphy, A. H., and E. S. Epstein, 1989: Skill Scores and Correlation Coefficients in Model Verification. Mon. Wea. Rev., 117, 572–582.
Wilks, D.S., Statistical Methods in the Atmospheric Sciences,

26. MSE SS with climatology for 2015
Atlantic:
MSE SS rapidly deteriorates
By forecast hour 90, Mean MSE SS < 0
Indication of more land influence and track deviations
E. Pacific:
Mean MSE SS stays positive
Note: Violin plots are similar to box plots but include probability density. The marker denotes the mean.
Atlantic
E. Pacific

27. MSE SS term contributions
Atlantic:
Phase association (large = higher MSE SS) drops over forecast period indicating shape, location, relative magnitude differences
Unconditional bias relatively steady
East Pacific:
Better phase association
Small unconditional bias
Atlantic
E. Pacific

28. Potential application to guidance-on-guidance
Can the 6 hr forecast MSE SS provide insight into intensity error?
MSE SS potentially correlated to 48 hr intensity error in the Atlantic
Weaker relationship in the East Pacific
Outlier analysis may provide more insight
Atlantic
E. Pacific

29. Total precipitable water summary
Conclusions:
Atlantic and East Pacific HWRF model runs have a positive bias and increasing MAE
MSE Skill Score contextualizes error sources
Phase association dominates at early forecast hours
Growing conditional biases at end of forecast
Unconditional bias is a minor factor in the MSE SS
Potential future applications:
Explore using 6 hr forecast to provide guidance on guidance for the 24/48/72 hr intensity error
Perform outlier analysis to identify what poor precipitable water initializations indicate

30. CIRA/NESDIS HFIP Products
Large-Scale Model Diagnostic Files
SPICE
RII
ECMWF
SHIPS
Real-time
Graphics
Real-time
Verification
Seasonal
Verification
SPICE 
Hybrid Wind Speed Probabilities
Real-time
Graphics
Seasonal
Verification
Satellite-based Model Evaluation
Synthetic
Infrared
Real-time
Graphics
Real-time
Verification
Seasonal
Verification
Total Precipitable
Water
Synthetic
Water-vapor

31. Questions?