1. 1 © CFI Group National Weather Service Marine/Tropical Survey Presentation of Customer Satisfaction Results March 16, 2004 Federal Consulting Group

2. 2 © CFI Group Introduction

3. 3 © CFI Group Snapshot of the American Customer Satisfaction Index (ACSI) The #1 national economic indicator of customer satisfaction Compiled by the National Quality Research Center at University of Michigan since 1994 using methodology licensed from CFI Group Measures 30 industries, 180+ organizations covering 75% of the U.S. economy –Over 70 U.S. Federal Government agencies have used ACSI to measure more than 120 programs/services Advanced methodology quantifiably measures and links satisfaction levels to performance and prioritizes actions for improvement Results from all surveys are published quarterly in The Wall Street Journal

4. 4 © CFI Group Scope of the 2003 NWS Customer Satisfaction Program  Four segments measured in 2003 –Media – completed –Emergency managers – completed –Aviation (pilots and dispatchers) – completed –Marine/Tropical – completed

5. 5 © CFI Group Marine Customer Satisfaction Results

6. 6 © CFI Group Project Background Target segment: Users of Marine and Coastal, Tropical, and/or Great Lakes products and services, including recreational boaters, coastal residents, beachgoers, and professionals. Survey content: Survey designed over July and August 2003, with feedback from process owners in NWS (Silver Spring and regions). Survey method: The survey was conducted online and hosted on CFI Group’s secure server. Customers contacted: NWS publicized the survey to interested parties and provided links to the survey on multiple web sites, including the National Hurricane Center. Response totals: The online survey accepted responses from September 9 through October 6, 2003. Completed a total of 6298 interviews.

7. 7 © CFI Group Completed Surveys, by Date Isabel Survey link removed from NHC site

8. 8 © CFI Group Who Are the Marine Respondents? Services Used

9. 9 © CFI Group Who Are the Marine Respondents? Primary Operating Area

10. 10 © CFI Group Who Are the Marine Respondents? Primary Marine Activity

11. 11 © CFI Group Who Are the Marine Respondents? Most-Used Products: Marine and Coastal

12. 12 © CFI Group Usefulness of Dissemination Methods 97% of respondents rate land methods 58% of respondents rate sea methods

13. 13 © CFI Group CSI Comparison across Government Agencies Customer Satisfaction Index (CSI) scores are based on three questions: Overall satisfaction with products and services Products and services compared to expectations Products and services compared to ideal

14. 14 © CFI Group Satisfaction Scores Services Used

15. 15 © CFI Group Satisfaction Scores Primary Operating Area

16. 16 © CFI Group Satisfaction Scores Primary Marine Activity 82 81 75 73 81 79 78 Law enforcement Passenger transport Coastal resident Beachgoer Science/research Search and rescue Freight transport Other Recreational boater Commercial fishing 1% of resp. 34% of resp. 15% of resp. 1% of resp. 2% of resp. 4% of resp. 12% of resp. 29% of resp. 1% of resp.

17. 17 © CFI Group NWS Marine CSI Model Confidence in NWS Confidence in NWS Take Action Contact NWS OUTPUTS SATISFACTION Customer Satisfaction Customer Satisfaction INPUTS Great Lakes Marine Forecasts Great Lakes Short-Fused And Other Forecasts Great Lakes Short-Fused And Other Forecasts Great Lakes Ice Forecasts NHC Forecasts Tropical Warnings and Watches Hurricane Local Statements Coastal /Offshore/High Seas Forecasts Marine/Coastal Short-Fused and Other Forecasts Marine/Coastal Short-Fused and Other Forecasts Coastal Flood and Surf Zone Information Marine Forecast Accuracy General Forecast Performance General Forecast Availability General Forecast Verification Quality of Ice Freeze-up Outlooks Timelines of Ice Break-up Outlooks Usefulness of tropical forecast products Ease of understanding tropical forecast products Clarity and utility of hurricane and tropical storm watches and warnings How well HLSs support mission needs in geographic area of interest Accuracy of Marine Warnings Overall accuracy of the Marine Forecasts NWS overall NWS vs. expectations NWS vs. ideal Contacted NWS to report problem or make suggestion Likelihood to take action based on NWS information Confidence in NWS Quality of Short Term Forecasts Timelines of Short Term Forecasts Quality of Forecasts Timelines of Forecasts Quality of Short Term Forecasts Timelines of Short Term Forecasts Quality of Area Forecast Discussions Timelines of Special Marine Warnings Quality of Forecasts Timelines of Forecasts Ease of understanding the data provided Ease of reading charts and graphs Timelines of warnings broadcast over NOAA Weather Radio Frequency of forecast updates Frequency NWS forecast events occur within six hours of predicted time frame

18. 18 © CFI Group NWS Marine Model Confidence in NWS Confidence in NWS Take Action Contact NWS INPUTS SATISFACTION OUTPUTS Great Lakes Marine Forecasts Great Lakes Short-Fused And Other Forecasts Great Lakes Short-Fused And Other Forecasts Great Lakes Ice Forecasts NHC Forecasts Tropical Warnings and Watches Customer Satisfaction Customer Satisfaction Hurricane Local Statements Coastal /Offshore/High Seas Forecasts Marine/Coastal Short-Fused and Other Forecasts Marine/Coastal Short-Fused and Other Forecasts Coastal Flood and Surf Zone Information Marine Forecast Accuracy General Forecast Performance General Forecast Availability General Forecast Verification 0.9 0.3 0.0 0.9 0.5 0.1 0.6 0.4 0.0 0.9 0.7 0.4 0.5 Impact: Measures leverage of component on target variable. If NHC Forecasts improves 5 points, CSI will improve 0.9 points. 76 78 63 85 80 76 79 78 74 78 76 69 78 13% 90 87 Score: Measures performance on a 0-100 scale.

19. 19 © CFI Group Priority Matrix: Where to Focus Attention

20. 20 © CFI Group Marine Forecast Accuracy Impact on CSI: 0.9

21. 21 © CFI Group NHC Forecasts Impact on CSI: 0.9 The usefulness of additional probabilistic information is rated an 86 – very useful When asked which area needs improvement most, 51% of respondents say ‘Track’. No other feature is mentioned more than 11% of the time.

22. 22 © CFI Group General Forecast Performance Impact on CSI: 0.7

23. 23 © CFI Group Coastal/Offshore/High Seas Forecasts Impact on CSI: 0.6

24. 24 © CFI Group Tropical Warnings and Watches Impact on CSI: 0.5 Most-mentioned minimum desired probability associated with a watch: 30% Most-mentioned minimum desired probability associated with a warning: 70% 50% of respondents prefer break points over county boundaries, and 28% are indifferent

25. 25 © CFI Group General Forecast Availability Impact on CSI: 0.4

26. 26 © CFI Group Marine and Coastal Short-Fused/Other Impact on CSI: 0.4

27. 27 © CFI Group Coastal Flood and Surf Zone Info Impact on CSI: 0.0

28. 28 © CFI Group Other Aspects of NWS Performance

29. 29 © CFI Group What Customers Want Customers express a clear preference for graphical forecasts. A constantly-evolving forecast is preferred over a regularly-scheduled forecast, 79% to 21%.

30. 30 © CFI Group What Customers Want (cont.) When asked if they want any weather information not currently provided by NWS, 82% say no. Needs are varied, from better barometric readings to more accurate lightning forecasts to better wave forecasts.

31. 31 © CFI Group Recommendations Keep up the good work! (e.g., Isabel forecasting) Improve perceived forecast accuracy by increasing the number of official forecasts Give customers more information to explain the forecast data Educate customer regarding reasonable expectations, especially with wave height and period forecasts As in the Media survey, improve the consistency of Area Forecast Discussions and terminology use Ensure storm track is a top priority for tropical/hurricane forecasts Focus on graphical products, but keep in mind that radio is still a dominant method of information acquisition Examine possible causes of lower scores for Great Lakes Ice and Coastal Surf Zone forecasts Consider how different regions/users may respond to the survey differently –These overall findings will have varying applicability to specific sub- populations

32. 32 © CFI Group Discussion

33. 33 © CFI Group For More Information Brian LaMarre, NOAA Marine Program Manager (301) 713-1677 x108 Brian.LaMarre@noaa.gov Doug Helmreich, CFI Group Senior Consultant (734) 623-1321 dhelmreich@mail.cfigroup.com Bernie Lubran, Federal Consulting Group Project Manager (202) 906-5642 bernie.lubran@ots.treas.gov