1. American Meteorological Society - January 15, 2009 National Weather Service River Flood Warning Verification Mary Mullusky, Ernest Wells, Timothy Helble, and Thomas Graziano Hydrologic Services Division Office of Climate, Water, and Weather Services National Weather Service

2. American Meteorological Society - January 15, 2009 2 Floods versus Flash Floods Flash Flood –Quick responding streams (reach flood stage in < 6 hrs after the heavy rain started) –Warnings issued for defined areas based on hydrometeorological criteria and observer reports River Flood –River rises to flood stage over many hours or days (> 6 hrs) –Longer lasting events –Based on forecasts for specific gaged locations, prepared by NWS hydrologists

3. American Meteorological Society - January 15, 2009 3 NWS River Flood Warning Verification Background The NWS issues river forecasts, watches and warnings for over 4000 locations nationwide.

4. American Meteorological Society - January 15, 2009 4 Flood Watch versus River Flood Warning Flood Watch –Conditions are favorable for flooding but flooding is not imminent or occurring –Not verified River Flood Warning –Issued for any high flow, overflow, or inundation event which is threatening lives and property and can be quantified or indexed at specific river locations –Verified WGUS42 KMHX 061600 FLWMHX BULLETIN - IMMEDIATE BROADCAST REQUESTED FLOOD WARNING NATIONAL WEATHER SERVICE NEWPORT/MOREHEAD CITY NC 1200 PM EDT FRI APR 6 2012...THE NATIONAL WEATHER SERVICE IN NEWPORT HAS ISSUED A FLOOD WARNING FOR THE FOLLOWING RIVER IN NORTH CAROLINA... NEUSE RIVER AT KINSTON AFFECTING CRAVEN AND LENOIR COUNTIES..HEAVY RAINFALL TOMORROW WILL CAUSE THE NEUSE RIVER TO RISE NEAR OR ABOVE FLOOD STAGE. SAFETY MESSAGE...NEVER DRIVE CARS...TRUCKS OR SPORT UTILITY VEHICLES THROUGH FLOODED AREAS. NCC049-107-070200- /O.NEW.KMHX.FL.W.0001.120407T1300Z-000000T0000Z/ /KINN7.1.ER.120407T1300Z.120412T0100Z.000000T0000Z.NO/ 1200 PM EDT FRI APR 6 2012 THE NATIONAL WEATHER SERVICE IN NEWPORT HAS ISSUED A * FLOOD WARNING FOR THE NEUSE RIVER AT KINSTON. * FROM SATURDAY MORNING UNTIL FURTHER NOTICE. * AT 900 AM EDT FRIDAY THE STAGE WAS 13.5 FEET. * MINOR FLOODING IS FORECAST. * FLOOD STAGE IS 14.0 FEET. * FORECAST...FLOOD STAGE WILL BE REACHED AT 900 AM SATURDAY. MAXIMUM STAGE WILL BE 15.0 FEET AT 900 PM EDT WEDNESDAY. THE RIVER MAY REMAIN ABOVE FLOOD STAGE FOR SEVERAL WEEKS. * IMPACT...AT 14 FEET...WATER WILL BEGIN TO OVERFLOW INTO LOWLANDS ADJACENT TO THE NEUSE RIVER. $$

5. American Meteorological Society - January 15, 2009 5 NWS River Flood Warning Verification Objective Objective: To provide national metrics of river flood warning lead time and accuracy similar to flash flood metrics –Hydrology Program committed to providing national metrics of river flood warning lead time and accuracy in both PPBES and OMB PART –The implementation of VTEC/HVTEC provided the data necessary to create a single source river flood warning verification system –OCWWS HSD and Regions coordinated on the requirements for the river flood warning verification system in Q2 FY08 –The OCWWS Performance and Awareness Branch began archiving all FLW’s (and all other products with the same WMO header) since October 1, 2007. –The OCWWS Performance and Awareness Branch delivered a prototype system September 2008

6. American Meteorological Society - January 15, 2009 6 Metrics –Probability of Detection –False Alarm Ratio –Critical Success Index –Lead Time –Absolute Timing Error Metric Stratification –by Time (day, multiple days, month, multiple months, year) –by location (point, groupings of points, WFO, RFC, Region, National) –by Typical River Response (slow, medium, fast) NWS River Flood Warning Verification Metrics and System Requirements

7. American Meteorological Society - January 15, 2009 7 Product Information Used 1 st Product Issuance –NWS location identifier, VTEC action code (NEW), product issuance time, rise above time (forecast) Last Product Issuance –NWS location identifier, VTEC action code (CAN,EXP), product issuance time, rise above time (observation), fall below time (observation) NCC049-107-070200- /O.NEW.KMHX.FL.W.0001.120407T1300Z-000000T0000Z/ /KINN7.1.ER.120407T1300Z.120412T0100Z.000000T0000Z.NO/ 1200 PM EDT FRI APR 6 2012 Issuances Used: Hydrologic VTEC used:

8. American Meteorological Society - January 15, 2009 8 NWS River Flood Warning Verification Logic Tree Is the last product action code equal to ‘CAN’ or ‘EXP’ or Is the last product PVTEC End time +120 hours EARLIER than the current time? 1 Unwarned Event (e.g., missed event) LT=0, ATE not calculated 1 Unknown Event Do NOT use in verification statistics 1 Warned Event (e.g., hit) LT = last product Rise Above time – 1st Product Issuance Time ATE = |last product Rise Above time-1st Product Rise Above Time| Is the Last product Rise Above Time And Crest Time and Fall Below Time Equal to a string of zeros and Is the action code equal to ‘CAN’? YES NO Is the Last product Rise Above Time EARLIER than the last product issuance time? YES NO YES NO YES Event Not Over Do NOT USE in official stats NO Is the 1st product Issuance Time EARLIER than or Equal to the last product Rise Above Time? 1 Incorrectly Warned Event (e.g., false alarm) LT and ATE not calculated 1 Incorrectly Warned Event (e.g., false alarm) LT and ATE not calculated 1 Unwarned Event (e.g., missed event) LT=0, ATE not calculated 1 Unknown Event Do NOT use in verification statistics Event Not Over Do NOT USE in official stats 1 Warned Event (e.g., hit) LT = last product Rise Above time – 1st Product Issuance Time ATE = |last product Rise Above time-1st Product Rise Above Time|

9. American Meteorological Society - January 15, 2009 9 Observations Issues –Quality Control of Observations - same level of WHFS QC applied prior to product issuance. Some QC automated with data ingest. –No Observation – event not included in metric –Reporting Frequency - system captures verification statistics for all locations where observations are available. However, we will only report statistics for those sites where the observational frequency is sufficient to assure the integrity of the verification data. Underestimates missed events. Prototype is still being tested – errors are likely Database incomplete – Currently only includes points that have river response classification and are in the HADS database NWS River Flood Warning Verification Limitations

10. American Meteorological Society - January 15, 2009 10 NWS River Flood Warning Verification WY2008 Preliminary Results

11. American Meteorological Society - January 15, 2009 11 NWS River Flood Warning Verification FY2008 Preliminary Results

12. American Meteorological Society - January 15, 2009 12 NWS River Flood Warning Verification Next Steps Expand Interface to get detailed reports of events Expand database - Each flood warning location must have a classified river response and an observation reporting frequency Rigorous Testing Develop River Flood Warning Lead Time and Accuracy performance goals

13. American Meteorological Society - January 15, 2009 Any Questions?