1. 2016 HEPEX Workshop Université Laval, Quebec, Canada
Evaluating the Usefulness of the US NWS Hydrologic Ensemble Forecast Service (HEFS) in the Middle Atlantic Region for Flood and Drought Applications
Seann Reed, Development and Operations Hydrologist
Yamen Hoque, Hydrologist
Alaina MacFarlane, Hydrologist
Kevin Hlywiak, Sr. Hydrologist
Peter Ahnert, Hydrologist-in-Charge
NOAA National Weather Service Middle Atlantic River Forecast Center,
State College, Pennsylvania, USA
Hello everyone 1

2. Introduction
MARFC currently uses two ensemble streamflow prediction systems operationally
Ensemble Streamflow Prediction (ESP)
Meteorological Model Ensemble Forecast System (MMEFS)
Both have shortcomings when it comes to quantifying uncertainty in streamflow prediction
Hydrologic Ensemble Forecast Service (HEFS) has potential to address these shortcomings
Input data: MAP averaged precip via rain gage, here is concept diagram, if three gages, in drainage area, take areal average, MAT areal averaged mean from daily max/min data, both MAP and MAT are 6-hourly, MAPE is daily, areal averaged from point PE at nearest NWS ASOS

3. Project Goals Evaluate quality of HEFS forecasts across spatial scales
Hindcasting studies
Comparison of recently archived forecasts
NOAA SARP, Penn State CSTAR, NYCDEP

4. ‘ESP’: Ensemble Streamflow Predictions (30 – 90 day forecasts)
Historical simulation: cumulative distribution function for 50 years of model simulations
Conditional simulation: use current model states and run 50 possible historical time series of precipitation and temperature through our models
30 day forecast
But!
Only historical climatic data
No bias/uncertainty accounting
ESP is current operational methodology, allows for day forecasts. Take 50 years of model simulations, develop cumulative distribution function, then use current model states, run 50 possible historical t-s of precip/temp w/ our models, develop streamflow based on that.
Only historical climatic data is taken into consideration, no accounting for bias or hydrologic uncertainty

5. MMEFS: Meteorological Model Ensemble Forecast System: 3
MMEFS: Meteorological Model Ensemble Forecast System: 3.5 to 7 day outlook
Run precipitation and temperature forecast ensembles through our models to produce streamflow ensembles.
MMEFS used as a ‘heads-up’ tool for possible flooding several days out.
But!
Short-Term
Hydrologic uncertainty not accounted for
In MMEFS, day outlook generated, as tool to be on lookout for possible flooding. Run precip/temp forecast ensembles, get streamflow.
Note that different model inputs give a different spread so neither model represents the true uncertainty. Also, as shown, short term.
NAEFS at Marietta, PA : 4/20/2015
7 day outlook
SREF at Marietta, PA : 4/20/2015
3.5 day outlook

6. Hydrologic Ensemble Forecast Service (HEFS)
HEFS aims to “capture” observed flow consistently
So, must account for total uncertainty & remove bias
Total = forcing uncertainty + hydrologic uncertainty
Goal: quantify total uncertainty in flow
Observed streamflow
Weather (forcing) uncertainty in flow
Hydrologic uncertainty
Streamflow
Total
Forecast horizon
Ultimately, the aim of any hydrologic ensemble forecasting system is to quantify the total uncertainty in the hydrologic forecasts, and this is the aim of the HEFS
Unless the total uncertainty is quantified adequately, the forecast probabilities cannot be accurate; in other words the ensemble spread will not consistently “capture” the observed streamflow
The total uncertainty originates from two main sources: 1) the weather forecasting or “forcing” uncertainty (i.e. uncertainty about the future values of temperature, precipitation and any other forcing variables used by the hydrologic models); and 2) the hydrologic uncertainty
The hydrologic uncertainty comprises all of the uncertainties associated with hydrologic modeling, including the initial conditions, model parameters, model structure and any other operations, such as manual modifications to the hydrologic model states and modeling of river regulations
The HEFS aims to quantify the total uncertainty, in order to produce accurate forecast probabilities. As such, the HEFS includes separate components for modeling the weather forecasting uncertainties and the hydrologic modeling uncertainties 6

7. What is HEFS? = forcing unc. = hydro. unc. = users
Ensemble Post-Processor (EnsPost)
= forcing unc.
= hydro. unc.
= users
WPC/RFC
forecasts
(1-5 days)
Flow bias / uncertainty accounting
Correct flow bias
Add spread to account for hydro. model uncertainty
GEFS
forecasts
(1-15 days)
Meteorological Ensemble Forecast Processor (MEFP)
Hydrologic
models (CHPS)
Bias-corrected ensemble flow forecasts
Correct forcing bias
Merge in time
Downscale (basin)
CFSv2
forecasts
( days)
The forecast horizon in HEFS is from hours to 1 year into the future.
This slide shows the main components of the HEFS and how they account for the two sources of uncertainty identified above, namely the weather forecasting uncertainties and the hydrologic modeling uncertainties
The forcing uncertainties are represented by green boxes and the hydrologic uncertainties are represented by blue boxes
The Meteorological Ensemble Forecast Processor (MEFP) captures the forcing uncertainties and corrects for biases in the forcing inputs to the HEFS
Specifically, the MEFP ingests raw forcing information from multiple weather and climate models, as well as the RFC deterministic weather forecasts. For example, RFC forecast (1-5), GEFS (1-15), CFSv2 (16-270), climatology (longer term). The MEFP corrects for biases in the forcing inputs and downscales them for input to the NWS hydrologic models
These forcing inputs are then used by the Community Hydrologic Prediction System (CHPS) to produce “raw” streamflow forecasts, which account for the forcing uncertainties and biases, but do not account for the hydrologic uncertainties and biases
A separate tool captures the hydrologic modeling uncertainties and corrects for biases in the raw streamflow forecasts. This is known as the Ensemble Postprocessor (EnsPost)
The final outputs from the HEFS include bias-corrected and downscaled precipitation and temperature forecasts, together with bias-corrected ensemble streamflow forecasts
NWS and external user applications
Climatology
(271+ days)
(MEFP forcing also available to users) 7

8. Experimental HEFS Graphics for Marietta, PA
Real-time HEFS Implementation
Running 14 customized points for NYCDEP to help manage water supply
Running for 156 MARFC daily forecast points for internal evaluation
No public products yet
Experimental HEFS Graphics for Marietta, PA
4/25/2015
Current state of HEFS: 14 points for NYCDEP, also running at 156 MARFC points, internal evaluation/verification only, not made publicly available yet 8

9. MARFC Hydrologic ‘Lumped’ Model
Spatially averaged precipitation, 6 hour time steps
Mean Areal Precip (MAP), Temperature (MAT) & Potential (MAPE) Evapotranspiration
rain + melt
SNOW -17
Unit Hydrograph
runoff
Continuous-API
API = Antecedent Precipitation Index
MARFC lumped model concept: input data – to SNOW17 model get rainfall and snowmelt – to CAPI to get runoff – construct UHG get flow – use w/ rating curve to get river stage
Rating Curve
stage
flow
Other Models: RES-SNGL, LAG-K, HEC-RAS

10. Hindcasting in the Susquehanna River Basin
Part of the motivation in selecting the study period was to ensure we included in , a time period during which the Susquehannah River basin experienced drought. With more time, we would like to extend the length of the period. We have results for the basins shown at the time of this presentation; however, we are close to having results for the entire Susquehannah.
Hindcast period: Feb Dec. 2010
Reference forecasts: resampled climatology

11. General Hindcasting Steps
Load deterministic input data: MAP, MAT, MAPE, QINE
Get meteorological reforecasts (forcings): GEFS, CFSv2
Update state so initial conditions are available
6-hr reforecasts for desired lead time on day 1 with initial conditions
Forecast on day 5 again with day 5 conditions as new initial
Carry on until end of hindcast period
Update state to obtain
initial condition for model
Forecast for day 1 w/ lead time of 90 days
Forecast again on day 5 for next 90 days, so on
Here rudimentary visualization of how we do hindcasting: first load input data, get forcings, update model state till day 1 of hindcast period so initial conditions are available (like so). Start hindcast 6-hrly for desired lead time (like so), on day 5 forecast again for lead-time with day 5 condition as new initial (like so) and go on. 11

12. This page shows the Continuous Rank Probability Skill Score
This page shows the Continuous Rank Probability Skill Score. This is a measure of fractional skill improvement in HEFS runs versus the reference forecast, which is reasmpled climatology in this case.
The top graphics are results using MEFP only. The top left graphic is for all flow levels lumped together and the right graphic is for low flows which will be of great interest in drought applications.
Upper left panel: over all flow levels, there is a greater skill improvement in larger basins. With MEFP alone, we assume the initial skill improvement starts off low because the initial conditions from the historical simulations (without Mods or data assimilation) can be off. The skill peaks sooner in small basins and later in large basins, not surprising due to the relative basin response times. For Lancaster, the skill score drops below zero soon after the 15 day lead time. For the larger basins, a small level of improvement persists out past day 20.
Upper right panel: Scores for low flows are not as good overall but still show improvement over climatology. Similar trends are observed with basin and lead time, but with greater noise.
Lower left panel: Difference trend between large and small basins still holds. Not surprisingly, skill at short lead times is significantly improves with ENSPOST because ENSPOST incorproates information from the most recent streamflow observation into the hindcasts.
Lower right panel: For low flows, the improvement in CRPSS at short lead times is even more dramatic when ENSPOST is added, likely for the same reason mentioned above.

13. Hindcast Comparison Caveats
Resampled climatology not an exact surrogate for ESP
No manual mods, no data assimilation
Direct comparison between archived ESP data and HEFS hindcasts required
Availability of sufficient contemporaneous archive data?
Work ongoing in this regard

14. Comparisons of Archived HEFS Operational Runs with Archived MMEFS Forecasts
Daily HEFS runs for 156 points archived since June 2015
MMEFS outputs also archived during this period
HEFS runs use 12z GEFS, so we will compare to NAEFS MMEFS which also runs at 12z (also, GEFS is a subset of NAEFS)
Initial comparison for 3 wide-spread cool season events
Feb 3 - 5, 2016
Feb 16-17, 2016
Feb 25, 2016

15. What actually happened?
Six locations with Minor flooding and numerous locations above action stage Feb 3, 4, and 5th

16. HEFS
NAEFS
NAEFS: Max, Min, and Observed Peaks from 2/1 12z forecast
HEFS: Max, Min, and Observed Peaks from 2/1 12z forecast
HEFS
NAEFS
“Max” values are the maximum peak values of any ensemble member during a 7 day forecast window.
“Min” values are the minimum peak values of any ensemble member during a 7 day forecast window.
Along the x-axis, basins are ordered from left to right by increasing size
Top graph shows that the max and min peaks predicted by HEFS bound the observed peaks for these 49 basins.
Bottom graph shows that observations are close to Min peaks from NAEFS, suggesting that NAEFS is over-simulating this event.
Trend is similar for basins and

17. What actually happened?
No points reached flood stage but peaks that neared flood stage occurred ~ Feb 16, 17

18. HEFS
NAEFS
HEFS
NAEFS
HEFS
NAEFS

19. HEFS 2/23 @12z NAEFS 2/23 @12z What actually happened?
Flooding on Feb. 25th

20. HEFS
NAEFS
HEFS
NAEFS
HEFS
NAEFS

21. Conclusions from Susquehanna Hindcasts
The greatest CRPSS improvements tend to be in the first 15 days which is the duration of the GEFS reforecast input
HEFS hindcasts show more skill (relative to climatology-based hindcasts) for larger basins than for smaller basins
Benefits of MEFP are less for low flows than for all flows
ENSPOST adds strong improvement for early lead times and particularly for low flows. We assume this is due to the use of the latest observation in the ENSPOST algorithm

22. Conclusions from HEFS vs. MMEFS/NAEFS
Results mixed
Feb 3 - 5, 2016: HEFS predicted peak ranges better captured observed peaks. NAEFS-based forecasts too high.
Feb 16-17, 2016: HEFS predicted peak ranges too low. NAEFS-based forecasts are better.
Feb 25, 2016: Similar results from both HEFS and NAEFS. HEFS may be a little better as the observed peaks tend to fall closer to the middle of the Min-Max range. Unfortunately, for several of the observed floods, the observed peaks were higher than the Max peak predicted by both models.
More analysis spanning multiple flood events required