Analysis of Blue Mesa Inflow Forecast Errors Tom Pagano, aka: “Wha’ happa’???”

Performance of 2008 forecasts

NRCS daily forecasts Official coordinated forecasts Daily forecast skill Observed

2008 Official historical published outlooks issued April 1

Blue Mesa + Taylor Park Reservoir Storage Kac ft Full Aug 1 Jan 1 Forecasts and storage have come up short since 2000

2008 obs Historical NRCS “End of season” daily forecast skill This takes into account all available data including spring/summer precipitation. Even then, this year still falls out as a big anomaly (major over-forecast)

April-July 2008 Blue Mesa Inflow Forecast Comparison Forecast issue date official norm April-July inflow kac-ft NRCS daily forecasts based on: Precip/Snow obs

April-July 2008 Blue Mesa Inflow Forecast Comparison Forecast issue date official norm April-July inflow kac-ft NRCS daily forecasts based on: Snow only Precip/Snow Precip only obs

April-July 2008 Blue Mesa Inflow Forecast Comparison Forecast issue date official NWS ESP norm April-July inflow kac-ft NRCS daily forecasts based on: Snow only Precip/Snow Precip only obs

NRCS Daily Forecasts for Apr-Jul Blue Mesa Inflow using snow information available by April 11 Forecast too high Forecast too low 2008 snowpack composite well beyond recent historical variability Daily forecast input is a composite of 7 snotel sites: North lost trail, butte, park cone, brumley, independence pass, porphyry creek, slumgullion

2008 Historical range

Park Cone Historical April 1 Snow Data Snow water equivalent (in) 2008… 2 nd biggest in last 42 years 6 th biggest in last 72 years

End of season wyprecip End of season snow Apr 11 snow Apr 11 wyprecip

Sources of predictability VIC model skill (University of Washington) Explained variance in predicting Apr-July runoff Blue – Snowpack Green – Soil Moisture Red – El Nino Darker colors- more important (courtesy of M Dettinger, Scripps) What basin processes are important? How have they been behaving recently? (using actual and model data) How is runoff driven by: snow and precipitation soil moisture evaporation/temperature Note: Model is not necessarily reality

snow and precipitation

Snow and precipitation are easily the dominant factor R 2 (% var expl) 1.0 = perfect better worse Blue Mesa April-July Inflow Forecast Skill Model swe + precip versus Model runoff Univ Washington model Snow + Precip NRCS Daily Forecasts Real world SNOTEL vs Real world runoff Issue date of forecast Snow/precipitation explain upwards of 90% of the year to year variability in runoff

Gunnison Basin April-June Precipitation Percent of average Data from WESTMAP: % % % avg: 75% norm Precipitation after April 1 is important

Spring precipitation, especially the sequencing with snowmelt is also important Runoff Snowmelt Rainfall Rainfall mixed with snowmelt “normal” April July

Spring precipitation, especially the sequencing with snowmelt is also important Runoff Snowmelt Rainfall Rainfall mixed with snowmelt “normal” Rainfall boosting snowmelt Larger volumes Snowmelt and rainfall separate Not enough “momentum” to produce big volumes All these complex interactions are tough to “cartoonize”; Simulation models can handle this… but still it’s tough to predict beyond 1-2 weeks. April July

Spring precipitation, especially the sequencing with snowmelt is also important April July Runoff Snowmelt Rainfall Rainfall mixed with snowmelt “normal” Rainfall boosting snowmelt Larger volumes Snowmelt and rainfall separate Not enough “momentum” to produce big volumes Even then, however, high heat and no rain can lead to “pouring sunshine” All these complex interactions are tough to “cartoonize”; Simulation models can handle this… but still it’s tough to predict beyond 1-2 weeks.

2008 Runoff Blue Mesa natural inflow kac-ft/day Total: Independ Pass Snowmelt Rainfall Schofield Snowmelt Rainfall (inches) Many fits and starts to snowmelt… Almost no spring rainfall

2008 Runoff Blue Mesa natural inflow kac-ft/day Total: Independ Pass Snowmelt Rainfall Schofield Snowmelt Rainfall (inches) 1999 Good mix of rain and snowmelt 1993

2008 Runoff Blue Mesa natural inflow kac-ft/day Total: Independ Pass Snowmelt Rainfall Schofield Snowmelt Rainfall (inches) 1999 Snowpack poor, but “perfect storm” for runoff efficiency

Soil moisture/Groundwater Are we still feeling the effects of 2002?

Blue Mesa Basin Soil Moisture (According to the Univ Washington Model- top 2 layers)

Blue Mesa Basin Soil Moisture (According to the Univ Washington Model- top 2 layers) (According to Park Cone Snotel- ~0-30” depth) Snotel does poorly in frozen soils, so that has been censored Model resembles snotel, but also remember we’re comparing basin average with point measurement

Blue Mesa Basin Soil Moisture (According to the Univ Washington Model- top 2 layers) (According to Park Cone Snotel- ~0-30” depth) Snotel does poorly in frozen soils, so that has been censored Univ Washington Model “deep” soil moisture layer

Colorado Active Well Level Network Only one Colorado USGS groundwater station in realtime (Pueblo) Gunnison: Period of record Measured 1x/year Crested Butte 8/1996 Taken in Mid-may

What influence humans? Does it matter? Blue Mesa For each site, all measurements Jan-Jun, Jul-Dec are averaged by year. Station half-year data then converted into standardized anomaly (o-avg(o))/std(o) vs period of record for the half year. Multiple stations are then averaged.

Univ Washington model Blue Mesa inflow basin total soil moisture (mm) January Start of 2008

Evapotranspiration/sublimation

Butte SNOTEL sublimation (as modeled by NOHRSC) Water year to date cumulative sublimation (inches) Oct-Jul Subl Year Precip Prcp ”30% ” 29% ”15% ”16% ”18% ”10% Long-term average evaporation + transpiration + sublimation = 73% of annual precipitation

Gunnison Basin March-May Average Temperature Departure from Data from WESTMAP: Warm spring temps recently except last year

How good are forecasts in general? “Perfect forecasts are all alike; Every bad forecast is bad in its own way.”

“Published Official” -Subjective (based on objective guidance) -Humans actively involved -Coordinated by NRCS+NWS

“NRCS Daily” -Objective -Statistical model-based -Highly automated -Only uses SNOTEL snow+wytd precip

University of Washington -Objective -Simulation model-based -Highly automated -Research grade

NWS ESP -Objective -Simulation model-based -Human controlled/vetted (in realtime)

Sources Natural flow: ( ) Blue Mesa outflow – Taylor Park change in storage Published official: ( ) ftp://ftp.wcc.nrcs.usda.gov/data/water/forecast/ With gaps filled in from other sources NWS ESP: ( ) University of Washington: ( ) NRCS Daily: ( ) All reforecasts available internally, some available online

Blue Mesa April-July Inflow Forecast Skill RMSE as % Normal 0 = perfect worse better Published official NRCS Daily Issue Month of Forecast (e.g. January 1) RMSE = sqrt(avg(f-o) 2 ) Period common to all datasets

Blue Mesa April-July Inflow Forecast Skill RMSE as % Normal 0 = perfect worse better Published official NRCS Daily NRCS Daily (residual) Issue Month of Forecast (e.g. January 1) RMSE = sqrt(avg(f-o) 2 )

Model Forecast Observed Model Simulated Forecasting runoff from a start date in June…

Model Forecast Observed Model Simulated Forecasting runoff from a start date in June… Is your April-July “forecast” your 1. observed + future only forecast or is it 2. simulated + future only forecast? Forecasting “residuals” (1.) is more accurate but is an “open book” exam

Blue Mesa April-July Inflow Forecast Skill RMSE as % Normal 0 = perfect worse better Published official NRCS Daily NRCS Daily (residual) Issue Month of Forecast (e.g. January 1) RMSE = sqrt(avg(f-o) 2 )

Blue Mesa April-July Inflow Forecast Skill RMSE as % Normal 0 = perfect worse better Published official NRCS Daily NRCS Daily (residual) University of Washington Issue Month of Forecast (e.g. January 1) RMSE = sqrt(avg(f-o) 2 )

Blue Mesa April-July Inflow Forecast Skill RMSE as % Normal 0 = perfect worse better Published official NRCS Daily NRCS Daily (residual) University of Washington NWS ESP Issue Month of Forecast (e.g. January 1) RMSE = sqrt(avg(f-o) 2 )

Blue Mesa April-July Inflow Forecast Skill R 2 (% var expl) 100 = perfect better worse Published official NRCS Daily NRCS Daily (residual) University of Washington NWS ESP Issue Month of Forecast (e.g. January 1) What’s different: This does not penalize for bias

Blue Mesa April-July Inflow Forecast Bias Forecast Bias as % Normal Forecasts too high Forecasts too low Issue Month of Forecast (e.g. January 1) Published official NRCS Daily University of Washington NWS ESP Bias = avg(f) – avg(o)

Blue Mesa April-July Inflow Forecast Bias Forecast Bias as % Normal Forecasts too high Forecasts too low Issue Month of Forecast (e.g. January 1) Published official NRCS Daily University of Washington NWS ESP (not available) Bias = avg(f) – avg(o) Note: were dry years. Comparatively, the normal had a “bias” of +36% (!)

Conclusions From a snow perspective, 2008 was an epic bust. Water year precipitation matched runoff however. Lack of spring precip and sequencing of snowmelt important All existing models have comparable skill. 30% normal error is typical Jan 1, 20-25% error on April 1 Snow, rainfall, evap, soil moisture important to runoff. Some easier to quantify than others. Recommendations Have local/anecdotal observations feed quantitative historical analyses to build appropriate models. Collect the right data to support creation of objective operational guidance, tempered by reason. Recognize the unknowable versus the unknown. Remember too that higher accuracy is not the only way to improve forecasts

Snow sublimation (inches/day) Snow sublimation (inches/day) Daily average wind speed (mph) Daily average relative humidity (%) Butte SNOTEL Site Data from NOHRSC model Nov 2002-Aug Note that winds don’t vary from year to year in most hydrologic models (e.g. NWS and UWashington)

Blue Mesa Basin Evaporation Losses by Month (according to the Univ Washington Model) Monthly average temperatures (deg F) Evapo-transpiration (kaf/month) April-June: temperatures and ET related July-September: ET moisture driven November-March: ET constant, small jan feb mar apr may jun jul aug sep oct novdec

Blue Mesa Basin Water Year Water Balance (according to the Univ Washington Model) Kac-ft/year Long-term average Evapotransporation (ET)/Precipitation = 73% ET doesn’t vary much from year to year. Even then, 65% of its variability depends annual precipitation (at least in the model’s reality…) Precipitation Runoff Evapotranspiration