GAINING CONFIDENCE IN HYDROLOGIC SIMULATIONS: AN ALTERNATIVE RESEARCH STRATEGY FOR THE DEVELOPMENT AND APPLICATION OF HYDROLOGIC MODELS Martyn Clark, Hilary McMillan, MS Srinivasan, and Ross Woods National Institute for Water and Atmospheric Research (NIWA) Christchurch, New Zealand Contact:

Outline A hydrologic emergency –The growing need for hydrologic models –The growing dissatisfaction with hydrologic models A research strategy for hydrologic modeling –The method of multiple working hypotheses –The dialog between the experimentalist and the modeler –The theory of diagnostic signatures –Complete representation of uncertainties in hydrologic simulations Development of a National Hydrological Model for NZ –Modeling approach –Model limitations

A hydrologic emergency?

Why are we in this situation? Decide which processes to include Define model equations Implement equations on a computer Build model Assemble datasets Estimate model parameters Analyze model simulations Apply model data error/uncertainty? inappropriate model parameters? ignore inconsistent results? missing processes? inappropriate equations? poor numerical implementation? Where is the science? Consider a typical modeling approach

Missing processes / inappropriate equations 1)Detailed physically-based conceptualization of snow processes 2)The real world –Sub-grid variability in snow is important to accurately model the timing of streamflow Shallow areas of snow melt first, and only contribute melt for a limited period of time; deep areas of snow contribute melt late into summer Early-season melt controlled by available energy; late-season melt controlled by snow covered area Clark, M.P., J. Hendrikx, A.G. Slater, R.A. Woods, E. Örn Hreinsson, T.R. Kerr, I.F. Owens, and N.J. Cullen, (2009) The use of field data to design distributed snow models. Paper submitted to Water Resources Research

The compensatory effect of model parameters Clark M.P., and J.A. Vrugt (2006): Unraveling uncertainties in hydrologic model calibration: Addressing the problem of compensatory parameters. Geophysical Research Letters, 33 (6): Art. No. L06406 MAR “reasonable” parameter set “inappropriate” parameter sets observations

The compensatory effect of model parameters Parameters can compensate for –Errors in model input –Weaknesses in model structure –Unrealistic values in related parameters Clark M.P., and J.A. Vrugt (2006): Unraveling uncertainties in hydrologic model calibration: Addressing the problem of compensatory parameters. Geophysical Research Letters, 33 (6): Art. No. L06406 MAR Remove persistent errors by data assimilation (SODA) more realistic model parameters by accounting for uncertainty in model inputs

Outline A hydrologic emergency –The growing need for hydrologic models –The growing dissatisfaction with hydrologic models A research strategy for hydrologic modeling –The method of multiple working hypotheses –The dialog between the experimentalist and the modeler –The theory of diagnostic signatures –Complete representation of uncertainties in hydrologic simulations Development of a National Hydrological Model for NZ –Modeling approach –Model limitations

A Research strategy for hydrologic modeling The method of multiple working hypotheses The dialog between the experimentalist and the modeler The theory of diagnostic signatures Complete representation of uncertainties in hydrologic simulations

The method of multiple working hypotheses Scientists often develop “parental affection” for their theories T.C. Chamberlain Chamberlin advocated the method of multiple working hypotheses “…the effort is to bring up into view every rational explanation of new phenomena… the investigator then becomes parent of a family of hypotheses: and, by his parental relation to all, he is forbidden to fasten his affections unduly upon any one” Chamberlin (1890)

Models as hypotheses of hydrologic behavior? Hydrologic models are an assemblage of hypotheses –Hypotheses of surface runoff generation –Hypotheses of recession behavior –Hypotheses of storage Different model components interact in complex ways Difficult to evaluate multiple modeling approaches in a controlled way –model inter-comparison experiments are a blunt instrument

PRMSSACRAMENTO ARNO/VICTOPMODEL pe S1TS1T S1FS1F S2TS2T S 2 FA S 2 FB q if qbAqbA qbBqbB q 12 ep q sx qbqb q 12 S2S2 S1S1 GFLWR S2S2 S1S1 ep q 12 q sx qbqb ep S 1 TA q sx q if qbqb S 1 TB S1FS1F q 12 S2S2 Clark, M.P., A.G. Slater, D.E. Rupp, R.A. Woods, J.A. Vrugt, H.V. Gupta, T. Wagener, and L.E. Hay (2008) Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44, W00B02, doi: /2007WR FUSE: Framework for Understanding Structural Errors

PRMSSACRAMENTO ARNO/VICTOPMODEL pe S1TS1T S1FS1F S2TS2T S 2 FA S 2 FB q if qbAqbA qbBqbB q 12 ep q sx qbqb q 12 S2S2 S1S1 GFLWR S2S2 S1S1 ep q 12 q sx qbqb ep S 1 TA q sx q if qbqb S 1 TB S1FS1F q 12 S2S2 Clark, M.P., A.G. Slater, D.E. Rupp, R.A. Woods, J.A. Vrugt, H.V. Gupta, T. Wagener, and L.E. Hay (2008) Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44, W00B02, doi: /2007WR Define development decisions: upper layer architecture

PRMSSACRAMENTO ARNO/VICTOPMODEL pe S1TS1T S1FS1F S2TS2T S 2 FA S 2 FB q if qbAqbA qbBqbB q 12 ep q sx qbqb q 12 S2S2 S1S1 GFLWR S2S2 S1S1 ep q 12 q sx qbqb ep S 1 TA q sx q if qbqb S 1 TB S1FS1F q 12 S2S2 Clark, M.P., A.G. Slater, D.E. Rupp, R.A. Woods, J.A. Vrugt, H.V. Gupta, T. Wagener, and L.E. Hay (2008) Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44, W00B02, doi: /2007WR Define development decisions: lower layer / baseflow

PRMSSACRAMENTO ARNO/VICTOPMODEL pe S1TS1T S1FS1F S2TS2T S 2 FA S 2 FB q if qbAqbA qbBqbB q 12 ep q sx qbqb q 12 S2S2 S1S1 GFLWR S2S2 S1S1 ep q 12 q sx qbqb ep S 1 TA q sx q if qbqb S 1 TB S1FS1F q 12 S2S2 Clark, M.P., A.G. Slater, D.E. Rupp, R.A. Woods, J.A. Vrugt, H.V. Gupta, T. Wagener, and L.E. Hay (2008) Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44, W00B02, doi: /2007WR Define development decisions: percolation

PRMSSACRAMENTO ARNO/VICTOPMODEL pe S1TS1T S1FS1F S2TS2T S 2 FA S 2 FB q if qbAqbA qbBqbB q 12 ep q sx qbqb q 12 S2S2 S1S1 GFLWR S2S2 S1S1 ep q 12 q sx qbqb ep S 1 TA q sx q if qbqb S 1 TB S1FS1F q 12 S2S2 Clark, M.P., A.G. Slater, D.E. Rupp, R.A. Woods, J.A. Vrugt, H.V. Gupta, T. Wagener, and L.E. Hay (2008) Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44, W00B02, doi: /2007WR Define development decisions: surface runoff

PRMSSACRAMENTO ARNO/VICTOPMODEL pe S1TS1T S1FS1F S2TS2T S 2 FA S 2 FB q if qbAqbA qbBqbB q 12 ep q sx qbqb q 12 S2S2 S1S1 GFLWR S2S2 S1S1 ep q 12 q sx qbqb ep S 1 TA q sx q if qbqb S 1 TB S1FS1F q 12 S2S2 Clark, M.P., A.G. Slater, D.E. Rupp, R.A. Woods, J.A. Vrugt, H.V. Gupta, T. Wagener, and L.E. Hay (2008) Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44, W00B02, doi: /2007WR Build unique models: combination 1

PRMSSACRAMENTO ARNO/VICTOPMODEL pe S1TS1T S1FS1F S2TS2T S 2 FA S 2 FB q if qbAqbA qbBqbB q 12 ep q sx qbqb q 12 S2S2 S1S1 GFLWR S2S2 S1S1 ep q 12 q sx qbqb ep S 1 TA q sx q if qbqb S 1 TB S1FS1F q 12 S2S2 Clark, M.P., A.G. Slater, D.E. Rupp, R.A. Woods, J.A. Vrugt, H.V. Gupta, T. Wagener, and L.E. Hay (2008) Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44, W00B02, doi: /2007WR Build unique models: combination 2

PRMSSACRAMENTO ARNO/VICTOPMODEL pe S1TS1T S1FS1F S2TS2T S 2 FA S 2 FB q if qbAqbA qbBqbB q 12 ep q sx qbqb q 12 S2S2 S1S1 GFLWR S2S2 S1S1 ep q 12 q sx qbqb ep S 1 TA q sx q if qbqb S 1 TB S1FS1F q 12 S2S2 Clark, M.P., A.G. Slater, D.E. Rupp, R.A. Woods, J.A. Vrugt, H.V. Gupta, T. Wagener, and L.E. Hay (2008) Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44, W00B02, doi: /2007WR Build unique models: combination 3 79 UNIQUE HYDROLOGIC MODELS ALL WITH DIFFERENT STRUCTURE

Clark, M.P., A.G. Slater, D.E. Rupp, R.A. Woods, J.A. Vrugt, H.V. Gupta, T. Wagener, and L.E. Hay (2008) Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44, W00B02, doi: /2007WR Application – MOPEX basins MOPEX (MOdel Parameter Estimation eXperiment) is a large international experiment for inter-comparison of hydrological models 12 basins in the southeast USA Two basins selected (the wettest and the driest) Each model is calibrated separately for each basin by minimizing RMSE Guadalupe French Broad

How good are the models? Clark, M.P., A.G. Slater, D.E. Rupp, R.A. Woods, J.A. Vrugt, H.V. Gupta, T. Wagener, and L.E. Hay (2008) Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44, W00B02, doi: /2007WR

Reasons for inter-model differences Mean and standard deviation of saturated area But, more work to be done –Model evaluation using multiple hydrologic indices RMSE is not enough –Model evaluation in experimental basins –Use of multiple models to quantify model uncertainty Applications of FUSE are just beginning Clark, M.P., A.G. Slater, D.E. Rupp, R.A. Woods, J.A. Vrugt, H.V. Gupta, T. Wagener, and L.E. Hay (2008) Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44, W00B02, doi: /2007WR

A Research strategy for hydrologic modeling The method of multiple working hypotheses The dialog between the experimentalist and the modeler The theory of diagnostic signatures Complete representation of uncertainties in hydrologic simulations

Lessons from Panola Recession analysis at three different spatial scales Clark, M.P., D.E. Rupp, R.A. Woods, H.J. Tromp-van Meerveld, N.E. Peters, and J.E. Freer (2009a) Consistency between hydrological models and field observations: Linking processes at the hillslope scale to hydrological responses at the watershed scale. Hydrological Processes, in press.

Modeled recessions —linear combination of reservoirs Hillslope + Ephemeral riparian reservoir Clark, M.P., D.E. Rupp, R.A. Woods, H.J. Tromp-van Meerveld, N.E. Peters, and J.E. Freer (2009a) Consistency between hydrological models and field observations: Linking processes at the hillslope scale to hydrological responses at the watershed scale. Hydrological Processes, in press. Hillslope + Ephemeral riparian reservoir + Perennial riparian reservoir

PRMSSACRAMENTO ARNO/VICTOPMODEL pe S1TS1T S1FS1F S2TS2T S 2 FA S 2 FB q if qbAqbA qbBqbB q 12 ep q sx qbqb q 12 S2S2 S1S1 GFLWR S2S2 S1S1 ep q 12 q sx qbqb ep S 1 TA q sx q if qbqb S 1 TB S1FS1F q 12 S2S2 Implications for model design? Hillslope Riparian ephemeral Riparian perennial Clark, M.P., D.E. Rupp, R.A. Woods, H.J. Tromp-van Meerveld, N.E. Peters, and J.E. Freer (2009a) Consistency between hydrological models and field observations: Linking processes at the hillslope scale to hydrological responses at the watershed scale. Hydrological Processes, in press. NON-LINEAR STORAGE DISCHARGE FUNCTIONS DO NOT SIMULATE INTERACTIONS BETWEEN DIFFERENT LANDSCAPE TYPES

A Research strategy for hydrologic modeling The method of multiple working hypotheses The dialog between the experimentalist and the modeler The theory of diagnostic signatures Complete representation of uncertainties in hydrologic simulations

The theory of diagnostic signatures

Premise is that standard approaches to model calibration based on sum of squared errors in streamflow have limited discriminatory power –The compensatory effect of model parameters Define a large set of hydrologic indices that relate to different parts of the hydrologic model Use the indices to diagnose weaknesses and estimate parameters in different parts of the hydrologic model Stepwise approach to calibration is similar to the Hay et al. approach Hay, L.E., G.H. Leavesley, M.P. Clark, S.L. Markstrom, R.J. Viger, and M. Umemoto (2006): A Multi-Objective, Step-Wise, Automated Calibration Approach Applied to Hydrologic Modeling of a Snowmelt-Dominated Basin in Colorado. Journal of the American Water Resources Association, 42,

A snow example (New Zealand) Clark, M.P., G. Martinez, E. Orn Hrennison, A.B. Tait, A.G. Slater, R.A. Woods, J. Schmidt (in preparation): Snow simulations for the South Island of New Zealand. Paper in preparation for the NZ Journal of Hydrology

A snow example (New Zealand) sum of positive monthly storage Clark, M.P., G. Martinez, E. Orn Hrennison, A.B. Tait, A.G. Slater, R.A. Woods, J. Schmidt (in preparation): Snow simulations for the South Island of New Zealand. Paper in preparation for the NZ Journal of Hydrology

A Research strategy for hydrologic modeling The method of multiple working hypotheses The dialog between the experimentalist and the modeler The theory of diagnostic signatures Complete representation of uncertainties in hydrologic simulations

Boulder Aspen Example 1—uncertainties in model inputs Clark, M.P., and A.G. Slater (2006): Probabilistic quantitative precipitation estimation in complex terrain. Journal of Hydrometeorology, 7, 3-22 Approach: Estimate uncertainty directly from the data - no need to specify error parameters - uncertainty estimates independent of the hydrological model

Step 1: Estimate precipitation CDF at each grid cell Step 2: Synthesize ensembles from the CDF corresponding observations Example 1—uncertainties in model inputs Clark, M.P., and A.G. Slater (2006): Probabilistic quantitative precipitation estimation in complex terrain. Journal of Hydrometeorology, 7, 3-22 (equally plausible representations of reality)

thin grey lines= individual ensemble members thick grey line= ensemble mean thick black line= observation Example 2—total hydrologic uncertainty Clark, M. P., D.E. Rupp, R.A. Woods, X. Zheng, R.P. Ibbitt, A.G. Slater, J. Schmidt, and M. Uddstrom, (2008) Hydrological data assimilation with the Ensemble Kalman Filter; Use of streamflow data to update the states in a distributed hydrological model. Advances in Water Resources, in press (available online 28 June 2008). (reduce uncertainty with data assimilation)

We test hydrologic models against ‘observed’ flow data This data is usually stage data transformed to flow via a rating curve Gauging location at Barnett’s Bank, Wairau River Significant errors can occur: Stage/Velocity measurement errors Rating Curve interpolation errors Rating Curve extrapolation errors Cross-section change due to vegetation growth or bed movement Worst Case Scenario? Example 3— uncertainty in flow measurements

Solution: Exact Specification of Flow Error By specifying uncertainty in validation data we give our models a ‘fair hearing’ Model structure/parameterisations are not forced to compensate for poor data Stage (m) Flow (cumecs) Uncertain Rating Curve Flow Quantiles McMillan, H., J. Freer, F. Pappenberger, T. Krueger and M. Clark, (in preparation) Impacts of uncertain flow data on rainfall-runoff model calibration and discharge predictions in a mobile-bed river. To be presented at the American Geophysical Union Meeting, San Francisco, CA, December 2008.

Outline A hydrologic emergency –The growing need for hydrologic models –The growing dissatisfaction with hydrologic models A research strategy for hydrologic modeling –The method of multiple working hypotheses –The dialog between the experimentalist and the modeler –The theory of diagnostic signatures –Complete representation of uncertainties in hydrologic simulations Development of a National Hydrological Model for NZ –Modeling approach –Model limitations

TopNet basin component Clark, M. P., D.E. Rupp, R.A. Woods, X. Zheng, R.P. Ibbitt, A.G. Slater, J. Schmidt, and M. Uddstrom, (2008) Hydrological data assimilation with the Ensemble Kalman Filter; Use of streamflow data to update the states in a distributed hydrological model. Advances in Water Resources, in press (available online 28 June 2008). Catchment processes

Extension to the river basin scale distributed hydrological model Catchment processes Network routing

Model calibration using inverse methods Water movement through a basin depends on basin characteristics; Approach: –Derive a-priori spatial distribution of model parameters; –Estimate a set of multipliers that adjust the model parameters for different parts if the basin. McMillan, H., and M.P. Clark (Water Resources Research: In Press) Rainfall-runoff model calibration using informal likelihood measures within a Metropolis-Hastings search algorithm: Case study in a catchment with heterogeneous geology. Flow Prediction: Pumice Subcatchment Flow Prediction: Greywacke Subcatchment

Model calibration using inverse methods Water movement through a basin depends on basin characteristics; Approach: –Derive a-priori spatial distribution of model parameters; –Estimate a set of multipliers that adjust the model parameters for different parts if the basin. McMillan, H., and M.P. Clark (Water Resources Research: In Press) Rainfall-runoff model calibration using informal likelihood measures within a Metropolis-Hastings search algorithm: Case study in a catchment with heterogeneous geology. Flow Prediction: Pumice Subcatchment Flow Prediction: Greywacke Subcatchment

Model limitations Model structure may be poorly suited to basins where the model is applied –Inappropriate equations –Missing processes Model parameters may be assigned unrealistic values –Standard model calibration methods can identify unrealistic parameters that compensate for weaknesses in the model and errors in the data –Individual model components may not mimic the processes they are intended to represent, even though model simulations of streamflow appear reasonable (right answers for the wrong reasons) Estimates of model uncertainty are inadequate –Compensatory effect of model errors

Summary The holy grail of computational hydrology –Build models that mimic nature –Reliably quantify model uncertainty Are we there yet? –Community has made incredible progress recently –Several exciting research directions to follow –…but substantial room for improvement What about model applications? –Provide the best guidance given the tools that are available –Let’s work together to improve our models