1 Introduction G. Thirel and V. Andréassian IAHS Hw15 22 July 2013

2

3

4

5 Modelling is like painting Catchments are hyper-dynamic systems: they change continuously For the sake of our ‘portrating’, we need to make simplifying assumptions The risk: that the simplifying hypotheses cause a catchment non- stationarity artefact

6 Non stationarity makes the life of hydrologists miserable Identifying parameters is already not an easy task within the stationarity hypothesis… … it is much worse when changes which we have neglected turn out to have a significant impact on the calibration process

7 Goal of this workshop To provide a factual diagnosis: describe / document the problem –based on common catchments –also on other datasets Can we agree on the problem? On how to assess it, numerically and graphically? Investigate solutions

8 Workshop preparation group Guillaume Thirel Valérie Borrell-Estupina Sandra Ardoin-Bardin Julien Lerat Olga Semenova Francesco Laio

9 Outline of this presentation The dataset The calibration and evaluation protocol Some results IAHS Hw15 22 July 2013

10 Outline of this presentation The dataset The calibration and evaluation protocol Some results IAHS Hw15 22 July 2013

11 Need for a dedicated website FOR DEFINING THE COMMON FRAMEWORK AND FOR PROVIDING THE COMMON DATABASE Website address: Description of the dataset for each basin Description of the calibration and evaluation protocol Possibility to download the data (password protected) IAHS Hw15 22 July 2013

12 The dataset 14 RIVER BASINS SHOWING NON-STATIONARITIES IAHS Hw15 22 July 2013

13 The dataset 14 RIVER BASINS SHOWING NON-STATIONARITIES Basins sizes from 0.2 km² to 100,000km² Several types of non-stationarities encountered: Temperature increase Precipitation change or high variability Urbanization Forest cover modification Period: variable according to the considered basin IAHS Hw15 22 July 2013

14 Which data? DATA COLLECTED FROM MANY PARTNERS Variables: precipitation (P), temperature (T), potential evapotranspiration (PE), discharge (Q). What we provided: basin-wide aggregated values of P, T and PE Q at the outlet (repartition of altitude within the basin if available) Time step: daily IAHS Hw15 22 July 2013

15 Temperature increase THE KAMP (622 KM²), ALLIER (2267 KM²), DURANCE (2170 KM²) AND GARONNE (9980 KM²) RIVERS All located in Europe, impacted by snowmelt Allier Kamp Durance Garonne IAHS Hw15 22 July 2013

16 The case of the Kamp River VERY LARGE FLOODS IN 2002 P Q IAHS Hw15 22 July 2013 (Komma et al., 2007; Blöschl et al., 2008; Reszler et al., 2008)

17 The case of the Allier River CONSTRUCTION OF A DAM IN 1983 FOR SUSTAINING LOW FLOWS IAHS Hw15 22 July 2013 Impact on low flows

18 Precipitation change or high variability THE AXE CREEK (237 KM²) AND THE WIMMERA RIVER (2000 KM²) Millenium drought in Australia ( ) Wimmera River Axe Creek IAHS Hw15 22 July 2013 Q Q

19 Decrease in rainfall and deep water recharge between before 1970 and after 1971 Precipitation change or high variability THE BANI RIVER (100,000 KM²) PQ IAHS Hw15 22 July 2013

20 Precipitation change or high variability THE GILBERT AND FLINDERS RIVERS (AROUND 1900 KM²) Arid catchments under cyclonic heavy rainfall influence. Major flood in The Flinders River IAHS Hw15 22 July 2013

21 Urbanization THE FERSON (134 KM²) AND BLACKBERRY CREEKS (182 KM²) Located in the USA The urbanization modifies the hydrological response Percentage of urbanization IAHS Hw15 22 July 2013

22 Forest cover modification THE FERNOW (0,2 KM²) AND MÖRRUMSÅN (97 KM²) RIVERS AND THE REAL COLLOBRIER (1,4 KM²) The Fernow Experimental watershed: forest cut of the lower part of the basin, then forest cut of the upper part of the basin, then plantation of firtrees. The Mörrumsån River: a severe storm (Gudrun), led to loss of forest in January The Real Collobrier: forest fire in August IAHS Hw15 22 July 2013

23 The dataset RiverCountryAreaPeriodChangeProviders Fernow RiverUSA0.2km² ForestUSDA Forest Service Real CollobrierFrance1.4km² ForestIrstea & Météo-France Mörrumsan RiverSweden97km² ForestSMHI Ferson CreekUSA134km² UrbanizationUSGS & DayMet Blackberry CreekUSA182km² Urbanization Axe CreekAustralia237km² P decreaseVictoria data Warehouse Kamp RiverAustria622km² T increaseTU Wien, UFZ Gilbert RiverAustralia1907km² P variabilityQueensland Government Flinders RiverAustralia1912km² P variability Wimmera RiverAustralia2000km² P decreaseVictoria data Warehouse Durance RiverFrance2170km² T increaseEDF Allier RiverFrance2267km² T increaseMétéo-France & Banque Hydro Garonne RiverFrance9980km² T increase Bani RiverW Africa103390km² P decreaseDMM & DNH

24 Outline of this presentation The dataset The calibration and evaluation protocol Some results IAHS Hw15 22 July 2013

25 The protocol A COMMON CALIBRATION AND EVALUATION FRAMEWORK Common calibration / evaluation periods Common minimum set of metrics Possibility that I produce a set of metrics and plots for the modellers (providing that they sent to me their simulations) Modellers are free to do more! IAHS Hw15 22 July 2013 Complete period Warm-up Time P1P4 P3 P2P5 Change

26 The protocol LEVEL 1: THE BEGINNER LEVEL Calibration has to be done on the “Complete period” or the model does not need calibration Models are run on the “Complete period” Evaluation is done on the “Complete period” + P1 to P5 IAHS Hw15 22 July 2013 Complete period Warm-up Time P1P4 P3 P2P5 Change

27 The protocol LEVEL 2: THE NORMAL LEVEL Calibration has to be done on each pre-defined sub-period P1 to P5 Models are run on the “Complete period” for each calibration Evaluation is done on the “Complete period” + P1 to P5 IAHS Hw15 22 July 2013 Complete period Warm-up Time P1P4 P3 P2P5 Change

28 The protocol LEVEL 3: THE EXPERT LEVEL The modellers found that their model failed at level 2 to deal with non- stationarities or could do better. They want to try to solve this issue, or at least to try to test solutions that could solve this issue. -> all solutions are allowed. Failing is fine, since it allows to discard a solution. IAHS Hw15 22 July 2013

29 The protocol THE METRICS Participants were asked to produce the following statistics on each sub-period: NSE and NSE on low flows (i.e. using 1/Q+ε instead of Q) Bias (Qsim/Qobs) Discharge quantiles: Q95, Q85, Q15 and Q05 Frequency of low flows (i.e. when Q<5% of mean Qobs) IAHS Hw15 22 July 2013

30 The protocol THE METRICS IAHS Hw15 22 July 2013 Participants were asked to produce the following statistics on each sub-period: NSE and NSE on low flows (i.e. using 1/Q+ε instead of Q) + their decomposition Bias (Qsim/Qobs) Discharge quantiles: Q95, Q85, Q15 and Q05 Frequency of low flows (i.e. when Q<5% of mean Qobs) KGE and its decomposition Nash and bias on sliding windows Flow regime Ranked discharges

31 The protocol THE GRAPHS Used for the bias, the Nash criteria, the KGE, and their decompositions For the quantiles and the frequency of low flows, the observed value is added Two different ways of showing the same thing IAHS Hw15 22 July 2013 The criterion value Six curves: one for each calibration Six values: one for each evaluation period The criterion value Six columns: one for each calibration Six lines: one for each evaluation period

32 The protocol THE GRAPHS Extension of some graphs for a 1-year frequency evaluation IAHS Hw15 22 July 2013

33 The protocol THE GRAPHS The discharges regimes and the ranked discharges -> one graph for each evaluation period IAHS Hw15 22 July 2013

34 The protocol THE COMPARISONS BETWEEN MODELS IAHS Hw15 22 July 2013 Not values, but differences between the criteria of two models Blue values indicate that this model has a higher criterion Red values indicate that this model has a higher criterion Blue values indicate that this model has a higher criterion Red values indicate that this model has a higher criterion A column compares a single calibration on each evaluation period A line compares each calibration on a single evaluation period Mod 1Mod 2 Mod 1 Mod 2 Over-estimation from model on top Δ Under-estimation from the model on top

35 Outline of this presentation The dataset The calibration and evaluation protocol Some results IAHS Hw15 22 July 2013

36 List of models used for the workshop 1k-DHM, AWBM, CLSM, COSERO, ECOMAG, GARDENIA, GR4J, GR5J, HBV, HYDROGEOIS, HYPE, HyMod, IHACRES, MISO, MORDOR, MORDOR6, SAFRAN-ISBA-MODCOU, SimHyd, SpringSim, TOPMODEL, Xinanjiang,… IAHS Hw15 22 July 2013

37 Brief presentation of some results from people who participated but could not come GARDENIA: D. Thiéry, BRGM, France COSERO: H. Kling, Austria SpringSim: A. Ramchurn, Australia IAHS Hw15 22 July 2013

38 Lumped model with slow compo- nents reservoirs Can take into account aquifer level measurements (not used here) 4 to 6 parameters Calibration metrics: MSE(sqrt(Q))+5%(Qsim-Qobs) Ran on 11 basins Ref: Thiéry, D. (2010) Reservoir Models in Hydrogeology, in Mathematical Models, Volume 2 (ed J.-M. Tanguy), John Wiley & Sons, Inc., Hoboken, NJ, USA. doi: / ch13 GARDENIA USED BY: Dominique Thiery BRGM, IAHS Hw15 22 July 2013

39 Low bias on calibration periods, but high bias on contrasted periods. The calibration on the complete period allows an « intermediary » solution but does not prevent from biased simulations. Calibrations on dryer periods gave higher soil reservoir capacities: the model tries to allow more evapotranspiration for compensating the lower Q. GARDENIA USED BY: Dominique Thiery BRGM, IAHS Hw15 22 July 2013 Bani Regime change from 1971

40 High bias on contrasted periods. Calibrations on dryer periods gave higher soil reservoir capacities: try of the model to allow more evapotranspiration for compensating the lower Q. GARDENIA USED BY: Dominique Thiery BRGM, IAHS Hw15 22 July 2013 Wimmera Millenium Drought

41 COSERO Continuous, semi-distributed rainfall- runoff model. Snow processes Soil moisture accounting (HBV-type) Surface-flow, inter-flow, base-flow (linear reservoirs) Nachtnebel et al. (1993) IAHS Hw15 22 July 2013 USED BY: Harald Kling Pöyry Energy GmbH,

42 COSERO Objective function: KGE on Q. Ran on 11 basins (i.e. all except US basins). Dam module (affects low flows) added for the Allier River. Riparian zone (affects evaporation) added for Australian rivers. IAHS Hw15 22 July 2013 USED BY: Harald Kling Pöyry Energy GmbH,

43 COSERO Allier IAHS Hw15 22 July 2013 USED BY: Harald Kling Pöyry Energy GmbH, New dam built in 1983

44 COSERO USED BY: Harald Kling Pöyry Energy GmbH, Wimmera COSERO GR4J Similar behaviour: clear over-estimation for the Millenium Drought Difference: no clear under-estimation of wet years for Cosero when calibrated on dry years IAHS Hw15 22 July 2013 Millenium Drought

45 SpringSIM New model, implemented to deal specifically with incorporation of long term droughts in the routine response to rainfall/evaporation of rainfall- runoff models 12 parameters IAHS Hw15 22 July 2013 USED BY: Avijeet Ramchurn Bureau of Meteorology,

46 SpringSIM IAHS Hw15 22 July 2013 USED BY: Avijeet Ramchurn Bureau of Meteorology,

47 SpringSIM Good simulations of the water volume IAHS Hw15 22 July 2013 USED BY: Avijeet Ramchurn Bureau of Meteorology, Bani

48 SpringSIM Low bias in contrasted periods IAHS Hw15 22 July 2013 USED BY: Avijeet Ramchurn Bureau of Meteorology, Bani Wet Dry Wet Dry

49 Thank you!

50 The protocol THE COMPARISONS BETWEEN MODELS Comparisons between the models Comparisons with observations Mod 1Mod 2 Mod 1 Mod 2 Over- estimation from model on top Δ Under- estimation from the model on top

51 The protocol THE GRAPHS The 10-year sliding windows plots IAHS Hw15 22 July 2013