1. Model calibration using

2. Pag. 5/3/20152 PEST program

3. Pag. Major steps of hydrologic modeling (Hydrologic) data Relate to model inputs Parameter estimation or model calibration Model -- parameters Predictions Prediction uncertainty Societal decisions

4. Pag. What is Model Calibration? 5/3/20154 PEST program Adjustment of the parameters of a mathematical or numerical model in order to optimize the agreement between observed data and the model's predictions. American meteorological society The process of adjusting model inputs so that model calculations match what we measure in the real-world. So, a good model calibration is not sufficient to develop a good model. Also need good data, a good model, and an adequate optimization method.

5. Pag. Real world data 5/3/20155 PEST program

6. Pag. Attempt 1 for model calibration 5/3/20156 PEST program

7. Pag. Attempt 2 for model calibration 5/3/20157 PEST program

8. Pag. Attempt 3 for model calibration 5/3/20158 PEST program

9. Pag. Calculation of residuals for model calibration 5/3/20159 PEST program

10. Pag. Statistical measures involving residuals 5/3/201510 PEST program

11. Pag. 5/3/201511 PEST program PEST is a nonlinear parameter estimation package capable of estimating parameters for any computer model. It solves a nonlinear least squares problem and minimizes the differences between the model's outputs and field measurements such as the calculated and measured discharges. PEST adapts to the model, the model does not need to adapt to PEST.

12. Pag. 5/3/201512 PEST program Minmizing Sum of Squared Residuals (SSR) The use of this equation, as an objective function to be minimized, implies certain assumptions about the residuals (Clarke, 1973): a)That the time series of residuals have zero mean and constant variance b)That the time series of residuals does not have a significant autocorrelation If confidence intervals are to be given for the estimated model parameters, then: c) residuals have to be distributed normally.

13. Pag. The role of PEST 5/3/201513 PEST program 1.(initial) Parameter Sensitivity Analysis : conducted on a larger parameter space, which allows excluding from the further analysis those parameters that have a relatively small impact on the model response.(initial) Parameter Sensitivity Analysis 2. Calibration: (parameter estimation): PEST “calibrates” a model by reducing the discrepancies between model outputs and field observations to a minimum in the weighted least squares sense.Calibration 3. Predictive analysis: Once a parameter set has been determined for which model behavior matches system behavior as well as possible, it is then reasonable to ask whether another parameter set exists which also results in reasonable simulation by the model of the system under study.Predictive analysis

14. Pag. PEST template, instruction, and control files 5/3/201514 PEST program 1.Template file: tells PEST how to write parameters in the model input fileTemplate file 2. Instruction file: tells PEST where it has to look for the model-generated output, and how to read it e.g. which column contains the simulated discharges.Instruction file 3. PEST conrol file: contains:User-supplied initial parameter values, range of permissible values that a parameter can take, observations, maximum number of iterations, etc.PEST conrol file

15. Pag. 5/3/201515 PEST program Optimization algorithm can be classified into two categories: 1.Global search e.g. Shuffled Complex Evolution-University of Arizona (SCE-UA, 1992). searches the entire parameter space and do a controlled random search in the direction of global optimum. 2. Local search e.g. PEST program (Doherty, 1994). Estimate parameters with observations using a gradient-based method.gradient-based method Gradient methods are much faster but can get stuck in local minima.

16. Pag. Local and global minima 5/3/201516 PEST program Response surface for an inverted goal function. Local minima are represented as peaks. It shows that for any given goal function, there exits many parameter sets for which the goal functions are not significantly different from each other, i.e., there are many potential solutions based on quite different parameter sets.

17. Pag. 5/3/201517 PEST program Based on Objective Function, the automated calibration procedures can be classified as: 1.Single objective procedures e.g. mean squared errors in PEST program. PEST trys to minimize the objective function in order to obtain a better fit between predicted and observed values. PEST offers the best parameter set. 2. Multiple objective procedures. The generated output from this approach is a set of solutions (called pareto solutions) instead of a unique solution.

18. Pag. When is a Model Calibrated? 5/3/201518 PEST program As the problem of parameter optimization is not unique, it is important that we define when a model is calibrated and what the magnitude of the prediction uncertainty is.

19. Pag. 5/3/201519 PEST program To see when a model is calibrated: 1.Calculate the 95% prediction uncertainty

20. Pag. 5/3/201520 PEST program 2. Normally, more than 80 percent of obsevations should be braketet by 95% confidence intervals (the upper and lower limits of the best parameter sets) a) is not a parameter calibration problem, b)calibration can obtain smaller uncertainty distribution, c) it can be expected that some measured data can fall outside the 95PPU.

21. Pag. 5/3/201521 PEST program 3. Upon reaching the above criteria, if there exits a significant r 2 and/or Nash-Sutcliff coefficient between the best simulation and the measured data for a calibration and a test (validation) data set, then the model can be considered calibrated.

22. Pag. Calibration issues 5/3/201522 PEST program After analysing residuals to check whether they are independent, homoscedastic and normally distributed with zero expectation; We need to consider following questions: 1.What is the data quality and error (data uncertainty)? How does it affect predictions? 2.What is the sensitivity of model matches to changes in model inputs? (non- uniqueness in solutions) 3.How good is good enough, for a well calibrated model?

23. Pag. 5/3/201523 PEST program Thank you

24. Pag. 5/3/201524 PEST program

25. Pag. 5/3/201525 PEST program WetSpa model parameter sensitivity for the Illinois River basin

26. Pag. 5/3/201526 PEST program Streamflow simulation using WetSpa at Illinois River basin, Oklahoma, USA

27. Pag. 5/3/201527 PEST program

28. Pag. PEST input template for WetSpa 5/3/201528 PEST program

29. Pag. PEST Instruction file for WetSpa 5/3/201529 PEST program

30. Pag. PEST conrol file for WetSpa 5/3/201530 PEST program