1. Complex Models in the Water Industry
Water Statisticians User Group
14 April 2010
Edward Glennie

2. Outline of Presentation
Replies to questionnaire
Modelling generally:
Sources of error
Methods of assessing the size of any errors
Two specific water industry applications
Optimising interventions to maintain a set of assets
Use of time series rainfall
Pointers for the future

3. Replies to questionnaire – Thank you!
Model description
Count
Asset failure probability, deterioration & influence of weather 5
Optimise investment in a set of assets 3
Per capita water consumption micro-component model 1
Economic level of leakage
Water resources with rainfall time series
DG5 flood predictor
Sewer network hydraulic modelling
Sewage treatment process models

4. Models and modelling issues
Model description
Count
Asset failure probability, deterioration & influence of weather 5
Optimise investment in a set of assets 3
Per capita water consumption micro-component model 1
Economic level of leakage
Water resources with rainfall time series
DG5 flood predictor
Sewer network hydraulic modelling
Time series rainfall and climate change projections -
Sewage treatment process models

5. Use of words
A model has outcomes (predictions, results), which are used to help make decisions
The outcomes are subject to errors (uncertainty)
The results are robust if the errors are small enough for us to take the decision with confidence

6. Walker et al (2003) quoted in Refsgaard (2007)
Sources of error
Source of error
Example
Context
Modelling asset failure rates – assume current operating conditions will apply in future
Input data
Modelling asset failure rates – early reporting of failures may be incomplete
Parameter uncertainty
Pipe roughness values in water distribution modelling
Model structure
Asset failure rates – use of linear model (or Weibull etc) to predict future failure rates
Model technical uncertainty
Numerical approximations, software bugs
Walker et al (2003) quoted in Refsgaard (2007)

7. Adapted from Brown (2004) quoted in Refsgaard (2007)
Types of error
Statistical
Size of potential errors in outcomes can be estimated
Qualitative
Potential errors in outcomes can be identified and described
Recognised ignorance
Aware of poorly understood potential errors
Unrecognised ignorance !
Adapted from Brown (2004) quoted in Refsgaard (2007)

8. Types of statistical error
If it is due to imperfect knowledge and could be reduced by more information, it is epistemic
If it is due to inherent variability and could not be reduced by more information, it is stochastic

9. Optimise interventions in a set of assets
Asset data
Renewal & planned maintenance data
Current and future failure rates
Failure rate model
Failure data
Weather data
System data, GIS data, judgement…
Actual failure impacts
Failure impacts model
Optimal interventions
Optimiser
Set of possible interventions

10. Failure rate modelling - methods
Divide assets into cohorts, regression / fit (e.g.) Weibull function
Bayesian approach: probability, severity & consequence of failure, partition assets, tune model using observed failures
Definitions of failure

11. Failure rate modelling - issues
Source of error
Example
Type
Context
e.g. Assume current ‘environment’ and operating conditions will apply in future
Qualitative or Recognised ignorance
Input data
e.g. Complete failure data may exist for only a few years
Civil assets are not operated to end of life failure
Qualitative?
Parameter uncertainty
Statistical
Model structure
Assumed form of model to predict future failure rates (linear, Weibull etc)
Model technical uncertainty
None?

12. Failure impact modelling - methods
Impacts: direct costs & customer service
Infrastructure, civil assets, M&E, ICA
Methods
GIS based e.g. for flooding using contours
Population potentially affected e.g. water supply low pressure
Use of actual data?
Large element of judgement

13. Failure impact modelling - issues
Source of error
Example
Type
Context
Input data
Parameter uncertainty
Model structure
Model technical uncertainty

14. Failure impact modelling issues (second try)
Is there a hierarchy of possible consequences for each failure?

15. Impact of failure – probability distribution

16. Failure impact modelling issues (second try)
Is there a hierarchy of possible consequences for each failure?
Major impacts result from a combination of several failures
Are non-failure impacts of deterioration taken into account? e.g. leakage, pump efficiency
Qualitative errors / recognised ignorance?

17. Optimisation Monetarise all impacts?
Use LoS targets as constraints in optimisation
‘It’s difficult to prove that you’ve reached the optimum’
Bring modeller’s and engineer’s perspectives together

18. Optimising investment – for discussion and development
Failure rate modelling
Cohort vs Bayesian approach
Failure impact modelling
Use of judgement: benchmark against other industries (nuclear? Little 1998)
Use of data to verify models (but gathering data costs money!)
Sensitivity analyses

19. Time Series Rainfall
Used because rainfall history affects system behaviour, as well as current rainfall, e.g. runoff depends on wetness of ground
Rainfall patterns are local
Actual time series are best, if they exist
Often time series have to be generated
Much data collection & research effort, but issues remain
(short duration storms)

20. Climate change and time series rainfall
How should a ‘current’ time series be modified to take account of climate change?
Focus on 1 in ~1 year storms, with sewer system storage volume in mind

21. Empirical relationships:
UKCP09 rainfall model
Parameters
Mean daily rainfall
Proportion of dry days
Variance
Skewness
Lag -1 autocorrelation
Time series generator
Daily rainfall time series
Empirical relationships:
1 hour to 24 hours
Hourly rainfall time series
Source: UKCP09 Guidance - Annex

22. Empirical relationship – 1 hour vs 1 day

23. What we did Specific location Historical 30 year hourly TSR Current
UKCP09
30 year hourly TSR
(100 times)
Scenario
UKCP
30 year hourly TSR
(100 times)
Summary HIST:
storm nrs. by depth & duration
Summary CNTR:
storm nrs. by depth & duration
Summary SCEN:
storm nrs. by depth & duration
Comparison:
CNTR ~= HIST?
Comparison:
Derive factors SCEN/CNTR
Adjusted historical TSR

24. A better way? Specific location Historical 30 year hourly TSR Current
UKCP09
5 parameters
Scenario
UKCP
5 parameters
Comparison:
CNTR ~= HIST?
Comparison:
Change factors SCEN/CNTR
5 parameters & uncertainties
5 parameters - changed
Adjusted historical TSR

25. Methods of assessing errors & their significance
Statistical estimates
Sensitivity analysis
Scenario analysis
Uncertainty matrix

26. Uncertainty matrix Source Type Importance Weight Uncertainty * Weight
Problem context
…………………………….
Input data
Parameter error
Model structure
Model technical uncertainty

27. Methods of assessing errors & their significance
Statistical estimates
Sensitivity analysis
Scenario analysis
Uncertainty matrix
Involve stakeholders
There are others – see Refsgaard (2007). Some also control / reduce errors.

28. Ideas to take forward Optimising investment Modelling of impacts
TSR & Climate change
Use basic parameters underlying the time series generator
Framework for understanding modelling errors
Consistent use: sources, types, how to assess (& minimise)

29. References
Brown JD, Knowledge, uncertainty and physical geography: towards the development of methodologies for questioning belief, 2004, Trans Inst. British Geographers vol.52 (6) pp
Little MP, Invited editorial in J Radio Prot, 1998, vol.18 p.239
Refsgaard JC et al, Uncertainty in the environmental modelling process – a framework and guidance, Environmental Modelling and Software, 2007, vol.22 pp
UKCP, 2009, UK Climate Projections,
Walker WE et al, Defining uncertainty – a conceptual basis for uncertainty management in model-based decision support, Integrated Assessment, 2003, vol.4 (1) pp.5-17