1. 1 © Trinity Horne Limited 2012 1 Analysing pollution and targeting prevention activity in a UK water company Alec Ross Senior Statistician Luke Cooper GIS Specialist

2. 2 Analysing pollution and targeting prevention activity in a UK water company Water Statistics User Group 16 May 2012

3. 3 © Trinity Horne Limited 2012 3 Contents Introducing Applied Analytics Pollution Analysis Continuous Improvement

4. 4 © Trinity Horne Limited 2012 4 Which factors really matter? Which factors don’t? How much can we improve? How do we do it? Input (e.g. Investment) Output (e.g. Performance) DATA Core Working Team Combining SMEs, Data & Analytics to Guide, Interpret and Validate Deep Insights + Clear Presentation + Mathematical Model Collect Data Analyse Combine Introducing Applied Analytics SMEs

5. 5 Case Study: Waste Water Pollution Analysis

6. 6 © Trinity Horne Limited 2012 6 Water and sewerage company performing poorly on OFWAT’s pollution incident metric for England and Wales. They invested significant time, effort and various approaches in trying to reduce pollution incidents, including data analysis. Despite some improvements the number remained stubbornly above an acceptable level. They felt they had a good strategy in place to reduce pollutions caused by failures at pumping stations, sewage treatment works and CSOs. They wanted us to address pollutions on gravity sewers. Formalised in the analysis plan: Project background & objectives “Identify and quantify the causes of pollution and the effectiveness of historical improvement activities and apply that knowledge to identify the highest risk network areas and to recommend the most effective actions to reduce that risk.”

7. 7 © Trinity Horne Limited 2012 7 Which factors really matter? Which factors don’t? How much can we improve? How do we do it? DATA Combine Recap project stages SMEs Report Collect Data Analyse Immerse Core Team Guides & Validates Throughout Action 1 2 3 4 5 6

8. 8 © Trinity Horne Limited 2012 8 What we did … Immersion (6 wks)Pre-processing (5 wks) Analysis (6 wks)Report (2 wks)Action Data collection (3 wks) sewer dimensions age property density criticality ownership locality food establishments land use watercourse weather survey grade maintenance

9. 9 © Trinity Horne Limited 2012 9 What we found … Blockage Risk factors for sewer blockages Pollution Risk factors for blockages and external floods that become pollution events Flooding Risk factors for blockages that become external floods Fixed risk factors Index of Insight for highest risk group (1.0=same as random) 3.6 2.17.8 21.6 16.4 (27.2 max)

10. 10 © Trinity Horne Limited 2012 10 Statistical approach to separating high and low risk sewers Actual risk scores used, not predicted Easy to understand and use Output quantifies insight Produces risk scores that can be applied to all sewers How we found it...

11. 11 © Trinity Horne Limited 2012 11 Entire asset base prioritised in one table Quantified risks and benefits of taking action Allowed assessment of expenditure needed to improve pollutions by different amounts How we prioritised risk...

12. 12 © Trinity Horne Limited 2012 12 Risk scores for all gravity sewers Evidence-based decision-making: identified important factors Numbers supported decisions about appetite to reduce pollutions Sharper focus resulting from the risk table plus RAG analysis plus COST per pollution incident Powerful argument for capital investment for executive and board Strong evidence to convince regulator that they were taking effective action to deal with the problem Plus Data Gap analysis Estimates of maintenance effectiveness built into cost-benefit analysis Impact

13. 13 © Trinity Horne Limited 2012 13 Integrated with GIS

14. 14 © Trinity Horne Limited 2012 14 Continuous Improvement The decision tree model for pollution risk scoring of sewers represented the best possible insight from the data available at the time of analysis It was, therefore, imperfect and incomplete – like all models of real world situations The implementation of the subsequent investment programme represented an opportunity to capture additional data that could be used to refine and improve the model Insights from new data were added to the model to provide a single continually improving reference for risk scores and preventive activity

15. 15 © Trinity Horne Limited 2012 15 Continuous Improvement using Bayes The decision tree model was the ‘gold standard’ but we had emerging data that could update and guide existing risk scores. The updates supplemented but could not supplant the original model. Posterior (Probability) = Prior (Probability) X Likelihood Updated Probability = Original Probability X A/B where A = the proportion of failed sewers with the characteristic and B = the proportion of all sewers with the data We used Bayesian updating to calculate a new relative risk or likelihood But we kept the overall blockage or pollution risk constant within risk groups.

16. 16 © Trinity Horne Limited 2012 16 Application to Sewers & Service Grade SEWER GROUP = MODEL NODE: Prior Risk = Gold Standard = Fixed SELECTED SEWER: Initial Prior = Node Risk Service Grade for SEWER Relative Risk for Service Grade Posterior for SEWER Adjust risk for OTHER SEWERS in GROUP to MAINTAIN GOLD STANDARD Relative Risk = Proportion of Failed Units with That SG Proportion of All Units with That SG Posterior Risk = Prior Risk X Relative Risk

17. 17 © Trinity Horne Limited 2012 17 Practical Considerations Relative risks for different variables are combined as a simple product: The relative risks are derived using incomplete, imperfect data plus analysis and expert judgement. If the data was better and we had more confidence we would use it in the original model that produced the priors. This method uses what we can. Some relative risks relate to blockage and some to pollution. Blockage posteriors are calculated separately and combined with pollution priors. Then pollution posteriors are calculated. We update the risks where we have information and, if appropriate, update risks on other sewers without information to keep the group risk constant. Implemented through a standalone Excel application using VBA L1L2L3L4L5L6L7 XXXXXX Posterior [Updated Risk] Prior [Original Risk] =X

18. 18 © Trinity Horne Limited 2012 18 The ‘Applied Analytics’ methodology implemented by the Trinity Horne statisticians identified sewers at risk of pollution and quantified this risk for different sewer cohort groups. The decision trees classification approach generated useful statistical insights that are being used to reduce pollution. This was made possible by a wide-ranging exploration of factors relevant to sewer blockage and pollution and the deep insight that comes from bringing subject matter experts together to guide the process and give access to previously untapped data sources. Bayesian updating allowed us to use emerging data to improve the targeting of sewers at risk of pollution while the programme was ongoing. Relative risks were assigned using a combination of analysis and judgement. The Bayesian approach has limitations but negative effects were limited by holding the group risk constant. Summary

19. 19 Questions?