1. SEPA Bathing Waters Signage Calum McPhail Environmental Quality Unit manager Ruth Stidson Bathing Waters Signage Officer

2. Contents  SEPA beach signage – overview and results  Development of the SEPA Signage Prediction Tool  Development of future modelling systems

3. Background on Bathing Waters  Scotland has had problems of poor quality bathing water in some areas  Combination of diffuse pollution, especially on the west coast, and CSO discharges  For some sites meeting the potential new Directive will be challenging

4. Signage Overview  SEPA makes a daily water quality prediction, relating to the EU standards for bathing water, at the 10 signage sites throughout the bathing season  This is based on relevant environmental (mainly rainfall) events from the previous two days  This information is then displayed at the beach via an electronic variable message sign and on the web and phone line.

5. Example of electronic beach sign (at Prestwick) and alternative sign face legends

6. EC Bathing Water with signage EC Bathing Water SEPA Bathing Waters Signage Scottish Executive initiated & funded Run as a project in 2003 & 2004 Now in place at 10 beaches 2005 - 2007

8. Based on 683 samples

10. Signage validation results 2003 - 2005

11. Aberdeen Signage results 2004 & 2005 (excellent, good, or poor status predictions)

12. First year - 2003 sign management  Signs set to predict poor quality if:  24 hr rain greater than 10 mm or  48 hr rain greater than 15 mm  These were known as ‘decision trigger levels’  This worked well but there was scope for improvement

13. Development of the SEPA Signage Prediction Tool  Known relationship between rainfall and coliform levels  SEPA archives for historical datasets (e.g. water quality results and environmental drivers)  Understanding site response to inputs, or recent infrastructure improvements/schemes  Predicting diffuse pollution:  rain events  run-off from fields  increased coliform levels

14. Further developing the relationship between rain & coliforms  For each of the signage sites:  Relevant rain and river gauging stations were identified  At each raingauge, 1 to 5 days rain was correlated against faecal and non-faecal coliforms and faecal streptococci  Strongest relationships at each site were identified

17. Conversion into a useful tool  Possible to use relationship to predict coliform levels on any given day  Use this information to predict if the coliform levels will be in exceedance of EC guidelines  Development of signage prediction tool to refine decision trigger levels

18. What does the tool do?  Site specific  Enables the testing of potential decision trigger levels against actual data from 2000/1 onwards  Instantly allows the user to see the outcomes of trial decision trigger levels  Allows the user to alter the coliform exceedence limits in anticipation of the new Directive

19. Copy and paste in data set: Coliform values (up to 3 types) Rainfall (up to 4 gauges and 5 time periods) River flow (up to 2 gauges) Years to include Microbiology values Start testing rain and river values SEPA Signage Prediction Tool Immediately see past results

20. Strengths of the current tool  Very effective at predicting compliance against mandatory standard in Scotland  98% correct or precautionary in 03 & 04, 99% in 05  Simple to:  Use  Update  Apply to additional sites  Transparent

21. Easy adaptations  Very easy to adapt for other factors IF they can be considered as a single variable  E.g. if sunshine is a major driver can add in test as per river flow  Input sunshine (eg) as hours  Use test such as ‘if sunshine < x hours predict poor’  Use tool to test different values of x  Can use similar technique for wind, tide, telemetered CSO spills etc

22. More challenging adaptations  It is possible to consider combined factors  IF rain > 10 mm → poor  IF rain > 8 mm AND Wind = onshore OR tide = incoming → poor  Rapidly becomes more complex !

23. Bathing Water Future Models Colin Gray Data Analyst Modeller, SEPA  Aim:  To try and improve current models  To develop models for future, more stringent EU directive  To utilise new developments and software within SEPA

24. Data Available to Models  Rainfall data for relevant gauges per beach  River flow data  High tide times & sample times  Weather  Salinity  Can not be used in predictions currently due to sampling methods  Wind direction and speed  Beach usage

25. Conclusions from Data Analysis  No clear cut splits between all fails and passes for current or future EU rules  Although more extreme levels of rain fall and river flow tend to be failures, there is a large amount of overlap at more moderate (normal) levels  Similar results from several beaches  Important factors are:  Rain fall over time periods  Total rain  River flow  High tide time  Salinity  No trends seen in weather, wind speed or direction, beach usage or other miscellaneous data  Very difficult to visualise multiple factor data and trends  E.g. if x is over this, and y is under this while z is this, then beach will fail  IMiner and S-plus modelling techniques can assist

26. Software and Techniques  SEPA Statistical and Modelling software  S-plus  ideal for data manipulation and graphing  Insightful Miner (IMiner)  designed for producing work flows and modelling large amounts of data  Both are closely integrated  Models Used  Scoring Method  Classification Trees (a.k.a. Decision Trees)  Neural Networks  Classification Regression  Naïve Bayes

27. General Principles  Performance will be very dependant on true trends being present  Lack of failure data can lead to models using incorrect assumptions  Computers know nothing of science!  All models need human validation and adjustment to ensure making sensible assumptions and relationships

28. Decision trees  Uses method called RPART (recursive partitioning)  Builds braches which represent relationships between factors  Helps highlight key factors affecting a bathing water  Easy to interpret and adjust  Very fast to generate and utilise  Widely used in other industries e.g. pharmaceutical  Easy to implement as an everyday prediction tool  Suited well to bathing water predictions  Uses predefined conditions to determine prediction

29. Decision Trees  Irvine, current EU Directive

30. Performance for Irvine  For Current EU directive:  Perfect prediction  Tree is very simple and scientifically reasonable

31. Performance for Irvine  For Future EU directive:  Tree becomes very complex

32. Performance for Irvine  For Future EU directive:  Although performs well:  No way of controlling the fact that it is preferable to predict a ‘Pass’ as a ‘Fail’ instead of vice versa as in above  Have altered the method to allow for weighting  Some final splits in the tree are likely not to be based on actual reason for failing  Splits will highlight difference in data for results and may have no scientific relevance

33. Summary of Decision Trees  Decision trees appear to provide a good method of modelling beaches  Easier to interpret and adjust than other methods  Better performance than scoring, neural networks or logistic regression  However careful manipulation of the weighting may be required  Care needed to ensure final splits are scientifically valid  Missing data needs to be handled in a standard method

34. Future Work  Decision tree models  Derive models for all bathing waters using 2003 and 2004 data  Then use 2005 data to assess performance  Ongoing project to assess usage of rain radar to improve predictions  Potential network expansion to new sites

35. A View from the Sun - Nov 2004

37. Models Developed  Scoring Method  Attempt to score factors  Add scores at the end and if over a certain number then it is predicted a fail  Very similar to current method but more flexible  Could improve predictions at Irvine but more difficult at Saltcoats  Very time consuming to develop and very ad-hoc  Neural Networks  Uses IMiner internal neural network method  Produces very complex relationships between factors  Can be very powerful and highly predictive  Is very dependant on quality of training data  Almost impossible to adjust or interpret  Is unlikely to perform well in new circumstances  Logistic Regression  Produces an equation to represent the bathing water  Can weight the outcome of pass or fail  May over generalise factors as applies one coefficient to each

38. Signage Roles 2003 – 2004 Scottish Executive initiated and funded the project as a pilot SEPA determined the daily water quality predictions 2005 - 2007 SEPA to run beach signage Faber Maunsell Installing and maintaining the electronic signs and communication linkages Local Authorities, Clean Coast Scotland, Public involvement and understanding of the signage project

39. BW Signage working: SEPA systems WEBSITE PHONELINE PHONE TEXT

40. Incorporating into predictive tool  If tide / wind / sunshine is to be incorporated into the tool, it needs to be a secondary consideration to rainfall  Say statistical tests show that an onshore wind at Irvine significantly increases coliform concentrations  If the trigger level for Irvine is set at 12 mm of rain within 24 hours, can code into Excel that:  IF rain is x % lower than the trigger level AND the wind is onshore, then OVERRIDE to POOR  IF rain is x % above the trigger level AND the wind is NOT onshore, then OVERRIDE to GOOD  Can potentially code for tide, wind and sunshine for multiple triggers, however this does considerably increase the complexity of the tool