Statistical Methods for Assessing Compliance – case studies Task 3.1B Anja Duffek, Hannah Green, Markus Lehmann, John Batty

CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty Task 3.1B Background and aims of task Results of the case studies Discussion Points Directive 2008/105/EC “… Member States may introduce statistical methods, such as a percentile calculation, to ensure an acceptable level of confidence and precision for determining compliance with the MAC-EQS.” 17-Nov-2011 CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty Task 3.1B – Background Assessing compliance with MAC-EQS (a) What type of monitoring regime? Typically monthly (not continuous) (b) What level of protection is in the standard? Acute short-term exposure Uncertainty In the standard (e.g. variable assessment factors) From intermittent monitoring 17-Nov-2011 CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty Task 3.1B – Background Two philosophies regarding protection goals and what risk of wrong conclusions is acceptable : High confidence of failure with small risk of false accusations (‘benefit of doubt’) E.g. if action to secure compliance is expensive need to be confident it is necessary At lower levels of confidence would increase monitoring frequency to improve understanding of environmental risk Low confidence of failure with small risk of wrongly concluding compliant (‘fail safe’) Highly protective approach 17-Nov-2011 CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty Task 3.1B – Progress Paper to CEMP 25/11/10 Questionnaire on MAC compliance (April 2011) Monitoring frequencies 4 – 12 samples per year Use of statistical methods Paper to CEMP 17/11/11 with case study examples of different assessment approaches Single site example (DE) Across a number of sites (UK) 17-Nov-2011 CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty Task 3.1B – Testing Approach in line with ISO 5667-20 Water quality – Sampling —Part 20: Guidance on the use of sampling data for decision making – Compliance with thresholds and classification systems. Use specified percentile with an assigned confidence of failure. 17-Nov-2011 CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty Task 3.1B – Key issues Compliance depends on sampling frequency Probability of at least 1 sample failing 17-Nov-2011 CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

Task 3.1B – probability of failure Probability a site exceeds an allowed time in failure for different sampling frequencies and observed fails >50% confidence (face-value) >95% confidence (significant) 17-Nov-2011 CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

Task 3.1B – Isoproturon example (DE) Response Additional intensive daily measurements showed short term pollution peaks occur over days and correlate with hydrological regime  measures and source controls to reduce the short-term emissions 17-Nov-2011 CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

Task 3.1B – Example – Isoproturon (DE) 17-Nov-2011 CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

Task 3.1B – Example – 2009 MAC fails (UK) LoQ issues 17-Nov-2011 CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

Task 3.1B – Statistical Methods – Why? Bring consistency, recognising sampling regime Identify the most significant failures Defensible judgement for legal action Target effort and resource to secure compliance Level playing field Fair comparison between places They are not a means to ignore failure All MAC exceedences should be investigated To identify any appropriate pollution control measures 17-Nov-2011 CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

Task 3.1B – Discussion points Choice of percentile and confidence level depends on expert judgement 95 percentile (if monthly sampling) ? 50% confidence identifies sites for further investigation into environmental risk (frequency, magnitude, duration of short-term pollution peaks) Can identify need for broad scale source control ? 95% confidence – identify actions to secure compliance Ceiling limit to respond to very high levels? Should balance risk of false accusations (benefit of doubt) and risk of not recognising failure (fail-safe) 17-Nov-2011 CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty