Considerations for Optimal Monitoring Program Design CASQA 9th Annual Conference, Squaw Valley, CA Monitoring Workshop September 9, 2013

Design Challenges Size and complexity of MS4s Spatial and temporal variability Ephemeral nature of many conditions Difficult sampling logistics More complex problems with interacting causes

The Use and Abuse of QAPPs 99.99% of attention focused on sampling and lab methods 0.01% * of attention focused on Right questions? Right design to answer these questions? Right data analysis method? Does design meet requirements of data analyses? Do data analysis results actually address the questions? * Estimates accurate to +/- 0.0001%

Data Quality What is data quality? What are data quality objectives? The ability of the study and its data to answer core questions What are data quality objectives? Characteristics of the data that ensure its ability to answer core questions

Aspects of Data Quality USEPA’s PARCCS construct Precision Accuracy Representativeness Completeness Comparability Sensitivity But even these are not enough Presume that already have goals, questions, appropriate designs

Larger Context for Data Quality All elements must be completed Elements must be coordinated Goals & questions of paramount importance Technical aspects driven by goals & questions

Data Quality? Have conditions changed over time at Huntington Beach?

How Much Do We Understand?

Organizing Our Understanding

The Right Amount of Complexity

Collaborative Modeling

The Tool Bag A wide range of approaches to choose from Choice depends on state of knowledge, questions, logistics, time constraints, etc. No single right answer for all situations

The Right Conceptual Model Power analysis for regression of diazinon trends assuming steady decline over time Data courtesy of Alameda County

Or, Maybe Not Power analysis for change in diazinon concentrations assuming a step-function decline Data courtesy of Alameda County

Design Options Monitor all sites Monitor random subset of sites Monitor only fixed trend sites Monitor only known problems Compare all data to standards Implement special studies Use hybrid designs

Monitor All Sites Pros Comprehensive picture of conditions Simple sampling strategy Track trends over time Cons Expensive and time consuming Lacks frame of reference for comparison

Monitor Random Subset of Sites Pros Statistically valid estimate of conditions Track trends in overall condition Design can be optimized for efficiency Cons May miss problem areas Unable to track trends at specific sites Lacks frame of reference for comparison

Monitor Only Fixed Trend Sites Pros Tracks trends over time Design can be optimized for efficiency Cons Does not describe overall conditions Lacks spatial frame of reference for comparison

Monitor Only Known Problems Pros Efficiently focuses monitoring effort Measures progress in solving problems Cons Does not describe overall conditions Lacks broader frame of reference for comparison

Compare All Data to Standards Pros Provides consistent basis for comparison Assesses compliance with regulations Cons Difficult to prioritize actions if exceedances widespread Standards may not provide meaningful action thresholds for local conditions

Implement Special Studies Pros Focus on specific issues Improve understanding More efficient Defined endpoint Cons Applicability may be limited May require specialized expertise

Look at Deviations Bacteria in Aliso Creek Predictable patterns Focus monitoring on subset of year Improved $$ and statistical efficiency of monitoring

Control for Covariates

Focus on Derived Variables

Enterococci Aliso Creek: Exponential Regression Drop in mean concentration over 11 years of 1 – 2 orders of magnitude Drop from mean of 2500 to 90

Questions?