1. SQO 4/7/05 INCORPORATING MULTIPLE LINES OF EVIDENCE INTO SEDIMENT QUALITY OBJECTIVES Stephen B. Weisberg Southern California Coastal Water Research Project

2. SQO 4/7/05 BACKGROUND For many years, scientist have advocated a triad approach for evaluating sediment quality –Individual lines of evidence each have potential limitations –Most applications have been site-specific and based on best professional judgment There are many challenges in translating scientific concept into a state-wide regulatory framework –Standardizing interpretation –Ph.D. biologists vs. B.S. engineers

3. SQO 4/7/05 LARGEST CONCERNS YOU EXPRESSED Reconsider using a linear, numerical scoring system –Implies a linearity that may not exist –Assures equal weighting among different lines or evidence Enhance interaction with stakeholders –Integration framework is a combination of science and policy Place greater emphasis on application guidance RELATIONSHIP BETWEEN SCIENCE AND POLICY There are a spectrum of decisions –Selecting indicators and thresholds from individual lines of evidence –Joining multiple lines of evidence to make a station assessment –Joining multiple stations to make a water body (on site) assessment Science is about linearizing complex information Policy is about establishing thresholds along those gradients –Policy becomes predominant at the higher levels of information

4. SQO 4/7/05 Species Abundance RELATIONSHIP BETWEEN SCIENCE AND POLICY There are a spectrum of decisions –Selecting indicators and thresholds from individual lines of evidence –Joining multiple lines of evidence to make a station assessment –Joining multiple stations to make a water body (on site) assessment Science is about linearizing complex information Policy is about establishing thresholds along those gradients –Policy becomes predominant at the higher levels of information

5. SQO 4/7/05 LARGEST CONCERNS YOU EXPRESSED Reconsider using a linear, numerical scoring system –Implies a linearity that may not exist –Assures equal weighting among different lines or evidence Enhance interaction with stakeholders –Integration framework is a combination of science and policy Place greater emphasis on application guidance APPLICATION GUIDANCE Moving from site-based objectives to water body assessments Selecting implementation options where there are no chemical-specific thresholds Consider a sequential of phased sampling approach where effort is proportional to the nature of the problem

6. SQO 4/7/05 LARGEST CONCERNS YOU EXPRESSED Reconsider using a linear, numerical scoring system –Implies a linearity that may not exist –Assures equal weighting among different lines or evidence Enhance interaction with stakeholders –Integration framework is a combination of science and policy Place greater emphasis on application guidance

7. SQO 4/7/05 MLOE WORKPLAN Task 1: Define the lines of evidence that will be used for each beneficial use –Select the indicators for those lines of evidence Task 2: Develop a scoring system for each LOE –When a single indicator is available –When multiple indicators are available Task 3: Integrate scoring across multiple LOEs to develop a station assessment Task 4: Develop a strategy for incomplete data Task 5: Develop application guidance BENEFICIAL USE PROTECTION CATEGORIES Aquatic life (Infaunal) effects Human health effects Fish and wildlife effects MLOE will be used within each, but independent assessments will be conducted for each beneficial use

8. SQO 4/7/05 MLOE WORKPLAN Task 1: Define the lines of evidence that will be used for each beneficial use –Select the indicators for those lines of evidence Task 2: Develop a scoring system for each LOE –When a single indicator is available –When multiple indicators are available Task 3: Integrate scoring across multiple LOEs to develop a station assessment Task 4: Develop a strategy for incomplete data Task 5: Develop application guidance

9. SQO 4/7/05 CLASSIC TRIAD APPROACH Score each LOE as a binary decision This yields 8 narratively-interpretable integrated endpoints Relies heavily on best professional judgement –Assessment tools are too crude for binary decisions –Eight endpoints don’t capture uncertainty within individual LOE TYPICAL TRIAD INTERPRETATION ChemToxBenthosInterpretation +++ Impact highly likely: Contaminant-induced degradation in field evident ++- Impact likely: Toxic contaminants probably stressing sediment-dwelling organisms -++ Impact likely: Unmeasured chemicals contributing to toxicity +-+ Impact likely: Toxicity test not sensitive enough +-- Impact unlikely: Contaminants unavailable to organisms in the field -+- Impact unlikely: Unmeasured factors contributing to toxicity --+ Impact unlikely: Effects on benthos not due to sediment contamination --- Impact highly unlikely: Contaminant-induced degradation not evident

10. SQO 4/7/05 CLASSIC TRIAD APPROACH Score each LOE as a binary decision This yields 8 narratively-interpretable integrated endpoints Relies heavily on best professional judgement –Assessment tools are too crude for binary decisions –Eight endpoints don’t capture uncertainty within individual LOE

11. SQO 4/7/05 ALTERNATIVE APPROACH Create multiple categories for each line of evidence –Recognize uncertainty associated with a single threshold Leads to many more than 8 combinations –A headache, but provides more information Allows for assessment of endpoints that describe magnitude of effects Also allows for endpoints that describe confidence in the result –Confidence based on level of effect or agreement among individual lines of evidence

12. SQO 4/7/05 AGREEMENT ABOUT CATEGORIES Individual lines of evidence MLOE site assessment LIKELY ASSESSMENT CATEGORIES FOR INDIVIDUAL LINES OF EVIDENCE Reference condition Slight deviation from reference –Probably defined by measurement error Moderate effect Severe effect

13. SQO 4/7/05 AGREEMENT ABOUT CATEGORIES Individual lines of evidence MLOE: Site assessment

14. SQO 4/7/05 POSSIBLE SITE ASSESSMENT CATEGORIES Unimpacted Likely unimpacted Possibly impacted Likely impacted Clearly impacted Inconclusive? Reference Unimpaired Unimpacted Unaffected Protected Affected –Individual line of evidence Impacted –Site level –Merging multiple lines of evidence Impaired –Water body level –Merging multiple sites

15. SQO 4/7/05 POSSIBLE SITE ASSESSMENT CATEGORIES Unimpacted Likely unimpacted Possibly impacted Likely impacted Clearly impacted Inconclusive?

16. SQO 4/7/05 CHEMISTRY: Reference ReferenceMinor deviation Moderate effect Severe effect Reference Unimpacted Likely Unimpacted Inconclusive Minor deviation Unimpacted Likely Unimpacted Possibly Impacted Moderate effect Likely Unimpacted Possibly Impacted Likely Impacted Severe effect Inconclusive Possibly Impacted Likely Impacted Likely Impacted BenthosBenthos Toxicity

17. SQO 4/7/05 CHEMISTRY: Minor Deviation ReferenceMinor deviation Moderate effect Severe effect Reference Unimpacted Likely Unimpacted Possibly Impacted Minor Deviation Likely Unimpacted Possibly Impacted Likely Impacted Moderate Effect Possibly Impacted Likely Impacted Severe Effect Possibly Impacted Likely Impacted BenthosBenthos Toxicity

18. SQO 4/7/05 CHEMISTRY: Moderate Effect ReferenceMinor deviation Moderate effect Severe effect Reference Likely Unimpacted Possibly Impacted Likely Impacted Minor Deviation Possibly Impacted Likely Impacted Moderate Effect Likely Impacted Clearly Impacted Severe Effect Likely Impacted Clearly Impacted BenthosBenthos Toxicity

19. SQO 4/7/05 CHEMISTRY: Severe Effect ReferenceMinor deviation Moderate effect Severe effect Reference Inconclusive Possibly Impacted Likely Impacted Minor Deviation Possibly Impacted Likely Impacted Moderate Effect Likely Impacted Clearly Impacted Severe Effect Likely Impacted Clearly Impacted BenthosBenthos Toxicity

20. SQO 4/7/05 NUMERICAL SYSTEM Can be simpler than look-up charts –Also can provide more gradation for prioritization or trends analysis Assumes a linearity Assumes an equal weighting Assumes the stakeholders community counts it

21. SQO 4/7/05 MLOE WORKPLAN Task 1: Define the lines of evidence that will be used for each beneficial use –Select the indicators for those lines of evidence Task 2: Develop a scoring system for each LOE –When a single indicator is available –When multiple indicators are available Task 3: Integrate scoring across multiple LOEs to develop a station assessment Task 4: Develop a strategy for incomplete data Task 5: Develop application guidance

22. SQO 4/7/05 APPLICATION GUIDANCE Inherently a policy issue Stakeholder advisory committee has been tasked to do this They have formed subcommittees to develop guidance for three applications –303d listings –Dredging decisions –NPDES permitting

23. SQO 4/7/05 TECHNICAL SUPPORT Uncertainty analysis Sampling design guidance Phased sampling design Guidance for identifying contaminants of concern

24. SQO 4/7/05 UNCERTAINTY ANALYSIS Assessing uncertainty at the station level –Measurement variability –Assessment tool error Defining background level of “impact” –No system is likely to have completely unimpacted sites –Identify systems (or subsystems) that are least influenced –Use project data base to create a frequency plot for those systems

25. SQO 4/7/05 TECHNICAL SUPPORT Uncertainty analysis Sampling design guidance Phased sampling design Guidance for identifying contaminants of concern

26. SQO 4/7/05 IDENTIFYING CONTAMINANTS OF CONCERN Chemical specific thresholds –Don’t want to rely on chemistry as a sole means for impairment decision, but chemical thresholds can be useful guidance for implementation actions Spatial gradient analyses –Probably better in concept than practice Sediment TIES –This is a key area for scientific advance in support of management

27. SQO 4/7/05 MLOE WORKPLAN Task 1: Define the lines of evidence that will be used for each beneficial use –Select the indicators for those lines of evidence Task 2: Develop a scoring system for each LOE –When a single indicator is available –When multiple indicators are available Task 3: Integrate scoring across multiple LOEs to develop a station assessment Task 4: Develop a strategy for incomplete data Task 5: Develop application guidance

28. SQO 4/7/05 WHICH INDICATORS? Infaunal effects –Sediment chemistry –Sediment toxicity –Benthic infaunal assemblage Human health –Sediment chemistry –Fish/bivalve tissue chemistry Fish/wildlife –Sediment chemistry –Tissue exposure –Biological effects

29. SQO 4/7/05 MULTIPLE INDICATORS WITHIN A LINE OF EVIDENCE Multiple toxicity tests available for the site –Chronic and acute tests Multiple ways to interpret the same data –Equilibrium partitioning vs. empirical thresholds Several alternatives for integrating such data

30. SQO 4/7/05 POSSIBLE SCORING APPROACHES Average score Worst score –They each measure different things Prioritizing among tests –Most sensitive test –Least sensitive test –“Best” test Highest quality data Some combination of frequency and severity

31. SQO 4/7/05 PROPOSED SCORING WHEN MULTIPLE INDICATORS ARE MEASURED Severe effectSevere effect for one indicator and concordance among indicators Moderate effectSevere effect for one indicator or concordance among indicators Minor deviationMeasured effect in at least one test ReferenceConcordance among all tests that there is no effect

32. SQO 4/7/05 WHAT IF ONLY TWO LINES OF EVIDENCE ARE AVAILABLE? Looking for a combination of concordance and magnitude Unimpacted: No effect from either indicator Likely unimpacted: Small effect with no indicator, but no effect for the other Inconslusive: Large effect with one indicator but no effect for the other Likely impacted: Some effect for both indicators Clearly impacted: High effect for both indicators