Steps 3 & 4: Evaluating types of evidence

Evaluate Data from the Case Define the Case List Candidate Causes Evaluate Data from Elsewhere Identify Probable Cause Detect or Suspect Biological Impairment As Necessary: Acquire Data and Iterate Process Identify and Apportion Sources Management Action: Eliminate or Control Sources, Monitor Results Biological Condition Restored or Protected Decision-maker Stakeholder Involvement Stressor Identification Step 3: Evaluate Data from the Case Step 4:

Use all available types of evidence to make an inferential assessment Types of evidence using data from the case Spatial/temporal co-occurrence Evidence of exposure or biological mechanism Causal pathway Stressor-response relationships from the field Manipulation of exposure Laboratory tests of site media Temporal sequence Verified predictions Symptoms Types of evidence using data from elsewhere Stressor-response relationships from other field studies Stressor-response relationships from laboratory studies Stressor-response relationships from ecological simulation models Mechanistically plausible cause Manipulation of exposure at other sites Verified predictions Analogous stressors italics indicates commonly available types of evidence

Basic analysis strategy Develop as many types of evidence, for as many candidate causes, as you can you won’t have all types of evidence, for all candidate causes most effective when you can compare results across candidate causes Show your work make your process transparent & reproducible make use of appendices

Definition Types of evidence are categories of relationships that provide logically distinct ways to support, weaken, or refute the case for a candidate cause.

Why so many types of evidence? Helps make sure that potential sources of evidence are not overlooked The label associated with a particular piece of evidence is not important, provided you: Are consistent across candidate causes Don’t double-count data

Use all available types of evidence to make an inferential assessment Types of evidence using data from the case Spatial/temporal co-occurrence Evidence of exposure or biological mechanism Causal pathway Stressor-response relationships from the field Manipulation of exposure Laboratory tests of site media Temporal sequence Verified predictions Symptoms Types of evidence using data from elsewhere Stressor-response relationships from other field studies Stressor-response relationships from laboratory studies Stressor-response relationships from ecological simulation models Mechanistically plausible cause Manipulation of exposure at other sites Verified predictions Analogous stressors italics indicates commonly available types of evidence

Spatial/temporal co-occurrence SUPPORTS Impairment occurs where stressor is present WEAKENS Impairment does not occur where stressor is present, or occurs where stressor is not present

Spatial/temporal co-occurrence SUPPORTS WEAKENS APPLICABLE DATA: Measures of proximate stressors & impairments Impaired & reference sites

Example: spatial co-occurrence Candidate Cause Measurement [total metals & ammonia (mg/L)] Upstream reference Watershed reference Impaired site Adverse change compared to references TOXICS Al 0.080 0.037 0.101 Yes Cd ND No Cr 0.005 Cu 0.004 Fe 0.395 0.208 0.695 Ni 0.001 Pb Zn 0.006 0.011 NH3 0.1

Spatial/temporal co-occurrence Issues & recommendations: Simple comparison – is exposure to proximate stressor greater where/when impairment occurs? Don’t consider whether magnitude of stressor is sufficient Magnitude considered under other types of evidence (e.g., stressor-response relationships from elsewhere) Consider uncertainty & variability in data set, but don’t rely on statistical tests Only use measurements of proximate stressor Other measurements are considered under “Causal pathway”

Causal pathway SUPPORTS WEAKENS Steps in causal pathway observed, and coincide with impairment WEAKENS Steps in causal pathway not observed, and do not coincide with impairment

Causal pathway APPLICABLE DATA: Measures of other steps in causal pathways & impairments Impaired & reference sites

Example: causal pathway Pathway supported compared to references Data Reference sites Impaired site Pathway supported compared to references Habitat Index [% of reference] 68 100 32 Yes BOD [mg/L] 1.1 1.6 Tree canopy moderate high low Bank stability 10 8 No Velocity 15 16 11 % Impervious surface Illicit discharge absent present Substrate composition % Boulder 33 25 % Cobble 50 % Gravel 13 % Sand 12 Candidate causes: ↓ dissolved oxygen; Δ food resources

Causal pathway Issues & recommendations: Similar to spatial co-occurrence, but uses data from entire causal chain When in doubt, assume a step exists Evidence of missing step is powerful; evidence of many intermediate steps increases confidence May be able to eliminate one pathway, but rarely can eliminate all pathways ↓ dissolved oxygen ↑ nitrogen ↓ fish ↑ algae

Stressor-response relationships from the field SUPPORTS Impairment decreases as exposure to stressor decreases WEAKENS Impairment increases as exposure to stressor decreases

Stressor-response relationships from the field APPLICABLE DATA: Measures of proximate stressors or other steps in causal pathways & impairments Sites varying in exposure to proximate stressor

Example: stressor-response from the field SUPPORT: Conductivity (uS/cm) WEAKEN: Nitrate + nitrite (mg/l)

Stressor-response relationships from the field Issues & recommendations: Evaluate direction & strength of relationship Look at r & r2 Need to define “strong” Be wary of statistical tests Correlation ≠ causation

Other types of evidence using data from case TYPE OF EVIDENCE SUPPORTING EVIDENCE Manipulation of exposure Impairment improves after stressor is removed Laboratory tests of site media Exposure to site media in lab tests results in effects similar to impairment Evidence of exposure or biological mechanism Measurements of biota (e.g., biomarkers, tissue residues) show proposed mechanism of exposure has occurred Verified predictions Predictions based on stressor’s mode of action are made & confirmed at site Temporal sequence Exposure to stressor precedes impairment Symptoms Only one stressor supports observed symptom

Use all available types of evidence to make an inferential assessment Types of evidence using data from the case Spatial/temporal co-occurrence Evidence of exposure or biological mechanism Causal pathway Stressor-response relationships from the field Manipulation of exposure Laboratory tests of site media Temporal sequence Verified predictions Symptoms Types of evidence using data from elsewhere Stressor-response relationships from other field studies Stressor-response relationships from laboratory studies Stressor-response relationships from ecological simulation models Mechanistically plausible cause Manipulation of exposure at other sites Verified predictions Analogous stressors italics indicates commonly available types of evidence

Stressor-response relationships from other field studies

Stressor-response relationships from the lab

Example: stressor-response from lab studies CT Values (ug/L) Daphnids (ug/L) Test EC20 MR1 Exceeded at MR1? MR3 Exceeded at MR3? Al None1 1900 540 82.2 No 107 Cd 0.62 0.15 0.75 0.4 Yes 0.5 Cr 100 <44 0.2 2 Cu 4.8 0.23 0.205 2.2 2.5 Fe 4380 - 522.8 532 Ni 88 <5 45 0.3 Pb 1.2 12.3 0.8 Zn 58.2 46.73 6.5 8.6 NH3 1430-2470 630 120

Applying stressor-response from elsewhere Use care when extrapolating from test systems → your system! OTHER FIELD STUDIES: taxa differ (EPT ≠ EPT) co-varying stressors confounding factors LAB STUDIES: different test organisms single-stressor exposures not representative of field conditions no biotic interactions criteria often protective, not effects-based

Other types of evidence using data from elsewhere TYPE OF EVIDENCE SUPPORTING EVIDENCE Mechanistically plausible cause Relationship between stressor & impairment is consistent with current scientific knowledge Stressor-response relationships from ecological simulation models Stressor is at levels associated with impairment in mathematical models simulating ecological processes Manipulation of exposure Impairment improves after stressor is removed at another site Analogous stressors Stressor is structurally similar to other stressors known to cause impairment Verified predictions Predictions based on stressor’s mode of action are made & confirmed at other sites

Looking ahead a little… In Step 5 [Identify probable cause], you score each type of evidence you have data for Some types of evidence are stronger than others, and these differences are reflected in how each type of evidence is scored

The scoring system R refutes D diagnoses +++ convincingly supports (or weakens - - -) ++ strongly supports (or weakens - -) + somewhat supports (or weakens - ) 0 neither supports nor weakens NE no evidence

Scoring of spatial/temporal co-occurrence + The effect occurs where or when the candidate cause occurs, OR the effect does not occur where or when the candidate cause does not occur. It is uncertain whether the candidate cause and the effect co-occur. - - - The effect does not occur where or when the candidate cause occurs, OR the effect occurs where or when the candidate cause does not occur. R The effect does not occur where or when the candidate cause occurs, OR the effect occurs where or when the candidate cause does not occur, AND the evidence is indisputable.

Types of evidence exercise Next: let’s try it out! Types of evidence exercise

Let’s go back to Pretend Creek… forest PC1 PC2 Pretend Springs city limit NC1 NC2 dairy farm subdivision unimpaired site impaired site WWTP industrial facility dam

Pair off & work on the types of evidence exercise