Download presentation
Presentation is loading. Please wait.
1
Steps 3 & 4: Evaluating types of evidence
2
Evaluate Data from the Case
Define the Case List Candidate Causes Evaluate Data from Elsewhere Identify Probable Cause Detect or Suspect Biological Impairment As Necessary: Acquire Data and Iterate Process Identify and Apportion Sources Management Action: Eliminate or Control Sources, Monitor Results Biological Condition Restored or Protected Decision-maker Stakeholder Involvement Stressor Identification Step 3: Evaluate Data from the Case Step 4:
3
Use all available types of evidence to make an inferential assessment
Types of evidence using data from the case Spatial/temporal co-occurrence Evidence of exposure or biological mechanism Causal pathway Stressor-response relationships from the field Manipulation of exposure Laboratory tests of site media Temporal sequence Verified predictions Symptoms Types of evidence using data from elsewhere Stressor-response relationships from other field studies Stressor-response relationships from laboratory studies Stressor-response relationships from ecological simulation models Mechanistically plausible cause Manipulation of exposure at other sites Verified predictions Analogous stressors italics indicates commonly available types of evidence
4
Basic analysis strategy
Develop as many types of evidence, for as many candidate causes, as you can you won’t have all types of evidence, for all candidate causes most effective when you can compare results across candidate causes Show your work make your process transparent & reproducible make use of appendices
5
Definition Types of evidence are categories of relationships that provide logically distinct ways to support, weaken, or refute the case for a candidate cause.
6
Why so many types of evidence?
Helps make sure that potential sources of evidence are not overlooked The label associated with a particular piece of evidence is not important, provided you: Are consistent across candidate causes Don’t double-count data
7
Use all available types of evidence to make an inferential assessment
Types of evidence using data from the case Spatial/temporal co-occurrence Evidence of exposure or biological mechanism Causal pathway Stressor-response relationships from the field Manipulation of exposure Laboratory tests of site media Temporal sequence Verified predictions Symptoms Types of evidence using data from elsewhere Stressor-response relationships from other field studies Stressor-response relationships from laboratory studies Stressor-response relationships from ecological simulation models Mechanistically plausible cause Manipulation of exposure at other sites Verified predictions Analogous stressors italics indicates commonly available types of evidence
11
Spatial/temporal co-occurrence
SUPPORTS Impairment occurs where stressor is present WEAKENS Impairment does not occur where stressor is present, or occurs where stressor is not present
12
Spatial/temporal co-occurrence
SUPPORTS WEAKENS APPLICABLE DATA: Measures of proximate stressors & impairments Impaired & reference sites
13
Example: spatial co-occurrence
Candidate Cause Measurement [total metals & ammonia (mg/L)] Upstream reference Watershed reference Impaired site Adverse change compared to references TOXICS Al 0.080 0.037 0.101 Yes Cd ND No Cr 0.005 Cu 0.004 Fe 0.395 0.208 0.695 Ni 0.001 Pb Zn 0.006 0.011 NH3 0.1
14
Spatial/temporal co-occurrence
Issues & recommendations: Simple comparison – is exposure to proximate stressor greater where/when impairment occurs? Don’t consider whether magnitude of stressor is sufficient Magnitude considered under other types of evidence (e.g., stressor-response relationships from elsewhere) Consider uncertainty & variability in data set, but don’t rely on statistical tests Only use measurements of proximate stressor Other measurements are considered under “Causal pathway”
15
Causal pathway SUPPORTS WEAKENS
Steps in causal pathway observed, and coincide with impairment WEAKENS Steps in causal pathway not observed, and do not coincide with impairment
16
Causal pathway APPLICABLE DATA:
Measures of other steps in causal pathways & impairments Impaired & reference sites
17
Example: causal pathway Pathway supported compared to references
Data Reference sites Impaired site Pathway supported compared to references Habitat Index [% of reference] 68 100 32 Yes BOD [mg/L] 1.1 1.6 Tree canopy moderate high low Bank stability 10 8 No Velocity 15 16 11 % Impervious surface Illicit discharge absent present Substrate composition % Boulder 33 25 % Cobble 50 % Gravel 13 % Sand 12 Candidate causes: ↓ dissolved oxygen; Δ food resources
18
Causal pathway Issues & recommendations:
Similar to spatial co-occurrence, but uses data from entire causal chain When in doubt, assume a step exists Evidence of missing step is powerful; evidence of many intermediate steps increases confidence May be able to eliminate one pathway, but rarely can eliminate all pathways ↓ dissolved oxygen ↑ nitrogen ↓ fish ↑ algae
19
Stressor-response relationships from the field
SUPPORTS Impairment decreases as exposure to stressor decreases WEAKENS Impairment increases as exposure to stressor decreases
20
Stressor-response relationships from the field
APPLICABLE DATA: Measures of proximate stressors or other steps in causal pathways & impairments Sites varying in exposure to proximate stressor
21
Example: stressor-response from the field
SUPPORT: Conductivity (uS/cm) WEAKEN: Nitrate + nitrite (mg/l)
22
Stressor-response relationships from the field
Issues & recommendations: Evaluate direction & strength of relationship Look at r & r2 Need to define “strong” Be wary of statistical tests Correlation ≠ causation
23
Other types of evidence using data from case
TYPE OF EVIDENCE SUPPORTING EVIDENCE Manipulation of exposure Impairment improves after stressor is removed Laboratory tests of site media Exposure to site media in lab tests results in effects similar to impairment Evidence of exposure or biological mechanism Measurements of biota (e.g., biomarkers, tissue residues) show proposed mechanism of exposure has occurred Verified predictions Predictions based on stressor’s mode of action are made & confirmed at site Temporal sequence Exposure to stressor precedes impairment Symptoms Only one stressor supports observed symptom
24
Use all available types of evidence to make an inferential assessment
Types of evidence using data from the case Spatial/temporal co-occurrence Evidence of exposure or biological mechanism Causal pathway Stressor-response relationships from the field Manipulation of exposure Laboratory tests of site media Temporal sequence Verified predictions Symptoms Types of evidence using data from elsewhere Stressor-response relationships from other field studies Stressor-response relationships from laboratory studies Stressor-response relationships from ecological simulation models Mechanistically plausible cause Manipulation of exposure at other sites Verified predictions Analogous stressors italics indicates commonly available types of evidence
27
Stressor-response relationships from other field studies
29
Stressor-response relationships from the lab
30
Example: stressor-response from lab studies
CT Values (ug/L) Daphnids (ug/L) Test EC20 MR1 Exceeded at MR1? MR3 Exceeded at MR3? Al None1 1900 540 82.2 No 107 Cd 0.62 0.15 0.75 0.4 Yes 0.5 Cr 100 <44 0.2 2 Cu 4.8 0.23 0.205 2.2 2.5 Fe 4380 - 522.8 532 Ni 88 <5 45 0.3 Pb 1.2 12.3 0.8 Zn 58.2 46.73 6.5 8.6 NH3 630 120
31
Applying stressor-response from elsewhere
Use care when extrapolating from test systems → your system! OTHER FIELD STUDIES: taxa differ (EPT ≠ EPT) co-varying stressors confounding factors LAB STUDIES: different test organisms single-stressor exposures not representative of field conditions no biotic interactions criteria often protective, not effects-based
32
Other types of evidence using data from elsewhere
TYPE OF EVIDENCE SUPPORTING EVIDENCE Mechanistically plausible cause Relationship between stressor & impairment is consistent with current scientific knowledge Stressor-response relationships from ecological simulation models Stressor is at levels associated with impairment in mathematical models simulating ecological processes Manipulation of exposure Impairment improves after stressor is removed at another site Analogous stressors Stressor is structurally similar to other stressors known to cause impairment Verified predictions Predictions based on stressor’s mode of action are made & confirmed at other sites
33
Looking ahead a little…
In Step 5 [Identify probable cause], you score each type of evidence you have data for Some types of evidence are stronger than others, and these differences are reflected in how each type of evidence is scored
34
The scoring system R refutes D diagnoses
+++ convincingly supports (or weakens - - -) ++ strongly supports (or weakens - -) + somewhat supports (or weakens - ) 0 neither supports nor weakens NE no evidence
35
Scoring of spatial/temporal co-occurrence
+ The effect occurs where or when the candidate cause occurs, OR the effect does not occur where or when the candidate cause does not occur. It is uncertain whether the candidate cause and the effect co-occur. - - - The effect does not occur where or when the candidate cause occurs, OR the effect occurs where or when the candidate cause does not occur. R The effect does not occur where or when the candidate cause occurs, OR the effect occurs where or when the candidate cause does not occur, AND the evidence is indisputable.
36
Types of evidence exercise
Next: let’s try it out! Types of evidence exercise
37
Let’s go back to Pretend Creek…
forest PC1 PC2 Pretend Springs city limit NC1 NC2 dairy farm subdivision unimpaired site impaired site WWTP industrial facility dam
38
Pair off & work on the types of evidence exercise
Similar presentations
![CADDIS Workshop [Causal Analysis/Diagnosis Decision Information System] New Jersey Water Restoration, Protection, and Planning Working Group Rutgers University,](/16/4919925/big_thumb.jpg "CADDIS Workshop [Causal Analysis/Diagnosis Decision Information System] New Jersey Water Restoration, Protection, and Planning Working Group Rutgers University,>")
