1. Effect Sizes…

2. What do P-values tell us?
Strength of evidence against a null hypothesis
Probability of getting the observed results (or something more extreme) if H0 (and associated assumptions) is correct (given observed variation).

3. P-values and Effect Sizes?
NS Sig NS Sig Sig

4. If not P, then what?
Examples…

5. 1. Hedge's d: 2. Difference: XE - XC 3. Ratio: XE/XC 4
1. Hedge's d: Difference: XE - XC 3. Ratio: XE/XC 4. Log response ratio: ln(XE/XC)

6. Need to motivate a more thoughtful approach…
So, are there metrics that don’t rely on signal/noise?

7. Examine variation in biological effects
What if you want to compare your "effect" with others (e.g., meta-analysis)?
Eliminate “effects” of the investigator (e.g., due to variation in duration, initial size, other variables…)
Examine variation in biological effects

8. Requires an effect size
that is linked with
"theory"
(or at least biology)

9. An illustration
Let's consider experiments that manipulate CO2 and look at the response of plants…

10. Study: A B C D E Ambient CO2: final size (g) 2 8 12 200 Elevated CO2:3 4 10 20
220
Initial size (g) 1 11
180
Duration (d) 5
Tally up: 1) which study gives the largest effect?
2) Which study gives the smallest effect?

11. Study: A B C D E
Ambient CO2:
final size (g) 2 8 12
200
Elevated CO2: 3 4 10 20
220
Initial size (g) 1 11
180
Duration (d) 5

12. Study: A B C D E
Ambient CO2:
final size (g) 2 8 12
200
Elevated CO2: 3 4 10 20
220
Initial size (g) 1 11
180
Duration (d) 5

13. Study: A B C D E
Ambient CO2:
final size (g) 2 8 12
200
Elevated CO2: 3 4 10 20
220
Initial size (g) 1 11
180
Duration (d) 5

14. Effects: plant response to CO2
How did you come up with your answers?
Did you have a “model” in mind?
How do plants grow?
How would CO2 effect that growth rate?

15. Here's one option: dM/dt = "constant" (dep. on env., but not M)
Mt,c=Mo + gct
Mt,x=Mo + gxt
Effect: e = gx-gc = (Mt,x-Mt,c)/t
Note: 1) this is a simple difference in "g" (thus, omnibus-like);
But 2) g is derived from the data; it is NOT the measured response variable (that was M).

16. Study: A B C D E
Ambient CO2:
final size (g) 2 8 12
200
Elevated CO2: 3 4 10 20
220
Initial size (g) 1 11
180
Duration (d) 5
Effect (linear):

17. Another option: Exponential growth: dM/dt = gM Or: Mt=Moegt
Thus: effect = Δg = ln(Mt,x/Mt,c)/t

18. Study: A B C D E
Ambient CO2:
final size (g) 2 8 12
200
Elevated CO2: 3 4 10 20
220
Initial size (g) 1 11
180
Duration (d) 5
Effect (linear):
Effect (expon):

19. Defining an "effect" forces you to think about the process you are studying.

21. Density dependence
Compare systems “with” vs. “without” density dependence, but…
Based on P-values; null hypothesis tests
Is this controversy real?
What drives apparent variation?
Osenberg et al (Ecology Letters)

22. 4 competing hypotheses to explain the variation in "effects"

23. “effect” : requires model
Yet, few meta-analyses discuss the dynamics of their system and propose a model (or alternate models) for the underlying process. They simply pick an “effect”, such as cohen’s d (essentially a t statistic), or hedges g, or a difference, or a log-ratio, without further exploration or justification.
This is a biological issue, not a statistical one.

24. Beverton-Holt Recruitment Function:
Density (N)
Inst. Mort. Rate
slope = β α
β : per capita effect of conspecifics
α : density-independent mortality rate

25. Integrated form of model:
α, β
Input (Settlers)
(Sub-Adults)
Output
N0 : initial density (e.g., settlers)
Nt : “final” density (e.g., recruits or adults)

26. Overall effect of density
= m2 fish-1 day-1
(CI: to )
[BUT extremely heterogeneous…
…controversy?]

27. Density-dependence: the debate (“no” vs. “yes”)Nt N0 Nt N0 Nt N0 Nt N0

28. β small Nt N0
β large
Den-indep.
Den-dep.

29. Density-dependence: the debate (“no” vs. “yes”)X X Nt N0 Nt N0 Nt N0 Nt N0

30. Den-indep.
Den-dep.

31. Den-indep.
Den-dep.

32. Density-dependence: the debate (“no” vs. “yes”)X X Nt N0 Nt N0 X Nt N0 Nt N0

33. Density-dependence: the debate (“no” vs. “yes”)
β similar (on average)
Ambient densities differ
Heterogeneity is large
…and unresolved Nt N0

34. Sources of variation? Predators Age-classes Taxonomic groups
Geographic regions
Study design

35. Australia
Caribbean
California
Indo-Pacific

36. Labrid
Gobiid
Acanthurid
Pomacentrid

37. Bottom-line… There are an infinite number of questions…
… so there should be an infinite number of effect size definitions.

38. The challenge is to determine how to best quantify "your question".
Take home message:
The challenge is to determine how to best quantify "your question".
[Statistical issues are important, but they are secondary to biological ones: get the question right, then adjust the tool.]

39. "but I don’t have a 'model'…"
Think a bit more about the process
Make up data and challenge yourself to "rank them" (why?)
Absolute vs. relative difference (XE-XC or ln(XE/XC))

40. End