Download presentation
Presentation is loading. Please wait.
1
Power
2
The Four Components to a Statistical Conclusion
Sample size The number of units (e.g., people) accessible to study The salience of the program relative to the noise The odds the observed result is due to chance The odds you’ll observe a treatment effect when it occurs Effect size Alpha level Power
3
The Four Components to a Statistical Conclusion
Sample size Amount of information Salience of program Willingness to risk Ability to see effect that’s there Effect size Alpha level Power
4
The Effect Size Is a ratio of...
5
The Effect Size Is a ratio of... Signal Noise
6
Difference between groups Standard error of the difference
The Effect Size Is a ratio of... Signal Noise Difference between groups Standard error of the difference
7
Given Values for Any Three, You Can Compute the Fourth.
n = f(effect size, a, power) effect size = f(n, a, power) a = f(n, effect size, power) power = f(n, effect size, a)
8
The Decision Matrix In reality What we conclude
9
The Decision Matrix In reality What we conclude Null true
Alternative false In reality... There is no real program effect There is no difference, gain Our theory is wrong
10
The Decision Matrix In reality What we conclude Null true
Alternative false In reality... There is no real program effect There is no difference, gain Our theory is wrong Accept null Reject alternative We say... There is no real program effect There is no difference, gain Our theory is wrong
11
# of times out of 100 when there is no effect, we’ll say there is none
The Decision Matrix In reality What we conclude Null true Alternative false In reality... There is no real program effect There is no difference, gain Our theory is wrong Accept null 1- Reject alternative THE CONFIDENCE LEVEL We say... The odds of saying there is no effect or gain when in fact there is none There is no real program effect There is no difference, gain Our theory is wrong # of times out of 100 when there is no effect, we’ll say there is none
12
The Decision Matrix In reality What we conclude Null true
Alternative false In reality... There is no real program effect There is no difference, gain Our theory is wrong Reject null Accept alternative We say... There is a real program effect There is a difference, gain Our theory is correct
13
# of times out of 100 when there is no effect, we’ll say there is one
The Decision Matrix In reality What we conclude Null true Alternative false In reality... There is no real program effect There is no difference, gain Our theory is wrong Reject null Accept alternative TYPE I ERROR We say... The odds of saying there is an effect or gain when in fact there is none There is a real program effect There is a difference, gain Our theory is correct # of times out of 100 when there is no effect, we’ll say there is one
14
The Decision Matrix In reality What we conclude Null false
Alternative true In reality... There is a real program effect There is a difference, gain Our theory is correct
15
The Decision Matrix In reality What we conclude Null false
Alternative true In reality... There is a real program effect There is a difference, gain Our theory is correct Accept null Reject alternative We say... There is no real program effect There is no difference, gain Our theory is wrong
16
# of times out of 100 when there is an effect, we’ll say there is none
The Decision Matrix In reality What we conclude Null false Alternative true In reality... There is a real program effect There is a difference, gain Our theory is correct Accept null Reject alternative TYPE II ERROR We say... The odds of saying there is no effect or gain when in fact there is one There is no real program effect There is no difference, gain Our theory is wrong # of times out of 100 when there is an effect, we’ll say there is none
17
The Decision Matrix In reality What we conclude Null false
Alternative true In reality... There is a real program effect There is a difference, gain Our theory is correct Reject null Accept alternative We say... There is a real program effect There is a difference, gain Our theory is correct
18
# of times out of 100 when there is an effect, we’ll say there is one
The Decision Matrix In reality What we conclude Null false Alternative true In reality... There is a real program effect There is a difference, gain Our theory is correct Reject null 1- Accept alternative POWER We say... The odds of saying there is an effect or gain when in fact there is one There is a real program effect There is a difference, gain Our theory is correct # of times out of 100 when there is an effect, we’ll say there is one
19
The Decision Matrix In reality What we conclude Null true Null false
Alternative false Alternative true In reality... In reality... What we conclude There is no real program effect There is no difference, gain Our theory is wrong There is a real program effect There is a difference, gain Our theory is correct Accept null 1- Reject alternative THE CONFIDENCE LEVEL TYPE II ERROR We say... The odds of saying there is no effect or gain when in fact there is none The odds of saying there is no effect or gain when in fact there is one There is no real program effect There is no difference, gain Our theory is wrong # of times out of 100 when there is no effect, we’ll say there is none # of times out of 100 when there is an effect, we’ll say there is none Reject null 1- Accept alternative TYPE I ERROR POWER We say... The odds of saying there is an effect or gain when in fact there is none The odds of saying there is an effect or gain when in fact there is one There is a real program effect There is a difference, gain Our theory is correct # of times out of 100 when there is no effect, we’ll say there is one # of times out of 100 when there is an effect, we’ll say there is one
20
The Decision Matrix In reality What we conclude THE CONFIDENCE LEVEL
Null true Null false Alternative false Alternative true In reality... In reality... There is no real program effect There is no difference, gain Our theory is wrong There is a real program effect There is a difference, gain Our theory is correct Accept null 1- Reject alternative THE CONFIDENCE LEVEL TYPE II ERROR We say... There is no real program effect There is no difference, gain Our theory is wrong Reject null 1- Accept alternative TYPE I ERROR POWER We say... There is a real program effect There is a difference, gain Our theory is correct
21
The Decision Matrix In reality What we conclude THE CONFIDENCE LEVEL
Null true Null false Alternative false Alternative true In reality... In reality... There is no real program effect There is no difference, gain Our theory is wrong There is a real program effect There is a difference, gain Our theory is correct Accept null 1- Reject alternative THE CONFIDENCE LEVEL TYPE II ERROR We say... CORRECT There is no real program effect There is no difference, gain Our theory is wrong Reject null 1- Accept alternative TYPE I ERROR POWER We say... CORRECT There is a real program effect There is a difference, gain Our theory is correct
22
The Decision Matrix In reality What we conclude
Null true Null false Alternative false Alternative true In reality... In reality... There is no real program effect There is no difference, gain Our theory is wrong There is a real program effect There is a difference, gain Our theory is correct Accept null 1- Reject alternative THE CONFIDENCE LEVEL TYPE II ERROR We say... The odds of saying there is no effect or gain when in fact there is none The odds of saying there is no effect or gain when in fact there is one If you try to increase power, you increase the chance of winding up in the bottom row and of Type I error. There is no real program effect There is no difference, gain Our theory is wrong # of times out of 100 when there is no effect, we’ll say there is none # of times out of 100 when there is an effect, we’ll say there is none Reject null 1- Accept alternative TYPE I ERROR POWER We say... The odds of saying there is an effect or gain when in fact there is none The odds of saying there is an effect or gain when in fact there is one There is a real program effect There is a difference, gain Our theory is correct # of times out of 100 when there is no effect, we’ll say there is one # of times out of 100 when there is an effect, we’ll say there is one
23
The Decision Matrix In reality
What we conclude Null true Null false Alternative false Alternative true If you try to decrease Type I errors, you increase the chance of winding up in the top row and of Type II error. In reality... In reality... There is no real program effect There is no difference, gain Our theory is wrong There is a real program effect There is a difference, gain Our theory is correct Accept null 1- Reject alternative THE CONFIDENCE LEVEL TYPE II ERROR We say... The odds of saying there is no effect or gain when in fact there is none The odds of saying there is no effect or gain when in fact there is one There is no real program effect There is no difference, gain Our theory is wrong # of times out of 100 when there is no effect, we’ll say there is none # of times out of 100 when there is an effect, we’ll say there is none Reject null 1- Accept alternative TYPE I ERROR POWER We say... The odds of saying there is an effect or gain when in fact there is none The odds of saying there is an effect or gain when in fact there is one There is a real program effect There is a difference, gain Our theory is correct # of times out of 100 when there is no effect, we’ll say there is one # of times out of 100 when there is an effect, we’ll say there is one
Similar presentations
