1. ARA0103 Aðferðafræði Rannsókna
mean/meðaltal
standard deviation/staðalfrávik
sample size/úrtaksstærð
standard error/staðalvilla
effect size/stærð áhrifa
power/styrkur
meta-analysis/eftirgreining
replication/endurtekning
p-gildi er < 0,01
Er áhrif klínískt marktækt?
ARA0103 Aðferðafræði Rannsókna
Fyrirlestrar 16 og 17
Power, Effect Sizes, Meta-Analysis, and Replication
staðalvilla
18/09/2018
Dr Andy Brooks

2. Diastolic Blood Pressure Data
blood pressure/blóðþrýstingur
measurement error/mælingarvilla
Diastolic Blood Pressure Data
Hver er mælingarvilla? ±1, ±0,01?
18/09/2018
Dr Andy Brooks

3. Null hypothesis Núlltilgáta
Hópur 1 gamalt sjúkrahús
Hópur 2
nýtt sjúkrahús
It is suspected that staff working in an old hospital have different blood pressures to staff working in a new hospital.
Gæti verið meiri streita í gömlu sjúkrahúsi.
The null hypothesis is that, on average, there is no difference between blood pressures.
Núlltilgátan, varðandi blóðþrýsting, að meðaltali, munur er ekki til.
We start by taking a random sample/slembiúrtak of 10 from the old hospital (Hópur 1) and 10 from the new hospital (Hópur 2).
18/09/2018
Dr Andy Brooks

4. t-próf Excel Group 1 Group 2
t-Test: Two-Sample Assuming Unequal Variances 76 70
Variable 1
Variable 2 74 82
Mean
76,6 73 90
Variance
62,2667
86,4444 80 68
Observations 10 60
Hypothesized Mean Difference 62 df 18
t Stat
0,9335
P(T<=t) one-tail
0,1814 72
t Critical one-tail
1,7341
P(T<=t) two-tail
0,3629
t Critical two-tail
2,1009
Excel
18/09/2018
Dr Andy Brooks

5. Niðurstaða (n=10) Meðalmunur er 3,6 en:
fjöldi í hverju hólfi sniðsins
Meðalmunur er 3,6 en:
p-gildi er 0,36 og miklu stærri en 0,05
Ekki hægt að hafna núlltilgátunni
Segjum “núlltilgátan er rétt”
The standard deviations are large compared to the average difference of 3,6.
Group 1 staðalfrávik = 7,9
Group 2 staðalfrávik = 9,3
The standard errors of the means are only slightly less than the average difference of 3,6.
Group 1 staðalvilla = 7,9/√10 = 2,5
Group 2 staðalvilla = 9,3/√10 = 2,9
“segjum 2,7”
18/09/2018
Dr Andy Brooks

6. Graph showing standard error bars
standard error bar/staðalvillusúla
Graph showing standard error bars
skörun er mikil
Standard error bars are approximate (± 2,7).
(Standard error not standard deviation bars are shown.)
18/09/2018
Dr Andy Brooks

7. Assumption σ known. Forsenda σ þekkt.
95% öryggisbil (alfastig=0,05) fyrir μ, þýðismeðaltal
99% öryggisbil (alfastig=0,01) fyrir μ, þýðismeðaltal
18/09/2018
Dr Andy Brooks

8. Confidence Interval (CI)/Öryggisbil
x-bar – úrtaksmeðaltal
s - úrtaksstaðalfrávik
Confidence Interval (CI)/Öryggisbil
The confidence interval for the population mean: Öryggisbil fyrir μ, þýðismeðaltal er:
The critical values of t can be read from tables in statistical books or calculated using statistical software (t.d. TINV in Excel).
 - óþekkt
n - úrtaksstærð
degrees of freedom/frígráður
18/09/2018
Dr Andy Brooks

9. 95% Öryggisbil n = 10 df (frígráður) = n-1 = 9 5% in the tails
2,5% left tail, 2,5% right tail
From a table of the t-distribution, the multiplier is 2,26.
2,26 * 2,7 (staðalvilla) ≈ 6,1
18/09/2018
Dr Andy Brooks

10. BOOKTABLE6 one tail 0,05 0,025 0,01 0,005 two tail 0,1 0,02 df 3 2,35
3,18
4,54
5,84 4
2,13
2,78
3,75
4,60 5
2,02
2,57
3,36
4,03 6
1,94
2,45
3,14
3,71 7
1,89
2,36
3,00
3,50 8
1,86
2,31
2,90 9
1,83
2,26
2,82
3,25 10
1,81
2,23
2,76
3,17 11
1,80
2,20
2,72
3,11 12
1,78
2,18
2,68
3,05 13
1,77
2,16
2,65
3,01 14
1,76
2,14
2,62
2,98 15
1,75
2,60
2,95 16
2,12
2,58
2,92 17
1,74
2,11 18
1,73
2,10
2,55
2,88
Critical Values of
Student´s t-Distribution
BOOKTABLE6
18/09/2018

11. Graph showing 95% confidence intervals
skörun er mikil
In research papers, sometimes it is not clear if standard error bars or standard deviation bars or 95% confidence intervals are being shown!
18/09/2018
Dr Andy Brooks

12. Possible error in conclusion
possible error/hugsanleg villa
real difference/raunverulegur munur
Possible error in conclusion
If there is a real difference, on average, of 3,6 in diastolic blood pressures,then it is an error to accept the null hypothesis that there is no difference.
Ef 3,6 að meðaltali sé raunverulegt, svo það er mistök að segja núlltilgátan er rétt.
If there is a real difference, on average, of 3,6 in diastolic blood pressures, then our samples (n=10) were not big enough.
úrtaksstærð ekki nógu stórt
the standard errors of the means are too big
our statistical test did not have enough power to detect a difference in means as small as 3,6
18/09/2018
Dr Andy Brooks

13. Type I and II errors/Mistök af tegund I og II brief introduction/stutt kynning
A Type I error is rejecting a correct null hypothesis. Mistök af tegund I er að hafna réttri núlltilgátu.
The probability of a Type I error occurring is the alpha level (often 0,05 or 0,01). Líkurnar á að Mistök af tegund I gerast er alfastigið (oft 0,05 eða 0,01).
A Type II error is not rejecting a wrong null hypothesis. Mistök af tegund II er að hafna ekki rangri núlltilgátu.
The probability of a Type II error is β. Líkurnar á að Mistök af tegund II gerast er β.
Is the sample big enough ? Er úrtakið nógu stórt ?
The power of a statistical test is 1- β. Styrkurinn tölfræðiprófs er 1- β.
We want power to be at least 0,8. Okkar vantar styrkur sé að minnsta kosti 0,8. n
allt tengt
18/09/2018
Dr Andy Brooks

14. Warning/Viðvörun If α = 0,05 If α = 0,01 If α = 0,001
There is a 1:20 chance you have committed a Type I error.
If α = 0,01
There is a 1:100 chance you have committed a Type I error.
If α = 0,001
There is a 1:1000 chance you have committed a Type I error.
If your sample size is small:
Statistical power may be very low.
And you may easily commit a Type II error.
β can be calculated for a test knowing the size of the effect/stærð áhrifa you are looking for, sample size/úrtaksstærð, and α level/alfastig.
18/09/2018
Dr Andy Brooks

15. fjöldi í hverju hólfi sniðsins
Dæmi
afköst
fjöldi í hverju hólfi sniðsins
Ef n = 10, líkur á því að finna mun (sem er til) er ≈0,3.
Ef n = 20, líkur á því að finna mun (sem er til) er ≈0,6.
18/09/2018
Dr Andy Brooks

16. t-próf (n=50) We measure another 40 workers at each hospital...
fjöldi í hverju hólfi sniðsins
t-Test: Two-Sample Assuming Unequal Variances
Variable 1
Variable 2
Mean
76,6 73
Variance
57,1837
79,3878
Observations 50
Hypothesized Mean Difference df 95
t Stat
2,1782
P(T<=t) one-tail
0,0159
t Critical one-tail
1,6611
P(T<=t) two-tail
0,0319
t Critical two-tail
1,9853
Excel
18/09/2018
Dr Andy Brooks

17. Graph showing standard error bars (n=50)
standard error bar/staðalvillusúla
Graph showing standard error bars (n=50)
Standard error bars are approximate (± 1,1).
18/09/2018
Dr Andy Brooks

18. Graph showing 95% confidence intervals (n=50)
2,02 * 1,1 (staðalvilla) = 2,22
18/09/2018
Dr Andy Brooks

19. point estimate/punktspá
effect size/stærð áhrifa
fjöldi í hverju hólfi sniðsins
Niðurstaða (n=50)
Við höfnum núlltilgátunni. An increased sample size has given us the power to detect a difference.
Núlltilgátan er röng, hin tilgátan er rétt.
Tölfræðileg marktekt p = 0,03 (< 0,05)
The point estimate for the effect size is 3,6.
En er áhrif klínískt marktækt?
Nei ?
Standard deviations are large at both hospitals.
Maybe we should be seeking explanations/útskýringar for these large standard deviations.
Hvaða fólk er að reykja?
Hvaða folk er með yfirvinnu?
Hvaða fólk er með næturvakt?
Maybe we should test to see if the standard deviations are statistically different?
18/09/2018
Dr Andy Brooks

20. Effect Sizes two sample case
effect size/stærð áhrifa
Effect Sizes two sample case
The size of the effect is usually normalised with respect to the standard deviation.
The effect size, assuming a common variance, is given by:
Cohen proposed:
0,2 is small effect size
0,5 is a medium effect size
0,8 is a large effect size
18/09/2018
Dr Andy Brooks

21. Estimate of effect size diastolic blood pressure experiment
estimate/spágildi
Estimate of effect size diastolic blood pressure experiment
Point estimate/punktspá of difference between means = 3,6
Estimate of variance = 70
For simplification, we assume a single common variance in the diastolic blood pressure experiment.
Estimate of standard deviation = 8,3666 sqrt(70)
Estimate of effect size = 0,4303
a small to medium effect
18/09/2018
Dr Andy Brooks

22. Java applets for power and sample size by Russ Lenth
Java applets for power and sample size by Russ Lenth
power = 0,04
18/09/2018
Dr Andy Brooks

23. Java applets for power and sample size by Russ Lenth
18/09/2018
Dr Andy Brooks

24. Java applets for power and sample size by Russ Lenth
18/09/2018
Dr Andy Brooks

25. Java applets for power and sample size by Russ Lenth
18/09/2018
Dr Andy Brooks

26. unacceptable/óaðgengilegur
α and β
As the α level gets more strict (0,05 -> 0,01), then you have less power β.
There is less chance of a Type I error.
But more chance of a Type II error (1-β).
As the α level gets less strict (0,01 -> 0,05), then you have more power β.
There is more chance of a Type I error.
But less chance of a Type II error (1-β).
Some researchers use an α level of 0,10, but this means a 1:10 chance of making a Type I error.
Many researchers find an alpha level of 0,10 to be unacceptable.
18/09/2018
Dr Andy Brooks

27. More power
A power of 0,5 means there is a 50% chance your experiment will fail to detect a difference that is real.
If an experiment costs $10 million to run, you want a power of 0,99 and not 0,5.
There may be no way of estimating power until you have performed the experiment.
Previous results by other researchers can sometimes be used to estimate the effect size.
18/09/2018
Dr Andy Brooks

28. Power calculations/Styrksútreikningar
Power calculations get more complicated with more complicated experimental designs.
Power calculations get more complicated when group sample sizes and/or group variances are unequal.
Professional software exists to support calculations of power for many types of statistical tests.
Power calculations are impossible unless you have an estimate of the effect size.
In research papers, a power analysis is often not reported because a power analysis was never done. It is becoming more common to insist a power analysis is done before a research paper is accepted for publication.
18/09/2018
Dr Andy Brooks

29. Missing effect size ?
In the absence of previous results, group sample sizes should be at least 10.
Have at least 20 participants if you plan to randomize patients into two groups of 10 and use an independent two-sample t-test.
Try if possible to have large numbers in each group (20, 30, 40, or 50...).
The more the better.
18/09/2018
Dr Andy Brooks

30. Java applets for power and sample size by Russ Lenth
18/09/2018
Dr Andy Brooks

31. Java applets for power and sample size by Russ Lenth
18/09/2018
Dr Andy Brooks

32. descriptive statistics/lýsandi tölfræði
outlier/einfari
Failure to reject the null hypothesis Ekki hægt að hafna núlltilgátunni?
If you cannot reject the null hypothesis, use descriptive statistics (average, standard deviation, standard error, minimum, maximum), histograms, boxplots and line graphs to present, compare, and interpret the data.
What happens if you use an α of 0,10 ?
This may allow you to interpret the experimental results statistically, but you need to emphasis the need to repeat the experiment with bigger samples.
Try and estimate the power of the experiment retrospectively.
This can help future researchers.
Find explanations of any outliers.
Sometimes this is where the real results of an experiment are.
18/09/2018
Dr Andy Brooks

33. meta-analysis/eftirgreining
Tilraun 1
Tilraun 2
Meta-analysis
Tilraun 3
Tilraun 4
Tilraun 5
Meta-analysis involves examining the results of experiments with the same null hypothesis.
A meta-analysis can simply involve counting the number of research papers that conclude the effect was present against the number of papers that conclude there was no effect.
Counts are based on the best quality experiments (t.d. Randomized Control Trial/Hrein Tilraun).
Simple counting of research papers is viewed by many researchers as insufficient.
The data has to be combined statistically.
18/09/2018
Dr Andy Brooks

34. Meta-analysis
Another form of meta-analysis involves pooling together raw data from several experiments.
að samlaga óunnin gögn úr nokkrir tilraunum
This pooling together data effectively increases group sample sizes and so increases the power of any statistical tests applied.
If we have data for 5 experiments where group sample sizes were 10, in the meta-analysis, group sample sizes become 50.
fjöldi í hverju hólfi sniðsins
18/09/2018
Dr Andy Brooks

35. Special software/sérstakur hugbúnaður
Meta-analysis
Another form of meta-analysis involves pooling together effect size estimates from several experiments.
Að samlaga áhrifastærðir
Special software exists to support meta-analytic procedures.
t.d. RevMan from the Cochrane Collaboration.
Dæmisaga, Fyrirlestur 10
18/09/2018
Dr Andy Brooks

36. Fig 3 Relative risk for mortality
Dæmisaga, Fyrirlestur 10
Fig 3 Relative risk for mortality
forest plot
(c) BMJ
18/09/2018
Dr Andy Brooks

37. Replication/Endurtekning
other explanations/aðrar útskýringar
dispute/rökræða
Replication/Endurtekning
The results from a RCT may be wrong.
A cause-and-effect relationship does not exist.
0rsakatengls er ekki til
Gæti verið að aðrar útskýringar eru til
People only start believing the result when the RCT is successfuly replicated by other research teams.
The results from several RCTs can be combined in a meta-analysis.
Even the results of a meta-analysis can be disputed...
18/09/2018
Dr Andy Brooks

38. Replication/Endurtekning
validity/réttmæti
improve/bæta við
Replication/Endurtekning
Often, when you decide to replicate an experiment, you also improve the experiment:
Measure O1 to check the groups are equal.
Breyta spurningalista
Bæta við spurningum, taka burt spurningar, breyta orðalag,...
Use a different questionnaire, one which has been validated.
But if you make too many improvements, it is a different experiment.
Svo er ekki hægt að samlaga óunnin gögn, osf.
18/09/2018
Dr Andy Brooks