1. Kylie Lange, CRE Biostatistician
Power calculations
adelaide.edu.au

2. Q12. Animal Number Justification
University of Adelaide

3. (a) What is the key outcome measure of interest?
1 outcome per experiment
1 power calculation per outcome
If one experiment contains multiple outcomes (eg histological, biochemical & behavioural outcomes), then report the power calculation for the outcome that requires the largest number of animals
Specify the exact form of the outcome, particularly for serial measurements
Eg: change over time (absolute or percentage?), difference between groups (at which time point?), a derived measure, ratios/indices
University of Adelaide

4. (b) What is the effect size of minimum biological significance?
What size effect in the key outcome has actual biological relevance?
This is a biological question, not statistical
Eg - weight loss in an obesity model – how much greater weight loss over control is needed before you can expect reduced cardiovascular risk, reduced mortality, improved quality of life, improved fertility outcomes?
Provide a justification of this value
Where does it come from (previous studies, pilot data)?
What hard outcome does it relate to?
How big is it relative to the effect observed for alternative interventions?
University of Adelaide

5. (c) What is the estimated standard deviation of this outcome measure?
Provide the source of the estimate
Previously published studies, pilot study, lab data
Provide a justification of why this estimate is relevant
Has the estimate been taken from same/similar/different:
Population
Intervention
Measurement methods
(Note that if the outcome is a proportion then SD is not applicable)
University of Adelaide

6. (d) What power level (1-beta) and statistical significance (alpha) will be employed?
Alpha = Probability of a type I error
Type I error = Find evidence of an effect when it truly does not exist (“false positive”)
Beta = Probability of type II error
Type II error = Fail to find evidence of an effect when it truly does exist (“false negative”)
Usual conventions are alpha = & 1-beta = but these are arbitrary and can be (should be!) varied depending on the context
Provide justification if using anything different
Risk/benefit balance may vary depending on the invasiveness of the intervention, side effects, cost, expected benefits. Implications of each type of error.
University of Adelaide

7. (e) What statistical test will be employed for data analysis
(e) What statistical test will be employed for data analysis? And, will data be analysed using a one- or two-sided test?
Two-sided test: Evidence of an effect will be concluded if a difference exists in either direction (eg, A>B or A<B)
One-sided test: Evidence of an effect will only be concluded if a difference exists in one pre-specified direction (eg, A>B)
Choice of test depends on
Form of the research hypothesis
Study design
Measurement types of the variables
Number of groups/treatments/time points
Any additional confounders to be controlled for
Sequential stopping rule (SSR): add a predetermined number of subjects at each stage and test repeatedly for significance until the experiment is stopped because (1) a sig effect is detected, (2) the effect is clearly not going to be significant, or (3) the predetermined maximal number of subjects has been reached. Holds the prob of type I error constant, maintains power. Requires a defined p-value criteria for significance and for futility.
University of Adelaide

8. Summary
Provide enough information that the power calculation can be replicated
Justify the choices made
This is the best time to engage with a statistician!
University of Adelaide

9. Clinical significance v statistical significance
Not significant
Significant
Not
important
Happy
Annoyed
Practical importance of observed effect
Want to ensure that statistical significance agrees with clinical significance
Power = probability that your study will find a clinically interesting finding statistically significant, if it exists
Statistical significance
The relationship is detectable over the background levels of variation/noise
Clinical significance
Does the observed effect correspond to a clinically relevant or interesting finding?
Important
Very sad
Elated
©Helena Oakey, 2009
University of Adelaide

10. Power analysis Requires you to know five things:
Statistical test to be used
Desired significance level
Desired power
Expected level of variation in the outcome
What size effect is biologically important
Once have defined/estimated these then formulae/software can calculate power and sample size
Power analysis will maximise the chance that your study will fall in the ‘happy’ or ‘elated’ scenarios, and not the ‘annoyed’ or ‘very sad’ scenarios
University of Adelaide