1. Biostatistics, statistical software V
Biostatistics, statistical software V. Statistical errors, one-and two sided tests. One-way and multifactor analysis of variance.
Krisztina Boda PhD
Department of Medical Informatics, University of Szeged

2. One- and two tailed (sided) tests
Two tailed test
H0: there is no change
Ha: There is change (in either direction)
One-tailed test
H0: the change is negative or zero
Ha: the change is positive
p-values: p(one-tailed)=p(two-tailed)/2
INTERREG

3. Significance
Significant difference – if we claim that there is a difference (effect), the probability of mistake is small (maximum - Type I error ).
Not significant difference – we say that there is not enough information to show difference. Perhaps
there is no difference
There is a difference but the sample size is small
The dispersion is big
The method was wrong
Even is case of a statistically significant difference one has to think about its biological meaning
INTERREG

4. Statistical errors Truth Decision
do not reject H0 reject H0 (significance)
H0 is true correct Type I. error its probability: 
Ha is true Type II. error correct its probability: 
INTERREG

5. Error probabilities The probability of type I error is known ( ).
The probability of type II error is not known ()
It depends on
The significance level (),
Sample size,
The standard deviation(s)
The true difference between populations
others (type of the test, assumptions, design, ..)
The power of a test: 1- 
ability to detect a real effect; probability to have a significant p-value
INTERREG

6. The power of a test on case of fixed sample size and , with two alternative hypotheses
INTERREG

7. ANOVA Analysis of Variance
Comparison the mean of of several (>2), normally distributed samples
Types:
One-way:
Control, treatment I, treatment II.
Two-way (treatment + sex)
Any „way” (factor) can be
„independent” („between-subjects”) sex, treatments
„repeated measures” („within-subjects”) data measured on the same patient
INTERREG

8. Why not t-test (pair wise)
Why not t-test (pair wise)? We can get significant result only by chance at every 20th case
INTERREG

9. The increase of type I error
It can be shown that when t tests are used to test for differences between multiple groups, the chance of mistakenly declaring significance (Type I Error) is increasing. For example, in the case of 5 groups, if no overall differences exist between any of the groups, using two-sample t tests pair wise, we would have about 30% chance of declaring at least one difference significant, instead of 5% chance.
In general, the t test can be used to test the hypothesis that two group means are not different. To test the hypothesis that three ore more group means are not different, analysis of variance should be used.
INTERREG

10. Each statistical test produces a ‘p’ value
If the significance level is set at 0.05 (false positive rate) and we do multiple significance testing on the data from a single clinical trial,
then the overall false positive rate for the trial will increase with each significance test.
INTERREG

11. False positive rate for each test = 0.05
Probability of incorrectly rejecting ≥ 1 hypothesis out of N testings
= 1 – (1-0.05)N=1-(1-)n
INTERREG

12. INTERREG

13. Compound hypotheses
(H01 and H02 and... H0n ) null hypotheses, the significance levels are 1, 2, …, n
How to choose i-s so that the level of the compound hypothesis (H01 and H02 and ... H0n ) would be no greater than  ? (0,1)
INTERREG

14. Bonferroni correction
The  is divided by the number of comparisons. (H01 and H02 and H0n ) is rejected, if at least one pi</n
In case of many comparisons, this is too conservative (will not show real differences).
INTERREG

15. Holm-modification (SAS: step-down Bonferroni)
The pi-s are sorted. p1p2...pn
H0i is tested at level
If any of them is significant, then reject (H01 and H02 and... H0n ) .
Pl. n=5
p1 /5= if p1 is not smaller, then finish
p2 /4= ha p2 is not smaller, then finish
p3 /3= is not smaller, then finish
p4 /2= ….
p5 /1=0.05
INTERREG

16. FDR (false discovery rate)
p1p2...pn
Begin with the greatest p-value, it remains the same
The next is tested at level
Pl. n=5
p5 
p4 /(4*5)
p3 /(3*5)
p2 /(2*5)
p1 /(1*5)=0.05
INTERREG

17. Correction of unique p-values
The SAS System
The Multtest Procedure
p-Values
False
Stepdown Discovery
Test Raw Bonferroni Hochberg Rate
INTERREG

18. One-Way ANOVA Assumptions:
Let us suppose that we have t independent samples (t “treatment” groups) drawn from normal populations with equal variances ~N(µi,).
Assumptions:
Independent samples
normality
Equal variances
Null hypothesis: population means are equal, µ1=µ2=.. =µt
INTERREG

19. Square root transformed
Cameron, E. and Pauling, L. (1978) Supplemental ascorbate in the supportive treatment of cancer: re-evaluation of prolongation of survival times in terminal human cancer. Proceedings of the National Academy of Science USA, 75, 4538Ð4542.
Original
Square root transformed
INTERREG

20. Method
If the null hypothesis is true, then the populations are the same: they are normal, and they have the same mean and the same variance. This common variance is estimated in two distinct ways:
between-groups variance
within-groups variance
If the null hypothesis is true, then these two distinct estimates of the variance should be equal
‘New’ (and equivalent) null hypothesis: 2between=2within
their equality can be tested by an F ratio test
The p-value of this test:
if p>0.05, then we accept H0. The analysis is complete.
if p<0.05, then we reject H0 at 0.05 level. There is at least one group-mean different from one of the others
INTERREG

21. INTERREG

22. The ANOVA table
INTERREG

23. Pairwise comparisons
As the two-sample t-test is inappropriate to do this, there are special tests for multiple comparisons that keep the probability of Type I error as . The most often used multiple comparisons are the modified t-tests.
Modified t-tests(LSD)
Bonferroni: α/(number of comparisons)
Scheffé
Tukey
Dunnett: a test comparing a given group (control) with the others
INTERREG

24. Example http://lib.stat.cmu.edu/DASL/Stories/ReadingComprehension.html
Researchers at Purdue University conducted an experiment to compare three methods of teaching reading.
Students were randomly assigned to one of the three teaching methods, and their reading comprehension was tested before and after they received the instruction. Several different measures of reading comprehension, from both the pre- and posttests are included in the dataset.
Reference: Moore, David S., and George P. McCabe (1989). Introduction to the Practice of Statistics. Original source: study conducted by Jim Baumann and Leah Jones of the Purdue University Education Department.
INTERREG

25. INTERREG

26. INTERREG

27. INTERREG

28. Nonparametric one-way ANOVA Kruskal-Wallis test.
As a result, it gives one p-value. If it is nit significant, the null hypothesis is accepted.
If the null hypothesis is rejected, further tests are required to make pairwise comparisons. These pairwise comparisons are generally not available in standard statistical packages. Pairwise comparisons can be performed by Mann Whitney U tests and p-values can be corrected by Bonferroni correction
INTERREG

29. Does systolic blood pressure depend on Diabetes or not Male or female
Two-way ANOVA, example
Does systolic blood pressure depend on
Diabetes or not
Male or female
Independent factors
INTERREG

30. Two-way repeated measurements ANOVA
Does QT widening in the Langendorff-perfused rat heart represent the effect of repolarization delay or conduction slowing? J Cardiovasc Pharmacol. 42 (2003)
INTERREG

31. Effect of regional ischemia and K+ content of the perfusion solution on the QT90 interval (A) and heart rate (B) in drug-free isolated rat hearts (n = 12 hearts per group). (mean ± SEM)
INTERREG

32. Frequently, separate univariate analyses are used for every time point and take no account the fact that data are related in time. A second problem is the frequent occurrence of missing values in the data. A repeated measurement ANOVA model is more appropriate (Brown and Prescott).
repeated testing is taking place and therefore a significant effect is more likely to occur at some time point by chance.
INTERREG

33. Repeated measurement ANOVA model
We can examine:
The treatment effect (K+)
Time-effect
Their interaction * * *
In high potassium concentration the heart rate is significantly higher, independently of the time it was measured
INTERREG

34. Review questions and exercises
Problems to be solved by hand-calculations
..\Handouts\Problems hand V.doc
Solutions
..\Handouts\Problems hand V solutions.doc
Problems to be solved using computer
..\Handouts\Problems comp V.doc,
..\Handouts\Problems comp V solutions.doc
INTERREG

35. Useful WEB pages http://www-stat.stanford.edu/~naras/jsm
INTERREG