Basic Statistics II

Significance/hypothesis tests

RCT comparing drug A and drug B for the treatment of hypertension 50 patients allocated to A 50 patients allocated to B Outcome = systolic BP at 3 months

Results Group A Mean = 145, sd = 9.9 Group B Mean = 135, sd = 10.0

Null hypothesis : “μ (A) = μ (B)” [ie. difference equals 0] Alternative hypothesis : “μ (A) ≠ μ (B)” [ie. difference doesn’t equal zero] [where μ = population mean]

Statistical problem When can we conclude that the observed difference mean(A) - mean(B) is large enough to suspect that μ (A) - μ (B) is not zero?

P-value : “probability of obtaining observed data if the null hypothesis were true” [eg. if no difference in systolic BP between two groups]

How do we evaluate the probability?

Test Statistic Numerical value which can be compared with a known statistical distribution Expressed in terms of the observed data and the data expected if the null hypothesis were true

Test statistic [mean (A) – mean (B)] / sd [mean(A)-mean(B)] Under null hypothesis this ratio will follow a Normal distribution with mean = 0 and sd = 1

Hypertension example Test statistic = [mean (A) – mean (B)] / sd [mean(A)-mean(B)] = [ 145 – 135 ] / 1.99 = 5 → p <0.001

Interpretation Drug B results in lower systolic blood pressure in patients with hypertension than does Drug A

Two-sample t-test Compares two independent groups of Normally distributed data

Significance test example I

Null hypothesis : “μ (A) = μ (B)” [ie. difference equals 0] Alternative hypothesis : “μ (A) ≠ μ (B)” [ie. difference doesn’t equal zero] Two-sided test

Null hypothesis : “μ (A) = μ (B) or μ (A) < μ (B) ” Alternative hypothesis : “μ (A) > μ (B)” One-sided test

A one-sided test is only appropriate if a difference in the opposite direction would have the same meaning or result in the same action as no difference

Paired-sample t-test Compares two dependent groups of Normally distributed data

Paired-sample t-test Mean daily dietary intake of 11 women measured over 10 pre-menstrual and 10 post-menstrual days

Dietary intake example Pre-menstrual (n=11): Mean=6753kJ, sd=1142 Post-menstrual (n=11): Mean=5433kJ, sd=1217 Difference Mean=1320, sd=367

Dietary intake example Test statistic = 1320/[367/sqrt(11)] = 11.9 p<0.001

Dietary intake example Dietary intake during the pre- menstrual period was significantly greater than that during the post- menstrual period

The equivalent non-parametric tests Mann-Whitney U-test Wilcoxon matched pairs signed rank sum test

Non-parametric tests Based on the ranks of the data Use complicated formula Hence computer package is recommended

Significance test example II

Type I error Significant result when null hypothesis is true (0.05) Type II error Non-significant result when null hypothesis is false [Power = 1 – Type II]

The chi-square test Used to investigate the relationship between two qualitative variables The analysis of cross-tabulations

The chi-square test Compares proportions in two independent samples

Chi-square test example In an RCT comparing infra-red stimulation (IRS) with placebo on pain caused by osteoarthritis, 9/12 in IRS group ‘improved’ compared with 4/13 in placebo group

Chi-square test example Improve? Yes No Placebo IRS

Placebo : 4/13 = 31% improve IRS: 9/12 = 75% improve

Cross-tabulations The chi-square test tests the null hypothesis of no relationship between ‘group’ and ‘improvement’ by comparing the observed frequencies with those expected if the null hypothesis were true

Cross-tabulations Expected frequency = row total x col total grand total

Chi-square test example Improve? Yes No Placebo IRS Expected value for ‘4’ = 13 x 13 / 25 = 6.8

Expected values Improve? Yes No Placebo IRS

Test Statistic =  (observed freq – expected freq) 2 expected freq

Test Statistic =  (O – E) 2 E = ( ) 2 /6.8 + (9 – 6.2) 2 /6.2 + ( ) 2 /6.8 + (9 – 6.2) 2 /6.2 = 4.9 → p=0.027

Chi-square test example Statistically significant difference in improvement between the IRS and placebo groups

Small samples The chi-square test is valid if: at least 80% of the expected frequencies exceed 5 and all the expected frequencies exceed 1

Small samples If criterion not satisfied then combine or delete rows and columns to give bigger expected values

Small samples Alternatively: Use Fisher’s Exact Test [calculates probability of observed table of frequencies - or more extreme tables-under null hypothesis]

Yates’ Correction Improves the estimation of the discrete distribution of the test statistic by the continuous chi-square distribution

Chi-square test with Yates’ correction Subtract ½ from the O-E difference  (|O – E|-½) 2 E

Significance test example III

McNemar’s test Compares proportions in two matched samples

McNemar’s test example Severe cold age 14 Yes No Severe Yes cold No age

McNemar’s test example Null hypothesis = proportions saying ‘yes’ on the 1 st and 2 nd occasions are the same  the frequencies for ‘yes,no’ and ‘no,yes’ are equal

McNemar’s test Test statistic based on observed and expected ‘discordant’ frequencies Similar to that for simple chi-square test

McNemar’s test example Test statistic = 31.4 => p <0.001 Significant difference between the two ages

Significance test example IV

Comparison of means 2 groups 2-sample t-test 3 or more groups ANOVA

One-way analysis of variance Example: Assessing the effect of treatment on the stress levels of a cohort of 60 subjects. 3 age-groups: 15-25, 26-45, Stress measured on scale 0-100

Stress levels GroupMean (SD) (n=20)52.8 (11.2) (n=20)33.4 (15.0) (n=20)35.6 (11.7)

Graph of stress levels

ANOVA Sum of squares DfMean square FSig Between groups <0.001 Within groups Total

Interpretation Significant difference between the three age-groups with respect to stress levels But what about the specific (pairwise) differences?

Stress levels GroupMean (SD) (n=20)52.8 (11.2) (n=20)33.4 (15.0) (n=20)35.6 (11.7)

Multiple comparisons Comparing each pair of means in turn gives a high probability of finding a significant result by chance A multiple comparison method (eg. Scheffé, Duncan, Newman-Keuls) makes appropriate adjustment

Scheffés test Comparison vs p< vs p< vs p=0.86

Stress levels GroupMean (SD) (n=20)52.8 (11.2) (n=20)33.4 (15.0) (n=20)35.6 (11.7)

Comparison of medians 2 groups Mann-Whitney 3 or more groups Kruskal-Wallis

Kruskal-Wallis Example: Stress levels Overall comparison of 3 groups: p<0.001

Multiple comparisons There are no non-parametric equivalents to the multiple comparison tests such as Scheffés Need to apply Bonferroni’s correction to multiple Mann-Whitney U-tests

Bonferroni’s correction For k comparisons between means: multiply each p value by k

Mann-Whitney U-test Comparison vs p< vs p< vs p=0.68 Need to multiple each p-value by 3

Significance test example V