1. Amsterdam Rehabilitation Research Center | Reade Testing significance - categorical data Martin van der Esch, PhD

2. Amsterdam Rehabilitation Research Center | Reade Relative Risk and Odds Ratios

3. Amsterdam Rehabilitation Research Center | Reade RD and RR only in prospective study Otherwise: Odds Ratio (OR) OR ≈ RR large differences can occur with small groups or low prevalence (‘rare diseases) Odds Ratio vs Relative risk

4. Amsterdam Rehabilitation Research Center | Reade Estimation 95% conficence interval Hypothesis testing Chi squared + continuity correction Fisher’s exact test Odds Ratio vs Relative Risk

5. Amsterdam Rehabilitation Research Center | Reade Analysis of categorical data

6. Amsterdam Rehabilitation Research Center | Reade 2x2 frequency table Categorical variables Recovery +Recovery -Total Interventionaba+b Controlcdc+d Totala+cb+da+b+c+d

7. Amsterdam Rehabilitation Research Center | Reade 2 independent groups

8. Amsterdam Rehabilitation Research Center | Reade For example: A new intervention for Multiple Sclerosis Patients Is the experimental intervention better than the control intervention? 2 independent groups Recovery +Recovery -Total Interventionaba+b Controlcdc+d Totala+cb+da+b+c+d

9. Amsterdam Rehabilitation Research Center | Reade Chance recovery intervention group: 18 / 27 = 0.67 Chance recovery control group: 6 / 23 = 0.26 Relative Risk (RR): 0.67 / 0.26 = 2.6 Risk Difference (RD): 67% - 26% = 41 % 2 independent groups Recovery +Recovery -Total Intervention18927 Controle61723 Total242650

10. Amsterdam Rehabilitation Research Center | Reade RD = 41% RR = 2.6 Statistically significant? 2 independent groups

11. Amsterdam Rehabilitation Research Center | Reade Hypothesis testing Chi-squared (p-value) Estimation 95% confindence interval RD RR 2 independent groups

12. Amsterdam Rehabilitation Research Center | Reade 1) Set neutral value: H 0 : no treatment effect = no difference = 0. And alternative : H 1 : difference 2) Compute expected values based on H 0 3) Compute X 2 4) Compute p-value 5) Accept or reject H 0 Hypothesis testing: Chi-squared (X 2 )

13. Amsterdam Rehabilitation Research Center | Reade 1) H 0 : RR = 1 or RD = 0 2) Compute expected values if H 0 is true. Recovery +Recovery -Total Intervention18927 Controls61723 Total242650 Hypothesis testing: Chi-squared (X 2 )

14. Amsterdam Rehabilitation Research Center | Reade 2) By hand: - Cell A: (chance recovery+ ) * (chance intervention ) * N = (24/50) * (27/50) * 50 = 13 Or… Recovery +Recovery -Total Intervention??27 Controls??33 Total342660 Hypothesis testing: Chi-squared (X 2 )

15. Amsterdam Rehabilitation Research Center | Reade 2) Hypothesis testing: Chi-squared (X 2 )

16. Amsterdam Rehabilitation Research Center | Reade 3) Chi-squared X 2 = Ʃ ((O – E) 2 / E ) O = observed values E = expected values And again : ‘signal / noise’ Hypothesis testing: Chi-squared (X 2 )

17. Amsterdam Rehabilitation Research Center | Reade OEO-E(O-E) 2 / E Intervention * recovery+181351.92 Intervention * recovery-914-51.79 Controls* recovery +611-52.27 Controls * recovery -171252.08 Total50 0X 2 = 8.06 X 2 = Ʃ ((O – E) 2 / E ) 3) Hypothesis testing: Chi-squared (X 2 )

18. Amsterdam Rehabilitation Research Center | Reade 3) X 2 = Ʃ ((O – E)2 / E ) = 8.06 4) DF (2x2 table) = 1 Table for p-value: p < 0.005 5) Reject H 0 in favour of H 1 Hypothesis testing: Chi-squared (X 2 )

19. Amsterdam Rehabilitation Research Center | Reade Hypothesis testing: Chi-squared (X 2 ) Fisher’s Exact Test and continuity (Yates) Continuity correction For small sample size: X 2 =  ((|O-E|-½) 2 /E) Fisher exact test Few samples, and cell(s) with frequency < 5 Do not use X 2, but Fisher’s exact test instead

20. Amsterdam Rehabilitation Research Center | Reade Hypothesis testing: Chi-squared (X 2 )

21. Amsterdam Rehabilitation Research Center | Reade 95% CI of RD se (p 1 -p 2 ) = 0.13;RD = 0.41 95% BI = 0.41 ± 1.96 x 0.13 = [0.16 - 0.66] We can assume for 95% that the real RD is between 16- 66% Estimation

22. Amsterdam Rehabilitation Research Center | Reade 95% CI of RR se[ln(RR)] = 0.38;RR = 2.6 95% CI =ln(2.6) ± 1.96 x 0.38 = [0.22 – 1.69] Back transformation: [1.24 – 5.43] We can assume with 95% confidence that the real RR is between 1.24 en 5.43 Estimation

23. Amsterdam Rehabilitation Research Center | Reade 2 independent groups Hypothesis testing: RD en RR: p < 0.01 Estimation: 95% CI RD: [0.16 – 0.66] 95% CI RR: [1.24 – 5.43] Conclusion: reject H 0 in favour of H 1

24. Amsterdam Rehabilitation Research Center | Reade 2 paired groups

25. Amsterdam Rehabilitation Research Center | Reade ‘Cross over trial’ For example: difference between medication A and medication B. 2 paired groups Medication B Total No improvement Improvement Medication A No improvement 18454238 Improvement146377 Total198117315

26. Amsterdam Rehabilitation Research Center | Reade Hypothesis testing McNemar test Estimation 95% CI for RD 2 paired groups

27. Amsterdam Rehabilitation Research Center | Reade 1) Set neutral value: H 0 : no treatment effect = no difference = 0. And alternative : H 1 : difference 2) Compute X 2 3) Compute p-value 4) Reject or accept H 0 Hypothesis testing: McNemar test

28. Amsterdam Rehabilitation Research Center | Reade Hypothesis testing: McNemar test ‘Cross over trial’ Medication B Total No improvement Improvement Medication A No improvement 18454238 Improvement146377 Total198117315

29. Amsterdam Rehabilitation Research Center | Reade 1) H 0 : RD = 0 2) X 2 = (b-c) 2 / (b+c) X 2 = (54 – 14) 2 / (54 + 14) = 23.53 Hypothesis testing: McNemar test Medication B Total No improvement Improvement Medication A No improvement 18454238 Improvement146377 Total198117315

30. Amsterdam Rehabilitation Research Center | Reade 1) H 0 : RD = 0 2) X 2 = (b-c)2 / (b+c) F X 2 = (54 – 14) 2 / (54 + 14) = 23.53 3) With 1 DF  p << 0.001 4) Reject H 0 in favour of H 1 Hypothesis testing: McNemar test

31. Amsterdam Rehabilitation Research Center | Reade Hypothesis testing: McNemar test In SPSS always with continuity correction

32. Amsterdam Rehabilitation Research Center | Reade 95% CI of RD RD = 0.13; se(p1-p2) = 0.026 95% CI = 0.13 ± 1.96 x 0.026 =[0.079 – 0.18] We can assume with 95% confidence that the real RD is between 16-66% Estimation

33. Amsterdam Rehabilitation Research Center | Reade >2 groups

34. Amsterdam Rehabilitation Research Center | Reade Larger tables Comparison of 3 different diets > 2 groups Lost weight? yesnoTotal Diet 1152843 Diet 2524698 Diet 36774141 Total134148282

35. Amsterdam Rehabilitation Research Center | Reade Same approach: Difference between observed and expected values Chi-squared test with more DF In r x c table: DF = (r – 1) x (c – 1) In ordered categories: Chi-squared test for trend (e.g. The three diets have an incremental amount of daily calories) > 2 groups

36. Amsterdam Rehabilitation Research Center | Reade 35 Variable Measurement level? nominal ordinal numerical Observations? independent (unpaired) dependent (paired) Groups? 2 groups >2 groups Which test? parametric (ASSUMPTIONS!) non-parametric

37. Amsterdam Rehabilitation Research Center | Reade Statistical tests ContinuousCategorical ParametricNon-parametric 1 group, 1 observation t-testSigned rank test Testing 1 measurement 2 independent groups Independent t-test Mann-Whitney U Χ 2 –test (with continuity correction Fisher exact test 2 paired groupsPaired t-testWilcoxonMcNemar >2 groupsANOVAKruskal-WallisX 2 – test for trend if applicable