Amsterdam Rehabilitation Research Center | Reade Testing significance - categorical data Martin van der Esch, PhD

Amsterdam Rehabilitation Research Center | Reade Relative Risk and Odds Ratios

Amsterdam Rehabilitation Research Center | Reade RD and RR only in prospective study Otherwise: Odds Ratio (OR) OR ≈ RR large differences can occur with small groups or low prevalence (‘rare diseases) Odds Ratio vs Relative risk

Amsterdam Rehabilitation Research Center | Reade Estimation 95% conficence interval Hypothesis testing Chi squared + continuity correction Fisher’s exact test Odds Ratio vs Relative Risk

Amsterdam Rehabilitation Research Center | Reade Analysis of categorical data

Amsterdam Rehabilitation Research Center | Reade 2x2 frequency table Categorical variables Recovery +Recovery -Total Interventionaba+b Controlcdc+d Totala+cb+da+b+c+d

Amsterdam Rehabilitation Research Center | Reade 2 independent groups

Amsterdam Rehabilitation Research Center | Reade For example: A new intervention for Multiple Sclerosis Patients Is the experimental intervention better than the control intervention? 2 independent groups Recovery +Recovery -Total Interventionaba+b Controlcdc+d Totala+cb+da+b+c+d

Amsterdam Rehabilitation Research Center | Reade Chance recovery intervention group: 18 / 27 = 0.67 Chance recovery control group: 6 / 23 = 0.26 Relative Risk (RR): 0.67 / 0.26 = 2.6 Risk Difference (RD): 67% - 26% = 41 % 2 independent groups Recovery +Recovery -Total Intervention18927 Controle61723 Total242650

Amsterdam Rehabilitation Research Center | Reade RD = 41% RR = 2.6 Statistically significant? 2 independent groups

Amsterdam Rehabilitation Research Center | Reade Hypothesis testing Chi-squared (p-value) Estimation 95% confindence interval RD RR 2 independent groups

Amsterdam Rehabilitation Research Center | Reade 1) Set neutral value: H 0 : no treatment effect = no difference = 0. And alternative : H 1 : difference 2) Compute expected values based on H 0 3) Compute X 2 4) Compute p-value 5) Accept or reject H 0 Hypothesis testing: Chi-squared (X 2 )

Amsterdam Rehabilitation Research Center | Reade 1) H 0 : RR = 1 or RD = 0 2) Compute expected values if H 0 is true. Recovery +Recovery -Total Intervention18927 Controls61723 Total Hypothesis testing: Chi-squared (X 2 )

Amsterdam Rehabilitation Research Center | Reade 2) By hand: - Cell A: (chance recovery+ ) * (chance intervention ) * N = (24/50) * (27/50) * 50 = 13 Or… Recovery +Recovery -Total Intervention??27 Controls??33 Total Hypothesis testing: Chi-squared (X 2 )

Amsterdam Rehabilitation Research Center | Reade 2) Hypothesis testing: Chi-squared (X 2 )

Amsterdam Rehabilitation Research Center | Reade 3) Chi-squared X 2 = Ʃ ((O – E) 2 / E ) O = observed values E = expected values And again : ‘signal / noise’ Hypothesis testing: Chi-squared (X 2 )

Amsterdam Rehabilitation Research Center | Reade OEO-E(O-E) 2 / E Intervention * recovery Intervention * recovery Controls* recovery Controls * recovery Total50 0X 2 = 8.06 X 2 = Ʃ ((O – E) 2 / E ) 3) Hypothesis testing: Chi-squared (X 2 )

Amsterdam Rehabilitation Research Center | Reade 3) X 2 = Ʃ ((O – E)2 / E ) = ) DF (2x2 table) = 1 Table for p-value: p < ) Reject H 0 in favour of H 1 Hypothesis testing: Chi-squared (X 2 )

Amsterdam Rehabilitation Research Center | Reade Hypothesis testing: Chi-squared (X 2 ) Fisher’s Exact Test and continuity (Yates) Continuity correction For small sample size: X 2 =  ((|O-E|-½) 2 /E) Fisher exact test Few samples, and cell(s) with frequency < 5 Do not use X 2, but Fisher’s exact test instead

Amsterdam Rehabilitation Research Center | Reade Hypothesis testing: Chi-squared (X 2 )

Amsterdam Rehabilitation Research Center | Reade 95% CI of RD se (p 1 -p 2 ) = 0.13;RD = % BI = 0.41 ± 1.96 x 0.13 = [ ] We can assume for 95% that the real RD is between % Estimation

Amsterdam Rehabilitation Research Center | Reade 95% CI of RR se[ln(RR)] = 0.38;RR = % CI =ln(2.6) ± 1.96 x 0.38 = [0.22 – 1.69] Back transformation: [1.24 – 5.43] We can assume with 95% confidence that the real RR is between 1.24 en 5.43 Estimation

Amsterdam Rehabilitation Research Center | Reade 2 independent groups Hypothesis testing: RD en RR: p < 0.01 Estimation: 95% CI RD: [0.16 – 0.66] 95% CI RR: [1.24 – 5.43] Conclusion: reject H 0 in favour of H 1

Amsterdam Rehabilitation Research Center | Reade 2 paired groups

Amsterdam Rehabilitation Research Center | Reade ‘Cross over trial’ For example: difference between medication A and medication B. 2 paired groups Medication B Total No improvement Improvement Medication A No improvement Improvement Total

Amsterdam Rehabilitation Research Center | Reade Hypothesis testing McNemar test Estimation 95% CI for RD 2 paired groups

Amsterdam Rehabilitation Research Center | Reade 1) Set neutral value: H 0 : no treatment effect = no difference = 0. And alternative : H 1 : difference 2) Compute X 2 3) Compute p-value 4) Reject or accept H 0 Hypothesis testing: McNemar test

Amsterdam Rehabilitation Research Center | Reade Hypothesis testing: McNemar test ‘Cross over trial’ Medication B Total No improvement Improvement Medication A No improvement Improvement Total

Amsterdam Rehabilitation Research Center | Reade 1) H 0 : RD = 0 2) X 2 = (b-c) 2 / (b+c) X 2 = (54 – 14) 2 / ( ) = Hypothesis testing: McNemar test Medication B Total No improvement Improvement Medication A No improvement Improvement Total

Amsterdam Rehabilitation Research Center | Reade 1) H 0 : RD = 0 2) X 2 = (b-c)2 / (b+c) F X 2 = (54 – 14) 2 / ( ) = ) With 1 DF  p << ) Reject H 0 in favour of H 1 Hypothesis testing: McNemar test

Amsterdam Rehabilitation Research Center | Reade Hypothesis testing: McNemar test In SPSS always with continuity correction

Amsterdam Rehabilitation Research Center | Reade 95% CI of RD RD = 0.13; se(p1-p2) = % CI = 0.13 ± 1.96 x =[0.079 – 0.18] We can assume with 95% confidence that the real RD is between 16-66% Estimation

Amsterdam Rehabilitation Research Center | Reade >2 groups

Amsterdam Rehabilitation Research Center | Reade Larger tables Comparison of 3 different diets > 2 groups Lost weight? yesnoTotal Diet Diet Diet Total

Amsterdam Rehabilitation Research Center | Reade Same approach: Difference between observed and expected values Chi-squared test with more DF In r x c table: DF = (r – 1) x (c – 1) In ordered categories: Chi-squared test for trend (e.g. The three diets have an incremental amount of daily calories) > 2 groups

Amsterdam Rehabilitation Research Center | Reade 35 Variable Measurement level? nominal ordinal numerical Observations? independent (unpaired) dependent (paired) Groups? 2 groups >2 groups Which test? parametric (ASSUMPTIONS!) non-parametric

Amsterdam Rehabilitation Research Center | Reade Statistical tests ContinuousCategorical ParametricNon-parametric 1 group, 1 observation t-testSigned rank test Testing 1 measurement 2 independent groups Independent t-test Mann-Whitney U Χ 2 –test (with continuity correction Fisher exact test 2 paired groupsPaired t-testWilcoxonMcNemar >2 groupsANOVAKruskal-WallisX 2 – test for trend if applicable