1. 1 Bandit Thinkhamrop, PhD.(Statistics) Dept. of Biostatistics & Demography Faculty of Public Health Khon Kaen University Overview and Common Pitfalls in Statistics and How to Avoid Them

2. 2 Roles of Statistics in Research Begin at a clear destination 7 What is the conclusion? Concluded based on what? Could it be wrong? Can it be wrong due to data analysis? Can the data be wrong?

3. 3 Statistics Quantify the effect and its error 7 Magnitude of effect Parameter estimation [95%CI] Hypothesis testing [P-value] Magnitude of effect Parameter estimation [95%CI] Hypothesis testing [P-value] Quantify errors for further judgments

4. 4 1. Over reliance on p-value Example: –Significant findings p-value <0.05 –Non-significant findings p-value > 0.05

5. 5

6. 6

7. 7

8. 8

9. 9

10. 10

11. 11 Diff เดิน = 1.3, 95%CI: 0.04 to 2.62 Diff เดิน + นับเลข = 3.0, 95%CI: 0.17 to 5.87 Diff เดิน + นับเดือน = 3.9, 95%CI: 1.16 to 6.62

12. 12

13. 13

14. 14 1. Over reliance on p-value (cont.) Example: significant findings p-value <0.05 Tips to avoid it: –Always report the magnitude of effect and its 95%CI –Always interpret the findings based on the magnitude of effect, either the lower or upper boundary of the CI, against the minimum meaningful level

15. 15 2. Test for baseline comparisons Factors Group A (n=20) Group B (n=20)P-value Age (years) 39.0  0.2 39.5  0.5 <0.001 Male, n(%) 2 (10.0%) 6 (30.0%) 0.114 Weight (kg) 60  52 30  55 0.084 Height (cm) 160  100 130  99 0.346 SBP at baseline (mmHg) 135  5 130  8 0.023 VAS (pain) at baseline 5  5 9  8 0.067 Number is mean  SD unless indicated otherwise

16. 16

17. 17 Test for baseline

18. 18 2. Test for baseline comparisons (cont.) Compare all variables that could related to an association between the exposure and the study outcome. Indication of imbalance is based on clinical judgment - no statistical test is needed. –Magnitude of the difference is matter, NOT p-value –Large or small difference is clinical judgment –If the variable is not highly correlated with the study outcome, it can be ignored even if the difference is high. If in doubt, use multivariable analysis where all imbalance variables were included in the model

19. 19 3. No magnitude of effect presented Example: See various examples in the class Tips to avoid it: –If you can’t count it, it doesn’t exist… Tribe, 1971, p.1360, 1361-2. –Always quantify magnitude of effect –Always provide the confidence interval of the effect that is the primary objective of the study

20. 20

21. 21 Factors affecting birth weight Factors Num- ber Mean Mean Diff 95%CI P- value 1. Being complete ANC Yes Yes No Noxxxxxxxx.xxxx0xx.x xx.x – xx.x 0.xxx 2. Education or mother Primary school or lower Primary school or lower Secondary school Secondary school College or higher College or higherxxxxxxxxxxx.xxx.xxx.x0xx.xxx.x xx.x – xx.x 0.xxx 3. Mother age (year) Less than 20 Less than 20 20 – 45 20 – 45 45 or older 45 or olderxxxxxxxxxxx.xxx.xxx.x0xx.xxx.x xx.x – xx.x 0.xxx

22. 22 Factors affecting low birth weight Factors Num- ber % LBW OR95%CI P- value 1. Being complete ANC Yes Yes No Noxxxxxxxx.xxx.x1x.xx x.xx – x.xx 0.xxx 2. Education or mother Primary school or lower Primary school or lower Secondary school Secondary school College or higher College or higherxxxxxxxxxxx.xxx.xxx.x1x.xxx.xx x.xx – x.xx 0.xxx 3. Mother age (year) Less than 20 Less than 20 20 – 45 20 – 45 45 or older 45 or olderxxxxxxxxxxx.xxx.xxx.x1x.xxx.xx x.xx – x.xx 0.xxx

23. 23 4. Applied inappropriate methods of analysis Example: –Inconsistent with type of the data –Not handle dependency among observation –Not accounted for sampling design –Not well handle missing data –Not accounted for confounding effects –Not investigated interaction effects Tips to avoid it: –Based on the objective and design of the study

24. 24 5. Described methods of analysis inappropriately Example: Too general Tips to avoid it: Specific and replicable

25. 25 6. Presented the results inappropriately Example: See various examples in the class and some examples as follow: –Sex (OR = 3.5) –Age (OR = 1.5) –Marital status (OR = 2.0) Tips to avoid it: –Always quantify magnitude of effect –Always provide the confidence interval of the effect that is the primary objective of the study

26. 26 Repeated measure ANOVA

27. 27 Logistic regression

28. 28 Student’s t-test

29. 29 Correlation coefficient

30. 30

31. 31 ANOVA and t-test

32. 32

33. 33 Regression Model

34. 34

35. 35 Regression model

36. 36 Concluded based on sample statistics NOT on population parameter

37. 37 Within or between group

38. 38

39. 39 7. Sample size unjustified Example: Simplified methods might be misleading Tips to avoid it:

40. 40 8. Interpret a confidence interval inappropriately Example: –Width -> wide vs narrow interval –Cross the null value -> sig- vs non-significant Tips to avoid it: –Compare magnitude of either lower or upper boundary of the interval with the meaningful level then make a judgment

41. 41 9. Categorization of the continuous variable inappropriately Example: –Continuous -> categorical –Numerical count -> categorical –Survival outcome -> categorical Tips to avoid it: –Based on the research question –Keep the intrinsic type of the variable – categorization of it can be done for exploratory purpose –Based on clinical judgments

42. 42 10. Handle the primary outcome inappropriately Example: Interchange among the following: –Continuous outcome –Categorical outcome –Numerical count –Survival outcome Tips to avoid it: –Based on the research question –Based on clinical judgments

43. 43 11. Before-after design Example: Possible approaches: –Post measurement only –Change score –Fraction –Post measurement adjusted for baseline Tips to avoid it: –Based on the research question –Preferably - post measurement adjusted for baseline

44. 44

45. 45 Between or within group comparisons?

46. 46 TimeGroup1(n=25)Group2(n=25)Group3(n=25) Diff (95%CI, p-value)* 2-13-1 Pre 1.5  1.0 1.8  1.2 2.5  2.0 NANA Post 1.7  1.0 2.5  2.0 3.5  3.0 0.8(0.2-1.6)P=0.011.8(1.2-5.6)P=0.03 Late 1.6  1.0 2.9  1.5 4.5  2.0 1.3(0.9-5.3)P=0.032.9(1.7-8.5)P=0.01 * Mean difference adjusted for baseline using ANCOVA Suggested format of presentation

47. 47 12. Jump to non-parametric test without through exploration of distribution of the data Example: “Since the sample size is small, we decided to use non-parametric test.” Tips to avoid it: –Raw data could be better than p-value obtained from non-parametric test –Small sample cannot be corrected by non- parametric statistics, in fact, we have NO SUFFICIENT evidence to allow any valid conclusions!

48. 48 13. Row total, Column total, Grand total fixed? Example: SexDiseaseNormalTotal Male8(80%)2(20%) 10 (100%) Female12(24%)38(76%) 50 (100%) Total20(33.3%)40(66.6%) 60 (100%) SexDiseaseNormalTotalMale8(40%)2(5%)10(16.7%) Female12(60%)38(95%)50(83.3%) Total 20 (100%) 40 (100%) 60 (100%) Row-total fixed -> Cohort studyColumn-total fixed -> Case-control study Tips to avoid it: Based on the study design Tips to avoid it: Based on the study design

49. 49 14. Concluded based on opinion or too general, not on the main findings or specific to the study results Example: “Effective prevention strategies should be formulated. Health education should be provided.” Tips to avoid it: –Logically link from the main finding that is the primary research question. –Specific to what was found in the study

50. 50 ผิดเป็นครู Q & A