1. Primer on Statistics for Interventional Cardiologists Giuseppe Sangiorgi, MD Pierfrancesco Agostoni, MD Giuseppe Biondi-Zoccai, MD

2. What you will learn Introduction Basics Descriptive statistics Probability distributions Inferential statistics Finding differences in mean between two groups Finding differences in mean between more than 2 groups Linear regression and correlation for bivariate analysis Analysis of categorical data (contingency tables) Analysis of time-to-event data (survival analysis) Advanced statistics at a glance Conclusions and take home messages

3. What you will learn Inferential statistics: –pivotal concepts –point estimation and confidence intervals –hypothesis testing: rationale and significance type I and type II error p values and confidence intervals multiple testing issues one-tailed and two-tailed power and sample size computation

4. Methods of inquiry Statistical inquiry may be… Descriptive (to summarize or describe an observation) or Inferential (to use the observations to make estimates or predictions)

5. Population and sample: at the heart of descriptive and inferential statistics Again: statistical inquiry may be… Descriptive (to describe a sample/population) or Inferential (to measure the likelihood that estimates generated from the sample may truly represent the underlying population)

6. Accuracy measures the distance from the true value Precision measures the spead in the measurements true value measurement Accuracy and precision

7. Accuracy measures the distance from the true value distance true value measurement Accuracy and precision

8. Accuracy measures the distance from the true value distance Precision measures the spead in the measurements spread true value measurement Accuracy and precision

9. Accuracy and precision test

13. example Accuracy and precision

14. example Schultz et al, Am Heart J 2004

15. Accuracy and precision

16. Thus: Precision expresses the extent of RANDOM ERROR Accuracy expresses the extent of SYSTEMATIC ERROR (ie bias)

17. Bias Bias is a systematic DEVIATION from the TRUTH Thus: in itself it cannot be ever recognized there is a need for one external gold standard, one or more reference standards, and/or permanent surveillance

18. An incomplete list of bias Simplest classification: 1. Selection bias 2. Information bias · Selection bias · Information bias · Confounders · Observation bias · Investigator’s bias (enthusiasm bias) · Patient’s background bias · Distribution of pathological changes bias · Selection bias · Small sample size bias · Reporting bias · Referral bias · Variation bias · Recall bias · Statistical bias · Selection bias · Confounding · Intervention bias · Measurement or information · Interpretation bias · Publication bias · Subject selection/sampling bias Sackett, J Chronic Dis 1979

19. Selection bias

20. Information bias

21. Validity Internal validity entails both PRECISION and ACCURACY (ie does a study provide a truthful answer to the research question?) External validity expresses the extent to which the results can be applied to other contexts and settings. It corresponds to the distinction between SAMPLE and POPULATION)

22. Validity

25. Meredith, EuroIntervention 2005

26. Validity Meredith, EuroIntervention 2005

27. Validity Meredith, EuroIntervention 2005 100 patients lesions ≤15 mm

28. Validity Fajadet, Circulation 2006

29. Validity Fajadet, Circulation 2006 1197 patients lesions 15-27 mm

30. Validity

31. Rothwell, Lancet 2005 Validity

32. What you will learn Inferential statistics: –pivotal concepts –point estimation and confidence intervals –hypothesis testing: rationale and significance type I and type II error p values and confidence intervals multiple testing issues one-tailed and two-tailed power and sample size computation

33. An easy comparison -0.100.200.400.600.801.0 Late loss Frequency Bx Velocity™ Cypher™

34. A tough comparison -0.1000.100.200.300.40 Late loss Frequency Cypher Select™ Cypher™

35. Point estimation & confidence intervals Using summary statistics (mean and standard deviation for normal variables, or proportion for categorical variable) and factoring sample size, we can build confidence intervals or test hypotheses that we are sampling from a given population or not This can be done by creating a powerful tool, which weighs our dispersion measures by means of the sample size: the standard error

36. Measure of dispersion are just descriptive Range –Top to bottom –Not very useful Interquartile range –Used with median –¼ way to ¾ way Standard deviation (SD) –Used with mean –Very useful 99% Confidence Interval (CI) 75% CI SD

37. From standard deviation… Standard deviation (SD): –approximates population σ as N increases Advantages: –with mean enables powerful synthesis mean±1*SD 68% of data mean±2*SD 95% of data (1.96) mean±3*SD 99% of data (2.86) Disadvantages: –is based on normal assumptions 1 )( 2 - -   N xx SD

38. Mean ± 1 standard deviation -1 SD mean +1 SD Frequency 68%

39. -1 SD+1 SD-2 SD+2 SD 95% mean Mean ± 2 standard deviations Frequency

40. -1 SD+1 SD-2 SD+2 SD 99% -3 SD+3 SD mean Frequency Mean ± 3 standard deviations

41. SD …to confidence intervals Standard error (SE or SEM) can be used to test a hypothesis or create a confidence interval (CI) around a mean for a continuous variable (eg lesion length) 95% CI = mean ± 2 SE n SE = 95% means that we can be sure at a proportion of 0.95 (almost 1!) of including the true population value in the confidence interval

42. What about proportions? We can easily build the standard error of a proportion, according to the following formula: Where variance=P*(1-P) and n is the sample size n SE = P * (1-P)

43. Point estimation & confidence intervals We can then create a simple test to check whether the summary estimate we have found can be compatible according to random variation with the corresponding reference population mean The z test (when the population SD is known) and the t test (when the population SD is only estimated), are thus used, and both can be viewed as a signal to noise ratio

44. Signal to noise ratio = Signal Noise

45. Z test Signal to noise ratio Z score = Absolute difference in summary estimates = Signal Noise Standard error Results of z score correspond to a distinct tail probability of the Gaussian curve (eg 1.96 corresponds to a 0.025 one-tailed probability or 0.050 two-tailed probability)

46. t test Signal to noise ratio t score = Absolute difference in summary estimates = Signal Noise Standard error Results of t score corresponding to a distinct tail probability of the t distribution (eg 1.96 corresponds to a 0.025 one-tailed probability or 0.050 two-tailed probability)

47. t test The t test differs from the z test as the variance is only estimated as follows: However, given the central limit theorem, when n>30 (ie with >29 degrees of freedom) the t distribution approximately corresponds to the normal distribution, thus we can use the z test and z score instead

48. What you will learn Inferential statistics: –pivotal concepts –point estimation and confidence intervals –hypothesis testing: rationale and significance type I and type II error p values and confidence intervals multiple testing issues one-tailed and two-tailed power and sample size computation

49. An easy comparison -0.100.200.400.600.801.0 Late loss Frequency Stent B Stent A

50. A tough comparison -0.1000.100.200.300.40 Late loss Frequency Stent D Stent C

51. Any comparison can be viewed as… A fight between a null hypothesis (H 0 ), stating that there is no big difference (ie beyond random variation) between two or more populations of interest (from which we are sampling) and an alternative hypothesis (H 1 ), which implies that there is a non-random difference between two or more populations of interest Any statistical test is a test that tries to tell us whether H 0 is false (thus implying H 1 ) may be true

52. Why falsifying H 0 instead of proving H 1 is true? You can never prove that something is correct in science, you can only disprove something, ie show it is wrong Thus, only falsifiable hypotheses are scientific

53. Sampling distribution of a difference 0 no difference big difference We may create a sampling distribution of a difference for any comparison of interest eg late loss, peak CK-MB or survival rates A B A-B

54. big difference True difference distribution Difference in our study Sampling distribution of a difference (ie null hypothesis or H 0 ) 0 no difference

55. Potential difference distributions 0 no difference big difference (ie null hypothesis or H 0 )

56. 0 no difference big difference Potential difference distributions (ie null hypothesis or H 0 )

57. 0 no difference big difference Potential difference distributions (ie null hypothesis or H 0 )

58. What you will learn Inferential statistics: –pivotal concepts –point estimation and confidence intervals –hypothesis testing: rationale and significance type I and type II error p values and confidence intervals multiple testing issues one-tailed and two-tailed power and sample size computation

59. Gray zone where…...we may inappropriately reject a true null hypothesis (H 0 ), ie providing a false positive result 0 no difference big difference Potential pitfalls (alpha error)

60. big difference Another gray zone where…...we may fail to reject a false null hypothesis (H 0 ), ie providing a false negative result 0 no difference Potential pitfalls (beta error)

61. True positive test Sampling here means correctly rejecting a false null hypothesis (H 0 ), ie providing a true positive result big difference 0 no difference

62. True negative test Sampling here means correctly retaining a true null hypothesis (H 0 ), ie providing a true negative result big difference 0 no difference

63. Statistical or clinical significance Clinical and statistical significance are to highly different concepts A clinically significant difference, if proved true, would be considered clinically relevant and thus worthwhile (pending costs and tolerability) A statistically significant difference is a probability concept, and should be viewed in light of the distance from the null hypothesis and the chosen significance treshold

64. Alpha and type I error Whenever I perform a test, there is thus a risk of a FALSE POSITIVE result, ie REJECTING A TRUE null hypothesis This error is called type I, is measured as alpha and its unit is the p value The lower the p value, the lower the risk of falling into a type I error (ie the HIGHER the SPECIFICITY of the test)

65. Alpha and type I error Type I error is like a MIRAGE Because I see something that does NOT exist

66. Beta and type II error Whenever I perform a test, there is also a risk of a FALSE NEGATIVE result, ie NOT REJECTING A FALSE null hypothesis This error is called type II, is measured as beta and its unit is a probability The complementary of beta is called power The lower the beta, the lower the risk of missing a true difference (ie the HIGHER the SENSITIVITY of the test)

67. Type II error is like being BLIND Because I do NOT see something that exists Beta and type II error

68. Non-invasive diagnosis of CAD Stress testing AbnormalNormal CAD Yes No

69. Non-invasive diagnosis of CAD Stress testing AbnormalNormal CAD Yes True positive No

70. Non-invasive diagnosis of CAD Stress testing AbnormalNormal CAD Yes True positive False negative No

71. Non-invasive diagnosis of CAD Stress testing AbnormalNormal CAD Yes True positive False negative No False positive

72. Non-invasive diagnosis of CAD Stress testing AbnormalNormal CAD Yes True positive False negative No False positive True negative

73. Non-invasive diagnosis of CAD Stress testing AbnormalNormal CAD Yes True positive False negative No False positive True negative OOOPS!

74. Summary of errors Experimental study H 0 acceptedH 0 rejected Truth H 0 true H 0 false

75. Summary of errors Experimental study H 0 acceptedH 0 rejected Truth H 0 true H 0 false

76. Summary of errors Experimental study H 0 acceptedH 0 rejected Truth H 0 true Type I error H 0 false

77. Summary of errors Experimental study H 0 acceptedH 0 rejected Truth H 0 true Type I error H 0 false Type II error

78. Pitt et al, Lancet 1997 Type I error

79. Pitt et al, Lancet 2000 Type I error

80. Type II error Burzotta, J Am Coll Cardiol

81. Type II error De Luca, Eur Heart J 2008

82. Another example of beta error? Kandzari et al, JACC 2006

83. Another example of beta error? Melikian et al, Heart 2007 The PROSPECT Trial Inclusion criteria Comparison Sample size Primary end-point Consecutive patients with PCI of up to 4 lesions Endeavor vs Cypher 8800 Stent thrombosis at 3- year follow-up and MACE

84. Shapes of distribution & analytical errors Value Frequency

85. 20 10 0 Frequency Value Another potential cause of analytic errors

86. What you will learn Inferential statistics: –pivotal concepts –point estimation and confidence intervals –hypothesis testing: rationale and significance type I and type II error p values and confidence intervals multiple testing issues one-tailed and two-tailed power and sample size computation

87. P values

88. 95% Confidence intervals The RANGE of values where we would have CONFIDENCE that the population value lies in 95 cases, if we were to perform 100 studies summary point estimate 95% confidence interval 99% confidence interval

89. Confidence intervals

90. P values & confidence intervals

91. Ps and confidence intervals P values and confidence intervals are strictly connected Any hypothesis test providing a significant result (eg p=0.045) means that we can be confident at 95.5% that the population average difference lies far from zero (ie the null hypothesis)

92. Thus this statistical analysis reports an odds ratio of 0.111, with 95% confidence intervals of 0.16 to 0.778, and a concordantly significant p value of 0.027 Ps and confidence intervals

93. significant difference (p<0.05) non significant difference (p>0.05) HoHo important difference trivial difference P values and confidence intervals

94. What you will learn Inferential statistics: –pivotal concepts –point estimation and confidence intervals –hypothesis testing: rationale and significance type I and type II error p values and confidence intervals multiple testing issues one-tailed and two-tailed power and sample size computation

95. Multiple testing What happens when you perform the same hypothesis test several times? …

96. Multiple testing What happens when you perform the same hypothesis test several times? … The answer is restricting the analyses only to prespecified and biologically plausible sub-analysis, and using suitable corrections: –Bonferroni –Dunn –Tukey –Keuls –interaction tests

97. Multiple testing & subgroups1.29 8.7% (20) 6.7% (15) 0.956 0.956 1.27 4.7% (24) 3.7% (19) Risk Ratio [95% CI] Risk RatioEndeavorTaxus P value* Diabetes Non-diabetes RVD  2.5mm >2.5 <3.0mm  3.0mm Lesion Length  10mm >10 10 <20mm  20mm Single Stent Multiple Stents 1.61 5.7% (39) 3.5% (23) 0.324 0.84 9.1% (4) 10.8% (8) 1.07 5.2% (12) 4.9% (11) 0.412 0.412 1.38 6.2% (26) 4.5% (18) 1.21 5.6% (5) 4.6% (5) 0.98 6.4% (16) 6.5% (17) 0.187 0.187 1.21 5.9% (17) 4.8% (14) 3.45 5.5% (11) 1.6% (3) Favors Endeavor Favors Taxus 110 0.1 ENDEAVOR IV – 24-month TLR rates *interaction p values calculated using logistic regression

98. What you will learn Inferential statistics: –pivotal concepts –point estimation and confidence intervals –hypothesis testing: rationale and significance type I and type II error p values and confidence intervals multiple testing issues one-tailed and two-tailed power and sample size computation

99. <2.5% One- or two-tailed tests mean± 1.96*SD

100. <2.5% When can we use a one-tailed test? When you assume that the difference is only in one direction: ALMOST NEVER for superiority or equivalence comparisons One- or two-tailed tests mean± 1.96*SD

101. <2.5% When can we use a one-tailed test? When you assume that the difference is only in one direction: ALMOST NEVER for superiority or equivalence comparisons When should you we use a two-tailed test? When you cannot assume the direction of the difference: ALMOST ALWAYS, except for non-inferiority comparisons One- or two-tailed tests mean± 1.96*SD

102. What you will learn Inferential statistics: –pivotal concepts –point estimation and confidence intervals –hypothesis testing: rationale and significance type I and type II error p values and confidence intervals multiple testing issues one-tailed and two-tailed power and sample size computation

103. Sample size calculation To compute the sample size for a study we thus need: 1. Preferred alpha value 2. Preferred beta value 3. Control event rate or average value (with measure of dispersion if applicable) 4. Expected relative reduction in experimental group Svilaas et al, NEJM 2008

104. Another sample size example Fajadet, Circulation 2006

105. Power and sample size Whenever designing a study or analyzing a dataset, it is important to estimate the sample size or the power of the comparison SAMPLE SIZE Setting a specific alpha and a specific beta, you calculate the necessary sample size given the average inter-group difference and its variation POWER Given a specific sample size and alpha, in light of the calculated average inter-group difference and its variation, you obtain an estimate of the power (ie 1-beta)

106. Power and sample size Whenever designing a study or analyzing a dataset, it is important to estimate the sample size or the power of the comparison SAMPLE SIZE Setting a specific alpha and a specific beta, you calculate the necessary sample size given the average inter-group difference and its variation POWER Given a specific sample size and alpha, in light of the calculated average inter-group difference and its variation, you obtain an estimate of the power (ie 1-beta)

107. Power analysis Biondi-Zoccai et al, Ital Heart J 2003 To compute the power of a study we thus need: 1. Preferred or actual alpha value 2. Control event rate or average value ( with measure of dispersion if applicable) 3. Expected or actual relative reduction in experimental group 4.Expected or actual sample size

108. Thank you for your attention For any correspondence: gbiondizoccai@gmail.com For further slides on these topics feel free to visit the metcardio.org website: http://www.metcardio.org/slides.html gbiondizoccai@gmail.com http://www.metcardio.org/slides.html