Primer on Statistics for Interventional Cardiologists Giuseppe Sangiorgi, MD Pierfrancesco Agostoni, MD Giuseppe Biondi-Zoccai, MD

What you will learn Introduction Basics Descriptive statistics Probability distributions Inferential statistics Finding differences in mean between two groups Finding differences in mean between more than 2 groups Linear regression and correlation for bivariate analysis Analysis of categorical data (contingency tables) Analysis of time-to-event data (survival analysis) Advanced statistics at a glance Conclusions and take home messages

What you will learn Inferential statistics: –pivotal concepts –point estimation and confidence intervals –hypothesis testing: rationale and significance type I and type II error p values and confidence intervals multiple testing issues one-tailed and two-tailed power and sample size computation

Methods of inquiry Statistical inquiry may be… Descriptive (to summarize or describe an observation) or Inferential (to use the observations to make estimates or predictions)

Population and sample: at the heart of descriptive and inferential statistics Again: statistical inquiry may be… Descriptive (to describe a sample/population) or Inferential (to measure the likelihood that estimates generated from the sample may truly represent the underlying population)

Accuracy measures the distance from the true value Precision measures the spead in the measurements true value measurement Accuracy and precision

Accuracy measures the distance from the true value distance true value measurement Accuracy and precision

Accuracy measures the distance from the true value distance Precision measures the spead in the measurements spread true value measurement Accuracy and precision

Accuracy and precision test

example Accuracy and precision

example Schultz et al, Am Heart J 2004

Accuracy and precision

Thus: Precision expresses the extent of RANDOM ERROR Accuracy expresses the extent of SYSTEMATIC ERROR (ie bias)

Bias Bias is a systematic DEVIATION from the TRUTH Thus: in itself it cannot be ever recognized there is a need for one external gold standard, one or more reference standards, and/or permanent surveillance

An incomplete list of bias Simplest classification: 1. Selection bias 2. Information bias · Selection bias · Information bias · Confounders · Observation bias · Investigator’s bias (enthusiasm bias) · Patient’s background bias · Distribution of pathological changes bias · Selection bias · Small sample size bias · Reporting bias · Referral bias · Variation bias · Recall bias · Statistical bias · Selection bias · Confounding · Intervention bias · Measurement or information · Interpretation bias · Publication bias · Subject selection/sampling bias Sackett, J Chronic Dis 1979

Selection bias

Information bias

Validity Internal validity entails both PRECISION and ACCURACY (ie does a study provide a truthful answer to the research question?) External validity expresses the extent to which the results can be applied to other contexts and settings. It corresponds to the distinction between SAMPLE and POPULATION)

Validity

Meredith, EuroIntervention 2005

Validity Meredith, EuroIntervention 2005

Validity Meredith, EuroIntervention patients lesions ≤15 mm

Validity Fajadet, Circulation 2006

Validity Fajadet, Circulation patients lesions mm

Validity

Rothwell, Lancet 2005 Validity

What you will learn Inferential statistics: –pivotal concepts –point estimation and confidence intervals –hypothesis testing: rationale and significance type I and type II error p values and confidence intervals multiple testing issues one-tailed and two-tailed power and sample size computation

An easy comparison Late loss Frequency Bx Velocity™ Cypher™

A tough comparison Late loss Frequency Cypher Select™ Cypher™

Point estimation & confidence intervals Using summary statistics (mean and standard deviation for normal variables, or proportion for categorical variable) and factoring sample size, we can build confidence intervals or test hypotheses that we are sampling from a given population or not This can be done by creating a powerful tool, which weighs our dispersion measures by means of the sample size: the standard error

Measure of dispersion are just descriptive Range –Top to bottom –Not very useful Interquartile range –Used with median –¼ way to ¾ way Standard deviation (SD) –Used with mean –Very useful 99% Confidence Interval (CI) 75% CI SD

From standard deviation… Standard deviation (SD): –approximates population σ as N increases Advantages: –with mean enables powerful synthesis mean±1*SD 68% of data mean±2*SD 95% of data (1.96) mean±3*SD 99% of data (2.86) Disadvantages: –is based on normal assumptions 1 )(   N xx SD

Mean ± 1 standard deviation -1 SD mean +1 SD Frequency 68%

-1 SD+1 SD-2 SD+2 SD 95% mean Mean ± 2 standard deviations Frequency

-1 SD+1 SD-2 SD+2 SD 99% -3 SD+3 SD mean Frequency Mean ± 3 standard deviations

SD …to confidence intervals Standard error (SE or SEM) can be used to test a hypothesis or create a confidence interval (CI) around a mean for a continuous variable (eg lesion length) 95% CI = mean ± 2 SE n SE = 95% means that we can be sure at a proportion of 0.95 (almost 1!) of including the true population value in the confidence interval

What about proportions? We can easily build the standard error of a proportion, according to the following formula: Where variance=P*(1-P) and n is the sample size n SE = P * (1-P)

Point estimation & confidence intervals We can then create a simple test to check whether the summary estimate we have found can be compatible according to random variation with the corresponding reference population mean The z test (when the population SD is known) and the t test (when the population SD is only estimated), are thus used, and both can be viewed as a signal to noise ratio

Signal to noise ratio = Signal Noise

Z test Signal to noise ratio Z score = Absolute difference in summary estimates = Signal Noise Standard error Results of z score correspond to a distinct tail probability of the Gaussian curve (eg 1.96 corresponds to a one-tailed probability or two-tailed probability)

t test Signal to noise ratio t score = Absolute difference in summary estimates = Signal Noise Standard error Results of t score corresponding to a distinct tail probability of the t distribution (eg 1.96 corresponds to a one-tailed probability or two-tailed probability)

t test The t test differs from the z test as the variance is only estimated as follows: However, given the central limit theorem, when n>30 (ie with >29 degrees of freedom) the t distribution approximately corresponds to the normal distribution, thus we can use the z test and z score instead

What you will learn Inferential statistics: –pivotal concepts –point estimation and confidence intervals –hypothesis testing: rationale and significance type I and type II error p values and confidence intervals multiple testing issues one-tailed and two-tailed power and sample size computation

An easy comparison Late loss Frequency Stent B Stent A

A tough comparison Late loss Frequency Stent D Stent C

Any comparison can be viewed as… A fight between a null hypothesis (H 0 ), stating that there is no big difference (ie beyond random variation) between two or more populations of interest (from which we are sampling) and an alternative hypothesis (H 1 ), which implies that there is a non-random difference between two or more populations of interest Any statistical test is a test that tries to tell us whether H 0 is false (thus implying H 1 ) may be true

Why falsifying H 0 instead of proving H 1 is true? You can never prove that something is correct in science, you can only disprove something, ie show it is wrong Thus, only falsifiable hypotheses are scientific

Sampling distribution of a difference 0 no difference big difference We may create a sampling distribution of a difference for any comparison of interest eg late loss, peak CK-MB or survival rates A B A-B

big difference True difference distribution Difference in our study Sampling distribution of a difference (ie null hypothesis or H 0 ) 0 no difference

Potential difference distributions 0 no difference big difference (ie null hypothesis or H 0 )

0 no difference big difference Potential difference distributions (ie null hypothesis or H 0 )

0 no difference big difference Potential difference distributions (ie null hypothesis or H 0 )

What you will learn Inferential statistics: –pivotal concepts –point estimation and confidence intervals –hypothesis testing: rationale and significance type I and type II error p values and confidence intervals multiple testing issues one-tailed and two-tailed power and sample size computation

Gray zone where…...we may inappropriately reject a true null hypothesis (H 0 ), ie providing a false positive result 0 no difference big difference Potential pitfalls (alpha error)

big difference Another gray zone where…...we may fail to reject a false null hypothesis (H 0 ), ie providing a false negative result 0 no difference Potential pitfalls (beta error)

True positive test Sampling here means correctly rejecting a false null hypothesis (H 0 ), ie providing a true positive result big difference 0 no difference

True negative test Sampling here means correctly retaining a true null hypothesis (H 0 ), ie providing a true negative result big difference 0 no difference

Statistical or clinical significance Clinical and statistical significance are to highly different concepts A clinically significant difference, if proved true, would be considered clinically relevant and thus worthwhile (pending costs and tolerability) A statistically significant difference is a probability concept, and should be viewed in light of the distance from the null hypothesis and the chosen significance treshold

Alpha and type I error Whenever I perform a test, there is thus a risk of a FALSE POSITIVE result, ie REJECTING A TRUE null hypothesis This error is called type I, is measured as alpha and its unit is the p value The lower the p value, the lower the risk of falling into a type I error (ie the HIGHER the SPECIFICITY of the test)

Alpha and type I error Type I error is like a MIRAGE Because I see something that does NOT exist

Beta and type II error Whenever I perform a test, there is also a risk of a FALSE NEGATIVE result, ie NOT REJECTING A FALSE null hypothesis This error is called type II, is measured as beta and its unit is a probability The complementary of beta is called power The lower the beta, the lower the risk of missing a true difference (ie the HIGHER the SENSITIVITY of the test)

Type II error is like being BLIND Because I do NOT see something that exists Beta and type II error

Non-invasive diagnosis of CAD Stress testing AbnormalNormal CAD Yes No

Non-invasive diagnosis of CAD Stress testing AbnormalNormal CAD Yes True positive No

Non-invasive diagnosis of CAD Stress testing AbnormalNormal CAD Yes True positive False negative No

Non-invasive diagnosis of CAD Stress testing AbnormalNormal CAD Yes True positive False negative No False positive

Non-invasive diagnosis of CAD Stress testing AbnormalNormal CAD Yes True positive False negative No False positive True negative

Non-invasive diagnosis of CAD Stress testing AbnormalNormal CAD Yes True positive False negative No False positive True negative OOOPS!

Summary of errors Experimental study H 0 acceptedH 0 rejected Truth H 0 true H 0 false

Summary of errors Experimental study H 0 acceptedH 0 rejected Truth H 0 true H 0 false

Summary of errors Experimental study H 0 acceptedH 0 rejected Truth H 0 true Type I error H 0 false

Summary of errors Experimental study H 0 acceptedH 0 rejected Truth H 0 true Type I error H 0 false Type II error

Pitt et al, Lancet 1997 Type I error

Pitt et al, Lancet 2000 Type I error

Type II error Burzotta, J Am Coll Cardiol

Type II error De Luca, Eur Heart J 2008

Another example of beta error? Kandzari et al, JACC 2006

Another example of beta error? Melikian et al, Heart 2007 The PROSPECT Trial Inclusion criteria Comparison Sample size Primary end-point Consecutive patients with PCI of up to 4 lesions Endeavor vs Cypher 8800 Stent thrombosis at 3- year follow-up and MACE

Shapes of distribution & analytical errors Value Frequency

Frequency Value Another potential cause of analytic errors

What you will learn Inferential statistics: –pivotal concepts –point estimation and confidence intervals –hypothesis testing: rationale and significance type I and type II error p values and confidence intervals multiple testing issues one-tailed and two-tailed power and sample size computation

P values

95% Confidence intervals The RANGE of values where we would have CONFIDENCE that the population value lies in 95 cases, if we were to perform 100 studies summary point estimate 95% confidence interval 99% confidence interval

Confidence intervals

P values & confidence intervals

Ps and confidence intervals P values and confidence intervals are strictly connected Any hypothesis test providing a significant result (eg p=0.045) means that we can be confident at 95.5% that the population average difference lies far from zero (ie the null hypothesis)

Thus this statistical analysis reports an odds ratio of 0.111, with 95% confidence intervals of 0.16 to 0.778, and a concordantly significant p value of Ps and confidence intervals

significant difference (p<0.05) non significant difference (p>0.05) HoHo important difference trivial difference P values and confidence intervals

What you will learn Inferential statistics: –pivotal concepts –point estimation and confidence intervals –hypothesis testing: rationale and significance type I and type II error p values and confidence intervals multiple testing issues one-tailed and two-tailed power and sample size computation

Multiple testing What happens when you perform the same hypothesis test several times? …

Multiple testing What happens when you perform the same hypothesis test several times? … The answer is restricting the analyses only to prespecified and biologically plausible sub-analysis, and using suitable corrections: –Bonferroni –Dunn –Tukey –Keuls –interaction tests

Multiple testing & subgroups % (20) 6.7% (15) % (24) 3.7% (19) Risk Ratio [95% CI] Risk RatioEndeavorTaxus P value* Diabetes Non-diabetes RVD  2.5mm >2.5 <3.0mm  3.0mm Lesion Length  10mm >10 10 <20mm  20mm Single Stent Multiple Stents % (39) 3.5% (23) % (4) 10.8% (8) % (12) 4.9% (11) % (26) 4.5% (18) % (5) 4.6% (5) % (16) 6.5% (17) % (17) 4.8% (14) % (11) 1.6% (3) Favors Endeavor Favors Taxus ENDEAVOR IV – 24-month TLR rates *interaction p values calculated using logistic regression

What you will learn Inferential statistics: –pivotal concepts –point estimation and confidence intervals –hypothesis testing: rationale and significance type I and type II error p values and confidence intervals multiple testing issues one-tailed and two-tailed power and sample size computation

<2.5% One- or two-tailed tests mean± 1.96*SD

<2.5% When can we use a one-tailed test? When you assume that the difference is only in one direction: ALMOST NEVER for superiority or equivalence comparisons One- or two-tailed tests mean± 1.96*SD

<2.5% When can we use a one-tailed test? When you assume that the difference is only in one direction: ALMOST NEVER for superiority or equivalence comparisons When should you we use a two-tailed test? When you cannot assume the direction of the difference: ALMOST ALWAYS, except for non-inferiority comparisons One- or two-tailed tests mean± 1.96*SD

What you will learn Inferential statistics: –pivotal concepts –point estimation and confidence intervals –hypothesis testing: rationale and significance type I and type II error p values and confidence intervals multiple testing issues one-tailed and two-tailed power and sample size computation

Sample size calculation To compute the sample size for a study we thus need: 1. Preferred alpha value 2. Preferred beta value 3. Control event rate or average value (with measure of dispersion if applicable) 4. Expected relative reduction in experimental group Svilaas et al, NEJM 2008

Another sample size example Fajadet, Circulation 2006

Power and sample size Whenever designing a study or analyzing a dataset, it is important to estimate the sample size or the power of the comparison SAMPLE SIZE Setting a specific alpha and a specific beta, you calculate the necessary sample size given the average inter-group difference and its variation POWER Given a specific sample size and alpha, in light of the calculated average inter-group difference and its variation, you obtain an estimate of the power (ie 1-beta)

Power and sample size Whenever designing a study or analyzing a dataset, it is important to estimate the sample size or the power of the comparison SAMPLE SIZE Setting a specific alpha and a specific beta, you calculate the necessary sample size given the average inter-group difference and its variation POWER Given a specific sample size and alpha, in light of the calculated average inter-group difference and its variation, you obtain an estimate of the power (ie 1-beta)

Power analysis Biondi-Zoccai et al, Ital Heart J 2003 To compute the power of a study we thus need: 1. Preferred or actual alpha value 2. Control event rate or average value ( with measure of dispersion if applicable) 3. Expected or actual relative reduction in experimental group 4.Expected or actual sample size

Thank you for your attention For any correspondence: For further slides on these topics feel free to visit the metcardio.org website: