Understanding Study Design & Statistics Dr Malachy O. Columb FRCA, FFICM University Hospital of South Manchester NWRAG Workshop, Bolton, May 2015

COIs: Interesting Confllicts! Editorial Board Roles: European Journal of Anaesthesiology British Journal of Anaesthesia International Journal of Obstetric Anesthesia

Manuscript Types (7) Meta-analysis & systematic reviews (6) Original research – PDBRCT (5) Original research – other RCT (4) Original research – observational (3) Original research – retrospective (2) Narrative reviews – including editorials (1) Case reports, abstracts & letters

Manuscript Types (7) Original research – PDBRCT (6) Original research – other RCT (5) Original research – observational (4) Original research – retrospective (3) Meta-analysis & systematic reviews (2) Narrative reviews – including editorials (1) Case reports, abstracts & letters

Statistics: Definition …the discipline concerned with the treatment of numerical data derived from groups of individuals…

Data …are always plural… ‘Datum’ is the singular…

Types of Data Numerical – continuous & discrete Categorical – binary, nominal, ordinal

Hypotheses Null hypothesis (H O ) Alternative hypothesis (H A ) P value and 95% confidence interval Two-sided by convention One-sided are rarely appropriate Equivalence, Non-inferiority, Superiority (Margins) Inequality is the usual H A Potencies and probabilities: One-sided P values suggest a one-sided story! Columb MO, Polley LS. Anesthesia & Analgesia 2001;92:278-9

Controlling Bias - Design Prospective > Retrospective Double Blind > Single Blind > Unblinded Randomised Controlled Trial > Unrandomised PDBRCT > Propensity Score Matching! PROBE (Single Blind)

Sample Size Power analysis and sample size calculations. Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.

Sample size Minimum difference that is (clinically) important Defines primary outcome! Multiple comparisons! Power analysis and sample size calculations. Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.

Estimate of SD Published research Pilot data Empirical approach 1/5th – ‘one fifth’ of the range Power analysis and sample size calculations. Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.

One-Fifth Range 4 SD = 95.4% of values 6 SD = 99.7% of values Take 1/5th range to approximate SD 20% of the range Power analysis and sample size calculations. Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.

Standardised Difference Difference / SD Power analysis and sample size calculations. Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.

Standardised Difference = 1.0

Nonparametric Adjustment Add 16% more subjects per group! Power analysis and sample size calculations. Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.

Sample Size - Proportions Power analysis and sample size calculations. Columb MO, Stevens A. Current Anaesthesia & Critical Care 2008; 19: 12-4.

Standardised Difference = 1.0

Descriptive Statistics Sample Mean (SD) – 68% of data Median [interquartiles, range] Count/frequency

Inferential Statistics - Precision Population estimates; precision Differences in means, medians, proportions Mean or mean difference Sampling theory!

Population (variable X) Distribution of sample means (variable ) Population of means (variable ) µ µ Sample 1 Sample jSample 3 Sample 2 x 1,x x n 1 23 j Randomization x

100 random samples of size random samples of size random samples of size random samples of size 20

Inferential Statistics - Precision SD of sampled means is the SE of mean SE mean = SD /  n SEM = 68%CI, (precision) SEM x 1.96 = 95%CI (precision) Test statistic = difference / SE difference P value

Significance P value – ‘probability of the observed difference or greater assuming the null hypothesis’ Type I or alpha error <0.05; false +ve Type II or beta error <0.20; false -ve Multiple comparisons - Bonferroni correction Corrections to 95% CI of difference

Group Tests

Statistical Analyses Correlation – Pearson, Spearman, intraclass Regression – linear, logistic, probit, survival Diagnostics – sensitivity, specificity, ROC curves Reference intervals – normal range Agreement – kappa, Bland-Altman plots

Transformations

Time-to-Event: Log Transformation

Analyses for RCT Per-Protocol (PP) Received allocated treatment and completed protocol Largest estimate of effect size Selection bias for post-treatment withdrawals Treatment-Received (TR) Received allocated treatment May not have completed the protocol Selection bias for pre-treatment withdrawals Intention-to-Treat (ITT) All randomised subjects – NO WITHDRAWALS May or may not have received the intervention Underestimates true effect size of treatment Most robust analysis

MOCPASS –