What do you think about a doctor who uses the wrong treatment, either wilfully or through ignorance, or who uses the right treatment wrongly (such as by giving the wrong dose of a drug)? Most people would agree that such behaviour is unprofessional, arguably unethical, and certainly unacceptable. Derived from: Altman DG. The Scandal of Poor Medical Research. BMJ, 1994; 308:283

What do you think about researchers who use the wrong techniques (either wilfully or in ignorance), use the right techniques wrongly, misinterpret their results, report their results selectively or draw unjustified conclusions? We should be appalled… but numerous studies of the medical literature have shown that all of the above phenomena are common. Derived from: Altman DG. The Scandal of Poor Medical Research. BMJ, 1994; 308:283

Understanding your results Research Talk 2015 Dr Emily Karahalios Office for Research, Western Centre for Health Research & Education Centre for Epidemiology and Biostatistics, Melbourne School of Population and Global Health, University of Melbourne

Overview Defining your research question – PICOS Describing data Understanding the results –Estimates reported in the literature –Interpreting 95% confidence intervals and p- values ~ Statistical Inference

Research question P articipants / population neonates I ntervention / exposure 14 day administration of antenatal corticosteroids C omparison 7 day administration of antenatal corticosteroids O utcome Neonatal mortality and neonatal morbidity S tudy design RCT

Murphy et al. The Lancet, 2008; 372: Research question

P articipants / population Neonates I ntervention / exposure 14 day administration of antenatal corticosteroids C omparison 7 day administration of antenatal corticosteroids O utcome Neonatal mortality and neonatal morbidity S tudy design RCT

Research question P articipants / population Women at high risk of preterm birth I ntervention / exposure 14 day administration of antenatal corticosteroids C omparison 7 day administration of antenatal corticosteroids O utcome Neonatal mortality and neonatal morbidity S tudy design RCT

Study designs The general idea… –Evaluate whether a risk factor (or preventative factor) increases (decreases) the risk of an outcome (e.g. disease, death, etc) exposure outcome time

Overview Defining your research question – PICOS Describing data Understanding the results –Estimates reported in the literature –Interpreting 95% confidence intervals and p- values ~ Statistical Inference

Study designs The general idea… –Evaluate whether a risk factor (or preventative factor) increases (decreases) the risk of an outcome (e.g. disease, death, etc) exposure outcome time

Murphy et al. The Lancet, 2008; 372: Summarising the data

Dreyfus et al. Journal of Pediatrics, 2015 online.

Summarising the data

Numerical Categorical Continuous (age, weight, height) Discrete (length of stay, # of hospital visits) Nominal (sex, blood group) Ordinal (tumour stage, quintile of SES) Discrete Nominal Ordinal

Which variables are categorical? –Sex (Male/Female) –Country of birth (Australia/Elsewhere) Which variables are continuous? –Age (years) –Length of stay (days) Summarising the data

Stata command: histogram Age

Summarising the data Standard deviation Mean = 49.8 years = 2.1 years Note, 95% of observations lie within approximately ±2×SD of the mean. In this example, 95% of observations lie within 45.6 and 54.0 years.

Summarising the data

Stata command: hist LOS

Summarising the data Stata command: hist LOS, normal

Summarising the data Mean = 5 days

Summarising the data Mean = 5 days Median = 50 th percentile = 4 days

Summarising the data Mean = 5 days Standard deviation Median = 4 days Mean is not a good measure of central tendency and standard deviation is not a good measures of spread for a skewed distribution Note, 95% of observations lie within approximately ±2SD of the mean. In this example, 95% of observations lie within -4.8 and 14.8 days BUT they don’t because LOS can’t be negative!

Summarising the data Inter-quartile range (IQR) = lower quartile – upper quartile = 25 th percentile – 75 th percentile = 2 to 6 days Median = 50 th percentile = 4 days

Summarising the data

Spread Central tendency Summarising the data

Positive skew Negative skew

Data variable - numerical Plot histogram Normally distributed NOT normally distributed UnimodalMultimodal Mean Standard deviation Minimum-maximum Median Inter-quartile range Minimum-maximum Categorise variable Summarising the data Simpson et al. J Fam Plan and Rep Health Care, 2001; 27:

Summarising the data Absolutely critical to choosing the appropriate form of statistical analysis Normally distributed Skewed Numerical Categorical Continuous (age, weight, height) Discrete (length of stay, # of hospital visits) Nominal (sex, blood group) Ordinal (tumour stage, quintile of SES)

Overview Defining your research question – PICOS Describing data Understanding the results –Estimates reported in the literature –Interpreting 95% confidence intervals and p- values ~ Statistical Inference

Study designs The general idea… –Evaluate whether a risk factor (or preventative factor) increases (decreases) the risk of an outcome (e.g. disease, death, etc) exposure outcome time

Estimates reported in the literature –Risk differences –Odds ratios / risk ratio – logistic regression –Beta-coefficients – linear regression

Summarising the data Normally distributed Skewed Numerical Categorical Continuous (age, weight, height) Discrete (length of stay, # of hospital visits) Nominal (sex, blood group) Ordinal (tumour stage, quintile of SES)

Measures of association – binary outcome Binary variables – two categories only (also termed – dichotomous variable) Examples: Outcome – diseased or healthy; alive or dead Exposure – male or female; smoker or non-smoker; treatment or control group

Comparing two proportions With outcome (diseased) Without outcome (disease free) Total Exposed (group 1) d1d1 h1h1 n1n1 Unexposed (group 0) d0d0 h0h0 n0n0 Totaldhn Proportion of all subjects experiencing outcome, p = d/n Proportion of exposed group, p 1 = d 1 /n 1 Proportion of unexposed group, p 0 = d 0 /n 0

Comparing two proportions - TBM Trial Adults with tuberculous meningitis randomly allocated into 2 treatment groups: 1.Dexamethasone 2.Placebo Outcome measure: Death during 9 months following start of treatment. Research question: Can treatment with dexamethasone reduce the risk of death among adults with tuberculous meningitis? Thwaites et al 2004

Comparing two proportions Death during 9 months post start of treatment Treatment groupYesNoTotal Dexamethasone (group 1) Placebo (group 0) Total Thwaites et al 2004

Comparing two proportions - TBM Trial Measure of effectFormula Risk differencep 1 -p 0 Risk Ratio (RR)p 1 /p 0 Odds Ratio (OR)(d 1 /h 1 )/(d 0 /h 0 ) When there is no association between exposure and outcome: –Risk difference = 0 –Risk ratio (RR) = 1 –Odds Ratio (OR) = 1

Comparing two proportions Death during 9 months post start of treatment Treatment groupYesNoTotal Dexamethasone (group 1) 87 (d 1 )187 (h 1 )274 (n 1 ) Placebo (group 0) 112 (d 0 )159 (h 0 )271 (n 0 ) Total Risk difference = p 1 -p 0 = (87/274)-(112/271) = Risk ratio = p 1 /p 0 = (87/274)/(112/271) = 0.77 Odds ratio = (d 1 /h 1 )/(d 0 /h 0 ) = (87/187)/(112/159) = 0.66 Thwaites et al 2004

Comparing two proportions - TBM Trial

Estimates reported in the literature –Risk differences –Odds ratios / risk ratio – logistic regression –Beta-coefficients – linear regression

Summarising the data Normally distributed Skewed Numerical Categorical Continuous (age, weight, height) Discrete (length of stay, # of hospital visits) Nominal (sex, blood group) Ordinal (tumour stage, quintile of SES)

Linear regression Dreyfus et al. Journal of Pediatrics, 2015 online.

There are four assumptions underlying our linear regression model: Linearity (outcome and exposure) Normality (residual variation) Independence (of observations) Homoscedasticity (constant variance) Linear regression

Overview Defining your research question – PICOS Describing data Understanding the results –Estimates reported in the literature –Interpreting 95% confidence intervals and p- values ~ Statistical Inference

Statistical Inference

We follow a standard four-step process 1)Sample size 2)Estimate of the effect size 3)Calculate a confidence interval 4)Derive a p-value to test the hypothesis of no association Statistical Inference

P-value How likely is it we would see a difference this big IF There was NO real difference between the populations? What is the probability (P-value) of finding the observed difference IF The null hypothesis is true? Statistical Inference

P-value Increasing evidence against the null hypothesis with decreasing P-value Weak evidence against the null hypothesis Interpretation of p-values Strong evidence against the null hypothesis Statistical Inference

Overweight and obese adults living in the UK 300 adults participating in a RCT comparing 2 dietary interventions Mean weight loss after 4 weeks Atkins group – 4.40 kg Weight Watchers group – 2.86 kg Source: Truby H et al. BMJ 2007

Example: Randomised controlled trial of weight loss programmes in the UK Groupn Sample mean Weight loss after 4 weeks (kg) Sample standard deviation Sample standard error Atkins Weight Watchers ) Estimate of difference in population mean weight loss after 4 weeks between Atkins & Weight Watchers groups = 4.40 – 2.86 = 1.54 kg 2) 95% CI: 0.67 kg to 2.41 kg Source: Truby H et al. BMJ 2007 Statistical Inference

Interpretation 1)We found a difference of 1.54 kg in mean weight loss after 4 weeks between the Atkins & Weight Watchers diet groups. 2)From the 95% confidence interval, the true difference could be as much as 2.41 kg (much greater weight loss for Atkins diet) or 0.67 kg (marginally greater weight loss for the Atkins diet compared with Weight Watchers). Statistical Inference

P-value: comparing two groups How likely is it we would see a difference this big IF There was NO real difference between the populations? What is the probability (P-value) of finding the observed difference IF The null hypothesis is true? Statistical Inference

Null hypothesis – There is no difference in the population mean weight loss after 4 weeks between the Atkins and Weight Watchers groups 2-sided p-value <0.001 Thus the probability of observing a difference of at least 1.54 kg in the sample means of the two groups, assuming the null hypothesis is true, is <0.001 or <0.1%. Statistical Inference

Presenting the results 1)Sample size 300 adults participating in a RCT comparing 2 dietary interventions 2)Estimate of the effect size Mean weight loss after 4 weeks for Atkins group compared to Weight watchers: 1.54 kg 3)Calculate a confidence interval 95% CI for difference in population means: 0.67 kg to 2.41 kg 4)Derive a p-value to test the hypothesis of no association P-value < Statistical Inference

Overview Defining your research question – PICOS Describing data Understanding the results –Estimates reported in the literature –Interpreting 95% confidence intervals and p- values ~ Statistical Inference