How to Teach Statistics in EBM Rafael Perera

Basic teaching advice Know your audience Know your audience! Create a knowledge gap Give a map of the main concepts Decide which ones to focus on Use plenty of examples Let them do the work/thinking

Main Concepts Bias and Measurement error P values and Confidence Intervals Which Statistical tests are needed and when Correlation and Association Models / Regression and alternatives for Adjustment Survival Analysis Meta Analysis Statistics for Diagnostic Studies

There is a time and place…

Fundamental Equation of Error Measure = Truth + Bias + Random Error Use good study design Use large numbers Researcher Critically Appraise Design Confidence Intervals and P-values Reader

true result Bias low high Random error high low Bias versus Random error

Bias and Measurement error Groups of 3-4 people 1 – subject 2 – measurers Measurers – measure (twice) and record the head size of the subject. Keep measurements hidden.

Bias and Measurement error Intra-Observer variability Measurement error Same answer Varied by < 0.5 cm Varied by < 1cm Varied by < 2 cm Varied by >2 cm

Bias and Measurement error Inter-Observer variability Measurement error Same answer Varied by < 0.5 cm Varied by < 1cm Varied by < 2 cm Varied by >2 cm

Bias and Measurement error Bias Included ears? Included nose? Which part of the head? Other?

Does it matter? In paediatric practice following meningitis, a head circumference that increases by 7mm in a day will result in urgent head imaging In obstetrics measurements of the fundal height can vary by up to 5cm (the difference between having a baby delivered early due to IUGR or not when opposite occur) The question is can you reproduce the test in your setting and will it perform as well in your setting

Measuring Random error Most things don’t work!

Two methods of assessing the role of random error P-values (Hypothesis Testing) –use statistical test to examine the ‘null’ hypothesis –if p<0.05 then result is statistically significant Confidence Intervals (Estimation) –estimates the range of values that is likely to include the true value Relationship between p-values and confidence intervals If the ‘no effect’ value falls outside the CI then the result is statistically significant

The Steps in Testing a Hypothesis State the null hypothesis H 0 Choose the test statistic that summarizes the data Based on H 0 calculate the probability of getting the value of the test statistic Interpret the P-value

Some Statistical tests Comparing groups –T-tests (1 or 2 groups, normally distributed) –Chi-squared (2 or more groups, categorical or binary data) –Mann-Whitney U (2 groups, non-normal data) –Log-rank test (2 groups, survival data) –ANOVA (multiple groups, normally distributed) –…–… Tips: –Understand what the hypothesis being tested is –Use the p-value to assess the level of evidence against it –(Experienced) Assess if the test was adequate for the question and data analysed

Hand outs 1.Incidence/ Prevalence and CI 2.Survival analysis 3.Regression models / Adjustment 4.Linear association / Correlation 5.Confounding / Odds Ratios / Logistic Regression 6.Diagnostic Tests 7.Meta-analysis

Reading confidence intervals

Clinically significant Vitamin X shortens a 5 day cold Would you take it twice per day if it shortened the cold by:

Clinically significant Vitamin X shortens a 5 day cold Would you take it twice per day if it shortened the cold by: 50%

Clinically significant Vitamin X shortens a 5 day cold Would you take it twice per day if it shortened the cold by: 50% 20%

Clinically significant Vitamin X shortens a 5 day cold Would you take it twice per day if it shortened the cold by: 50% 20% 10%

Clinically significant Vitamin X shortens a 5 day cold Would you take it twice per day if it shortened the cold by: 50% 20% 10% 5%

Clinically significant Vitamin X shortens a 5 day cold Would you take it twice per day if it shortened the cold by: 50% 20% 10% 5% 1%

(a)(b) (c)(d) Minimum clinical Important difference No difference Which are clinically significant? 01020

Thank you

EXTRAS

Different types of measurements use different types of statistics Dichotomous:  –Male,female OR infected, non-infected Categorical:  –Red, green, blue OR Ordinal:  –Nil, +, ++ of glucose Interval: –temperature STATISTICS Proportion, Risk Mode, Proportions Mode, Median? Mean, Median

Between proportions for categorical data Flowchart of Statistical Tests for Hypothesis Testing

Between distributions Between one observed variable and a theoretical distribution Independence between two or more variables  2 test for goodness of fit  2 test for independence McNemar’s test for related groups

Flowchart of Statistical Tests for Hypothesis Testing Between means for continuous data Two samples t-test independent samples Rank sum test for independent samples Sign test for related samples t-test difference for related samples ANOVA Kruskal – Wallis Parametric > two samples Non Parametric Parametric Non Parametric

Flowchart of Statistical Tests for Hypothesis Testing Between proportions for categorical data One sample vs. H0 Two samples Z-score Z-score equal proportions Summarising proportions One sample: Risk, Odds Two samples: Relative risk, Odds ratios, Risk differences