Critical Appraisal for MRCGP Jim McMorran Coventry GP GP trainer Editor GPnotebook (

Can all elephants be washed by children?

Analysing studies Confounding factors.. Sample size.. Applicability of results.. Placebo controlled.. Blinded… Primary endpoints…

Critical Appraisal Methodology (from GPnotebook) 1. what is the research question - for most studies there should be a single sentence describing why the study was done and what hypothesis the authors were testing 2. the type of study design used 3. the study population investigated 4. comparator drugs and doses studied 5. sample size and power used in the study 6. primary versus secondary endpoints and subgroup analysis 7. duration of the study 8. study follow-up 9. is there a sponsorship or publication bias

Critical Appraisal Methodology (from GPnotebook) 1. other considerations 1. intention to treat analysis 2. what p-values are used 3. what confidence intervals are used 4. comparing statistical and clinical significance of results 5. analysis of results e.g absolute risks, relative risks, odds ratios, hazard ratios, numbers needed to treat, numbers needed to harm, Kaplan-Meier survival curves, simple statistical test

Research question PROACTIVE (PROspective pioglitAzone Clinical Trial In macroVascular Events) to find out whether pioglitazone reduces cardiovascular (CV) morbidity and mortality in high-risk patients with type 2 diabetes

Type of Study Design Used Type of study: retrospective e.g. case control study prospective e.g. cohort study, randomised controlled trial Other points regarding study design include: was the study blinded? If so was the study blinded for the patient and the investigator (double blinded)? was the study a crossover study? The findings from a prospective study are more valid than those from a retrospective one

Different outcomes Different studies Coronary Heart Disease and HRT CHD and HRT - General Practice Notebook Management of febrile child – use of tepid sponging or not? prevention - General Practice Notebook

Study Population investigated Age of population Exclusion criteria used study population - General Practice Notebook

Study populations investigated Age when statins are effective Statins effective for primary prevention up to the age of 70 years WOSCOPS - General Practice Notebook Statins effective for primary prevention up to the age of 85 years MRC / BHF Heart Protection Study - General Practice Notebook Is the study population comparable to those patients in primary care Proactive study - General Practice Notebook FIELD study - General Practice Notebook

Comparitor drugs and doses studied Is there a comparitor Drug comparator drugs in statistical studies - General Practice Notebook comparator drugs in statistical studies - General Practice Notebook

Interesting comparisons Bupropion versus varenicline for smoking cessation varenicline - General Practice Notebook Is the indicated length of use of medication the same for both medications Exercise versus use of metformin in prevention of diabetes exercise in the prevention of type 2 diabetes - General Practice Notebook exercise in the prevention of type 2 diabetes - General Practice Notebook What about the dose of metformin

Sample Size and Power What is statistical power? What is the typical value of the power of the study? Why is the power of the study important when considering study endpoints? sample size and power of study - General Practice Notebook sample size and power of study - General Practice Notebook

Sample Size and Power

Power = 1 – type II error Higher Power Occurs with Less Overlap of Distributions

Primary Versus Secondary Endpoints The power of a study is based on the primary endpoint of a study primary versus secondary endpoints - General Practice Notebook primary versus secondary endpoints - General Practice Notebook Headline studies where the primary endpoint was not significant include Proactive study - General Practice Notebook FIELD study - General Practice Notebook

Duration of study Sufficient period to allow events to occur Long enough to relate findings to use in chronically ill patient duration of study - General Practice Notebook Efficacy of topical NSAIDs versus oral NSAIDs topical non-steroidal anti-inflammatory drugs (NSAIDs) in osteoarthritis - General Practice Notebook topical non-steroidal anti-inflammatory drugs (NSAIDs) in osteoarthritis - General Practice Notebook

Study follow-up Affects the power of the study study follow-up - General Practice Notebook

Is there sponsorship or publication bias Is the study sponsored by the pharmaceutical industry or government sponsored is there sponsorship or publication bias - General Practice Notebook is there sponsorship or publication bias - General Practice Notebook

Other considerations – a selection Intention to treat analysis intention to treat analysis - General Practice Notebook Comparing statistical and clinical significance statistical versus clinical significance - General Practice Notebook statistical versus clinical significance - General Practice Notebook NNT number needed to treat (NNT) - General Practice Notebook number needed to treat (NNT) - General Practice Notebook Consider how many patients are required to be treated to prevent a single stroke based on the SPARCL trial (look at CIs) SPARCL (Stroke Prevention by Aggressive Reduction in Cholesterol Levels) trial - General Practice Notebook SPARCL (Stroke Prevention by Aggressive Reduction in Cholesterol Levels) trial - General Practice Notebook

The End..