Critically Appraising a Medical Journal Article Bill Burton, Ph.D. Associate Director Office of Educational Resources AECOM

What Is Critical Appraisal? A technique for judging the methodological quality of a study. It allows you to … quickly exclude papers that are of too poor a quality to inform practice systematically evaluate those that “make the grade” in order to extract their salient points

Why Is Critical Appraisal a Necessary Skill for Physicians? 20,000 biomedical journals, publishing 6 million articles a year Most of this is of limited relevance and utility All studies have flaws You can’t always depend on a good review As physicians, you need to become “critical yet comprehending” readers of the medical literature

Critical Appraisal Is Part of Evidence-Based Medicine FIND (the best evidence) APPRAISE (its quality) APPLY (the findings to your practice)

How to Become a “Critical” Reader Don’t accept everything an author says at face value Pay special attention to the Methods section Continually ask yourself questions (and try to answer them) as you’re reading an article Do the numbers add up? Do terms like “random selection”, “double-blind”, and so on accurately reflect what was done?

Critical Appraisal Involves 3 Stages UNDERSTAND the article What was the question? What methods were used? What did the authors find? Determine its VALIDITY Were the study methods sound? Determine its UTILITY How useful (i.e., generalizable) are the findings to your practice and patients?

How Do You Decide if an Article Is… Worth Reading? Believable?

Use Your Knowledge of the Following to Evaluate an Article Study design Epidemiology Hypothesis testing Statistics

Review of Study Design Observational Studies Experimental Studies Case report/series Cross-sectional Case-control Cohort Experimental Studies

Other Terms/Concepts You Should Be Familiar With Null hypothesis (vs. research hypothesis) Sources of confounding and bias Statistical power P-value (vs. alpha level) Confidence interval Intention-to-treat analysis Sensitivity & specificity

1 - Understand the Article What is the research question? What study design was used? Who are the subjects and how were they selected? What did the intervention consist of? What are the measures, and how were they obtained? How were the data analyzed? What are the findings and conclusions?

2 - Determine the Article’s Validity (Internal Validity) Is the research question focused and clearly stated? Were the methods appropriate given the question? Did the investigators obtain a representative sample of the population? Did they have a large enough sample? Were the groups (e.g., intervention and control) similar at the start of the study? Were exposures/outcomes accurately measured to minimize bias? Have confounding factors been accounted for in the design and/or analysis? Are the conclusions validly based on the data and analysis?

3 - Determine the Article’s Utility (External Validity) Is the question a clinically important one? Are the subjects similar to the patients you see? Were all important outcomes considered (e.g., potential harms as well as benefits)? The difference between the treatment and control groups was found to be statistically significant, but is it clinically significant? Could you apply the same treatment or intervention to your patients?

You Need to Choose an Article That Is … An original report of a research study (do not choose a review article) From a professional journal On a topic relevant to ambulatory care, and ideally dealing with an urban population An investigation of a specific hypothesis