1. Critically Appraising a Medical Journal Article
Bill Burton, Ph.D.
Associate Director
Office of Educational Resources
AECOM

2. What Is Critical Appraisal?
A technique for judging the methodological quality of a study. It allows you to …
quickly exclude papers that are of too poor a quality to inform practice
systematically evaluate those that “make the grade” in order to extract their salient points

3. Why Is Critical Appraisal a Necessary Skill for Physicians?
20,000 biomedical journals, publishing 6 million articles a year
Most of this is of limited relevance and utility
All studies have flaws
You can’t always depend on a good review
As physicians, you need to become “critical yet comprehending” readers of the medical literature

4. Critical Appraisal Is Part of Evidence-Based Medicine
FIND
(the best evidence)
APPRAISE
(its quality)
APPLY
(the findings
to your practice)

5. How to Become a “Critical” Reader
Don’t accept everything an author says at face value
Pay special attention to the Methods section
Continually ask yourself questions (and try to answer them) as you’re reading an article
Do the numbers add up?
Do terms like “random selection”, “double-blind”, and so on accurately reflect what was done?

6. Critical Appraisal Involves 3 Stages
UNDERSTAND the article
What was the question?
What methods were used?
What did the authors find?
Determine its VALIDITY
Were the study methods sound?
Determine its UTILITY
How useful (i.e., generalizable) are the findings to your practice and patients?

7. How Do You Decide if an Article Is…
Worth Reading?
Believable?

8. Use Your Knowledge of the Following to Evaluate an Article
Study design
Epidemiology
Hypothesis testing
Statistics

9. Review of Study Design Observational Studies Experimental Studies
Case report/series
Cross-sectional
Case-control
Cohort
Experimental Studies

10. Other Terms/Concepts You Should Be Familiar With
Null hypothesis (vs. research hypothesis)
Sources of confounding and bias
Statistical power
P-value (vs. alpha level)
Confidence interval
Intention-to-treat analysis
Sensitivity & specificity

11. 1 - Understand the Article
What is the research question?
What study design was used?
Who are the subjects and how were they selected?
What did the intervention consist of?
What are the measures, and how were they obtained?
How were the data analyzed?
What are the findings and conclusions?

12. 2 - Determine the Article’s Validity (Internal Validity)
Is the research question focused and clearly stated?
Were the methods appropriate given the question?
Did the investigators obtain a representative sample of the population? Did they have a large enough sample?
Were the groups (e.g., intervention and control) similar at the start of the study?
Were exposures/outcomes accurately measured to minimize bias?
Have confounding factors been accounted for in the design and/or analysis?
Are the conclusions validly based on the data and analysis?

13. 3 - Determine the Article’s Utility (External Validity)
Is the question a clinically important one?
Are the subjects similar to the patients you see?
Were all important outcomes considered (e.g., potential harms as well as benefits)?
The difference between the treatment and control groups was found to be statistically significant, but is it clinically significant?
Could you apply the same treatment or intervention to your patients?

14. You Need to Choose an Article That Is …
An original report of a research study (do not choose a review article)
From a professional journal
On a topic relevant to ambulatory care, and ideally dealing with an urban population
An investigation of a specific hypothesis