Reading and evaluating the scientific and medical literature Robert Silbergleit, MD Department of Emergency Medicine

Goals Reading Philosophy Habit Strategies Evaluating Question Design Other Bias Statistics

Philosophy The scientific literature is a discussion

Philosophy Data are plentiful but truth is elusive

Habit You must read.

Habit You must read. There is too much to read.

“It’s all right, sweetie. In the information age, everybody feels stupid.”

Strategies Strategies for reading –Grazing –Searching –Delivery

Strategies Information age strategic aids. –On-line journals –Weekly search agents Traditional strategic aids. –Review articles –Lectures –Journal clubs

Strategies Putting it together- a clinician’s strategy –NEJM (or JAMA or BMJ) –Clinical specialty journal (or 2) –Special interest or subspecialty journal –Abstract service

Strategies A research scientist’s strategy –Science (or Nature) –A research focus journal (or 2) –A weekly literature search agent

Strategies Pitfalls –Intimidation –Education that is not education –Honor and beware the opinion leaders

Strategies Strategies for skimming –Table of contents –Abstract (don’t believe it) –Figures (or Results text) –Methods –Intro and discussion

Abstracts were found to contain errors or inaccurately represent the article 48% of the time, and ranged from 18% in one journal to 68% in another. JAMA 1999;281:1110

Evaluating Assessing quality Being critical Determining value

Question Did they ask the right question? Did they answer the right question? Is the outcome they measured the one they are interested in?

Question What makes a good outcome measure? –Meaningful –Accurate (Gold standard) –Objective –Distinct

Question Is the correct population studied, or the appropriate animal model used? –Referral bias –Species differences

Design Randomized and controlled Observational (cohort, case-control)

Bias In science we mean: Any systematic uncontrolled influence on data that favors a particular result.

Bias But sometimes it really is the lay definition: A preference or inclination that inhibits impartial judgment.

Bias Publication bias Spectrum bias

Statistics Clinical v. statistical “significance” Hypothesis testing and the p-value Multiple comparisons Regression and confounding

Take home points Science is a discussion Make scientific reading a habit Be skeptical, don’t be cynical