1. JAMA: Users’ guide to evidence-based medicine

2. Therapy Diagnosis Harm Prognosis
7/9/13 8/6/13 11/26/13 12/24/13

3. How to use an article about therapy or prevention

4. MKSAP 16 General Internal Medicine Question 92
A 75-year-old man is hospitalized with sepsis leading to multi-organ failure. A meeting with family members is convened to discuss goals of care for the patient. The treatment team, including infectious disease and critical care consultants, has indicated that the patient is deteriorating despite optimized therapy, and the prognosis is poor. The daughter brings an Internet printout of a trial of a new medication for sepsis. The abstract states “We gave drug ‘X’ to 100 consecutive patients with refractory sepsis in our five intensive care units located in the same geographic region. Eight percent were alive at 30 days.” Although drug “X” is marketed in the United States, it is not FDA-approved for treatment of sepsis. A quick literature search reveals no other studies of drug “X” in the treatment of sepsis.
Which of the following is the main reason that it is difficult to determine the effectiveness of drug “X” based on the published study?
No comparison group
Outcome assessment not blinded
Patients not randomly assigned to treatment
Small study size

5. The three questions Are the results of the study valid?
What were the results?
Will the results help me in caring for my patients?

6. Are the results valid? Primary guides:
Was the assignment of patients to treatments randomized?
Were all patients who entered the trial properly accounted for and attributed at its conclusion?
Was follow-up complete?
Were patients analyzed in the groups to which they were randomized?

7. Are the results valid? Secondary guides:
Were patients, health workers, and study personnel blind to treatment?
Were the groups similar at the start of the trial?
Aside from the experimental intervention, were the groups treated equally?

8. Randomization Clinical outcomes may result from
Underlying severity of illness
Presence of comorbid conditions
Known and unknown prognostic factors
Treatment effect
What if there are no randomized trials?
cohort > case control > case series
Did they adjust for confounding variables?

9. Follow-up
In positive trials, if the number of patients “lost to follow-up” is large, assume that:
All patients in the treatment arm did badly
All patients in the control arm did well
If the conclusion would change, the strength of inference is weakened

10. Intent-to-treat
As in routine practice, patients in randomized trials sometimes forget to take their medicine or even refuse their treatment altogether.
Non-compliant patients tend to fare worse, regardless on prognostic factors.
If the study attributes all patients to the group to which they were randomized, it is an intent-to-treat analysis

11. Primary guides of validity
Randomization
Complete follow-up
Intent-to-treat analysis

12. Blinding
If physicians and patients could not be blinded (e.g. surgery), then were those who assess clinical outcomes?

13. Group similarity
Randomization doesn’t always produce groups balanced for known prognostic factors
Magnitude is important, statistical significance of the difference is not
Look for adjustments, and reasons for adjusting

14. Equal treatment “Cointerventions”
interventions other than the treatment under study, differentially applied to the treatment and control groups
E.g. giving steroids for a COPD exacerbation in a study of a β2 agonist
Permissible cointerventions should be listed, and frequency of administration documented

15. Secondary guides of validity
Blinding
Group similarity
Equal treatment of all groups

16. What were the results? How large was the treatment effect?
How precise was the estimate of the treatment effect?

17. Magnitude of effect
In a study of 100 patients, 20% in the control group died, 15% of the treatment group died
Risk without therapy (Baseline risk):X 20/100=0.20 or 20%
Risk with therapy: Y /100=0.15 or 15%
Absolute Risk Reduction (Risk Difference): X – Y =0.05
Relative Risk: Y/X /0.20 = 0.75
Relative Risk Reduction (RRR): [1-0.75]x100=25%
[1-Y/X] x 100 or [(X-Y) / X] x [0.05/0.20]x100=25%
95% Confidence Interval for the RRR -38% to +59%

18. Precision of the estimate
Confidence interval helps interpret both positive and negative trials
The larger the sample, the narrower the interval

19. No CI for RRR? Examine the p-value Use the standard error (SE)
p=0.05 → lower bound of the 95% CI = 0
Use the standard error (SE)
upper and lower bounds of the 95% CI = X ± 2SE
Calculate the 95% CI yourself

20. Will the results help me in caring for my patients?
Can the results be applied to my patient care?
Were all clinically important outcomes considered?
Are the likely treatment benefits worth the potential harms and costs?

21. MKSAP 16 General Internal Medicine Question 127
A 50-year-old woman is evaluated for nonischemic cardiomyopathy. Her exercise tolerance is not limited. She takes an ACE inhibitor daily. She took a β-blocker briefly but discontinued because of fatigue. Results of the physical examination are normal.
The patient inquires whether she should receive drug “H”. Drug H was studied in 2000 patients ages 40 to 80 years (mean age 63 years) with New York Heart Association functional class III or IV heart failure. Patients were randomized to receive drug H or a placebo in addition to usual medications. Eighty percent of patients in the trial also took a β-blocker and 70% an ACE inhibitor. At the end of 3 years, patients taking drug H had a significantly reduced rate of a composite outcome of death or heart failure exacerbations. Approximately 5% of the patients taking drug H had serious adverse events, compared with 2% in the placebo group.
Which of the following is the main reason why this patient should not be treated with drug H?
her heart failure is too mild
she is too young
she should be treated with a β-blocker first
the drug’s adverse event rate is too high

22. Does it apply to my patients?
Any compelling reasons why it shouldn’t?
Beware of the subgroup analyses
Often not planned ahead of time
“Data mining” in negative studies
Can use them if:
The difference in effects is very large;
Analysis was specified before the study began;
There was a very small number of subgroups;
Results were replicated in other studies.

23. Were outcomes clinically important?
Statins improve lipid profiles Metformin lowers HgA1c Isosorbide mononitrate improves CO Flecainide supresses ventricular depolarizations Even when one outcome is clinically important, watch out for effects on others (e.g. mortality versus QoL)

24. MKSAP 16 General Internal Medicine Question 58
A physician is asked to advise the Pharmacy and Therapeutics Committee of the hospital regarding a new drug to prevent deep venous thrombosis (DVT), drug “Z.” The physician reviews a recent randomized controlled trial of 5000 patients that compared drug Z with drug C, which is commonly used and is on the hospital’s formulary. The following data are abstracted from the trial:
Drug DVT Cases
Drug Z (n = 2500) 25
Drug C (n = 2500) 50
Based on these data, how many patients need to be treated (number needed to treat, NNT) with drug Z, compared with drug C, to prevent one extra case of DVT? 1 2 25
100
167

25. Number needed to treat Always compare to NNH
Risk with drug C (baseline risk): X 50/2500=0.02 or 2%
Risk with drug Z (therapy): Y 25/2500=0.01 or 1%
Absolute Risk Reduction (Risk Difference): X – Y =0.01 or 1%
Relative Risk: Y/X /0.02 = 0.5 or 50%
Relative Risk Reduction (RRR):
[1-Y/X] x 100 or [(X-Y) / X] x [1-0.5]x100=50%
95% Confidence Interval for the RRR +19% to +68%
Number needed to treat: 1/ARR 1/0.01=100
Always compare to NNH
The NNT:

26. Remember the questions
Are the results of the study valid?
Primary: randomization, follow-up, intent
Secondary: blinding, group similarity and equality
What were the results?
Effect magnitude (A/RRR) and accuracy (CI)
Will they help me in caring for my patients?
Applicable, clinically important outcomes, NNT