Critical Appraisal of an Article on Therapy

Why critical appraisal? Why therapy?

Where’s the Meat in Clinical Journals? Top 20 Journals Contributing to ACPJC in 1992 Journal# articles with abstract % articles meeting criteria N Engl J Med JAMA Ann Intern Med Lancet Arch Intern Med BMJ J Intern Med

Where’s the Meat in Clinical Journals? Top 20 Journals Contributing to ACPJC in 1992 (Cont’d) Journal# articles with abstract % articles meeting criteria Diabetes Care Circulation Am J Cardiol Chest Am Rev Respir Dis J Am Coll Cardiol Am J Med

Where’s the Meat in Clinical Journals? Top 20 Journals Contributing to ACPJC in 1992 (Cont’d) Journal# articles with abstract % articles meeting criteria Gastroenterology J Clin Epidemiol J Gen Intern Med Neurology Can Med Assoc J Spine

Guides to Critical Appraisal Users’ guides to the medical literature EBM working group JAMA

Users‘ Guides to the Medical Literature Are the results valid? What were the results? Will the results help me in caring for my patients?

Are the Results of the Study Valid? 1.Was the assignment of patients to treatment randomized?  Levels of evidence Primary guides

Are the Results of the Study Valid? 2.Were all patients who entered the trial properly accounted for and attributed at its conclusion? Was follow-up complete? Worst case scenario Were patients analyzed in the groups to which they were randomized? Intention-to-treat analysis Primary guides

Are the Results of the Study Valid? 3.Were patients, their clinicians, and study personnel ‘blind’ to treatment? Secondary guides

Are the Results of the Study Valid? 4.Were the groups similar at the start of the trial? Secondary guides

Are the Results of the Study Valid? 5.Aside from the experimental intervention, were the groups treated equally? Co-intervention? Contamination? Compliance? Secondary guides

Any questions? I Are the results of the study valid? Was the assignment of patients to treatment randomized? Were all patients who entered the trial properly accounted for and attributed at its conclusion? Were patients, their clinicians, and study personnel ‘blind’ to treatment? Were the groups similar at the start of the trial? Aside from the experimental intervention, were the groups treated equally?

What Were the Results? How large was the treatment effect? –Absolute risk reduction (ARR) –Relative risk reduction (RRR)

What Were the Results? 1.How large was the treatment effect? –Risk without therapy (baseline risk) 20/100 (20%) –Risk with therapy 15/100 (15%) Risk reduction (RR): 5% 25% ARR= 20%-15%RRR= (20%-15%)/20%

RRR Vs ARR Example 1 –Risk without therapy (baseline risk) 20/100 (20%) (0.2) –Risk with therapy 10/100 (10%) (0.1) RRR= (20%-10%)/20%=50% (0.5) ARR= 20%-10%= 10% (0.1)

RRR Vs ARR Example 2 –Risk without therapy (baseline risk) 2/1000 (0.2%) (0.002) –Risk with therapy 1/1000 (0.1%) (0.001) RRR= (0.2%-0.1%)/0.2%=50% (0.5) ARR= 0.2%-0.1%= 0.1% (0.001)

RRR Vs ARR Example 1Example 2 CER(20%) 0.2(0.2%) EER(10%) 0.1(0.1%) ARR(50%) 0.5 RRR(10%) 0.1(0.1%) 0.001

What Were the Results? 2.How precise was the estimate of the treatment effect? –Confidence intervals?  The 95% confidence interval (or 95% limits) would include 95% of the results from studies of the same size & design in the same population

What Were the Results? 1.How large was the treatment effect? 2.How precise was the estimate of the treatment effect? More questions?

Will the Results Help Me in Caring for My Patients? 1.Can the results be applied to my patient care? –Pts similar for demographics, severity, co- morbidity and other prognostic factors? –Compelling reason(s) why the results should not be applied?

Will the Results Help Me in Caring for My Patients? 2.Were all clinically important outcomes considered? –Are substitute endpoints valid?

Will the Results Help Me in Caring for My Patients? 3.Are the likely treatment benefits worth the potential harms and costs? –‘Number needed to treat’ (NNT) for different outcomes? NNT= 1 / ARR NNH= 1/ ARI

Number Needed to Treat (NNT) Example 1 –Risk without therapy (baseline risk) 20/100 (20%) (0.2) –Risk with therapy 10/100 (10%) (0.1) RRR= (20%-10%)/20%=50% (0.5) ARR= 20%-10%= 10% (0.1) NNT= 1/ 0.1= /10= 10

Number Needed to Treat (NNT) Example 2 –Risk without therapy (baseline risk) 2/1000 (0.2%) (0.002) –Risk with therapy 1/1000 (0.1%) (0.001) RRR= (0.2%-0.1%)/0.2%=50% (0.5) ARR= 0.2%-0.1%= 0.1% (0.001) NNT= 1/ 0.001= /0.1= 1000

Will the Results Help Me in Caring for My Patients? 3.Are the likely treatment benefits worth the potential harms and costs? –‘Number needed to treat’ (NNT) for different outcomes? NNT= 1 / ARR NNH= 1/ ARI

Aspirin and Primary Prevention Benefit CER=4.8% EER=4.2% ARR= =0.6% NNT=167 Harms IC bleeds: NS EC bleeds: ARI= =0.3% NNH=333

Will the Results Help Me in Caring for My Patients? 1.Can the results be applied to my patient care? 2.Were all clinically important outcomes considered? 3.Are the likely treatment benefits worth the potential harms and costs? More questions?

Users‘ Guides to the Medical Literature Are the results valid? What were the results? Will the results help me in caring for my patients?

Thank you …