Evidence-Based Medicine Critical Appraisal of Therapy Department of Medicine - Residency Training Program Tuesdays, 9:30 a.m. - 12:00 p.m., UW Health Sciences Library

Strategies for Critical Appraisal of Studies on Therapy ? Clinical Importance Validity Applicability

Strategies for Critical Appraisal of Studies on Therapy ? Validity

Judging validity with just 6 questions! 1. Randomized trial with “concealment”. * 2. Follow-up of patients sufficiently long and complete (> 80% follow-up). 3. Intention-to-treat analysis based on initial assignment. 4. Adequate blinding of treatment when possible. 5. Groups treated equally. 6. Groups similar at the start.

No Randomized Trials or Published Studies Fall Short? Try repeating your search. Is the treatment effect so huge in other published studies you can’t ignore it. If non-randomized trial says a treatment is worthless it probably is. Consider an “n-of-1” trial. Consider other treatment options. See textbook for details.

Strategies for Critical Appraisal of Studies on Therapy ? Clinical Importance

Judging clinical importance with just 2 questions! 1. What is the magnitude of the treatment effect? RRR = (Control ER - Experimental ER)/Control ER ARR = Control ER - Experimental ER NNT = 1/ARR (Conversely NNH = 1/ARI) 2. How precise is this estimate of the treatment effect? 95% CI - range of values within which we can be 95% sure that the population value lies.

Calculating NNT/NNH 1. A randomized trial of new drug “Ligatite” reveals that 25% of World Cup skiers who take the drug for one year have ACL tears whereas 50% of World Cup skiers who take the placebo for the year have ACL tears. What is the NNT? NNT = 1/ARR = 1/( ) = 4 2. An advertisement for a new drug designed to prevent pheochromocytoma highlights a relative risk reduction over 5 years of 90%. You read the study supporting the claim and find the study valid. What is the NNT? Unknown without knowing the event rate in the control population. 3. The study of the drug “Ligatite” also notes that 5% of athletes taking the drug develop clinical depression whereas 3% of athletes taking the placebo develop depression. What is the NNH? NNH = 1/ARI = 1/( ) = 50

Examples of NNT

Strategies for Critical Appraisal of Studies on Therapy ? Applicability

Applicable to Our Patient? 1. Is our patient so different from those in the study that its results cannot apply? 2. Is the treatment feasible in the setting? 3. What are our patient’s potential benefits and harms from the therapy? 4. What are our patient’s values and expectations for both the outcome and the treatment? See textbook for details of LHH (likelihood of being helped vs. harmed)

Estimating Our Patient’s Expected Event Rates (PEER) 1. Assign our patient the overall control event rate from the study. 2. If there is a subgroup of patients in the study with similar characteristics assign the event rate for that subgroup. 3. If a validated clinical predication guide is available use it to assign an event rate. 4. Look for a different paper that describes the prognosis of untreated patients more similar to our patient and use its results to assign an event rate.

Returning to “Ligatite” A randomized trial of new drug “Ligatite” reveals that 25% of World Cup skiers who take the drug for one year have ACL tears whereas 50% of World Cup skiers who take the placebo for the year have ACL tears. Your patient reads about this in Ski Magazine and asks you to write a prescription. In discussing the medication with her you want to provide her with an estimate of the magnitude of risk reduction she would realize. Is her NNT = 4? No: 1. Her risk of an ACL tear is substantially less so you have to re-estimate her expected event rate. 2. She is unlikely to be skiing year round. So: Look to see if there are data estimating the seasonal risk of ACL tear in a population representing your patient and adjust the ARR in the trial to reflect 6 months of treatment.

Clinical Tools for Estimating PEER Available at:

Therapy Questions