Evidence-Based Medicine Critical Appraisal of Harm Department of Medicine - Residency Training Program Tuesdays, 9:30 a.m. - 12:00 p.m., UW Health Sciences Library

Steps in Practicing EBM 1.Convert the need for information into an answerable question. 2.Track down the best evidence with which to answer that question. 3.Critically appraise the evidence for its validity, impact, and applicability. 4.Integrate the evidence with our clinical expertise and our patient’s characteristics and values.

Review Last Week’s Session

Steps in Practicing EBM 1.Convert the need for information into an answerable question. 2.Track down the best evidence with which to answer that question. 3.Critically appraise the evidence for its validity, impact, and applicability. 4.Integrate the evidence with our clinical expertise and our patient’s characteristics and values.

The Answerable Question

Good questions are the backbone of practicing EBM. It takes practice to ask the well-formulated question.

Well-Built Clinical ?’s Directly relevant to the care of the patient and our knowledge deficit. Contains the following elements: –the patient or problem being addressed –the intervention or exposure being considered –the comparison intervention or exposure, when relevant –the clinical outcomes of interest.

Well Formulated ?’s Focus scarce learning time on evidence directly relevant to patient’s needs and our particular knowledge needs. Suggest high-yield search strategies. Suggest forms that useful answers might take. Help us to model life-long learning techniques for our colleagues and students. Are answerable and, thus, reinforce the satisfaction of finding evidence that makes us better, faster clinicians.

Harm Questions

Steps in Practicing EBM 1.Convert the need for information into an answerable question. 2.Track down the best evidence with which to answer that question. 3.Critically appraise the evidence for its validity, impact, and applicability. 4.Integrate the evidence with our clinical expertise and our patient’s characteristics and values.

General Resources META-SEARCH ENGINES PrimeAnswers TRIP+ SUMSearch SYSTEMATIC REVIEWS/META-ANALYSES Cochrane Library PubMed Clinical Queries EVIDENCE GUIDELINES/SUMMARIES AHRQ Evidence Reports Clinical Evidence AHRQ Preventive Services CLINICAL RESEARCH CRITIQUES ACP Journal Club Bandolier BestBETs CASE REPORTS/SERIES, PRACTICE GUIDELINES, ETC National Guideline Clearinghouse PubMed MeSH Headings

Steps in Practicing EBM 1.Convert the need for information into an answerable question. 2.Track down the best evidence with which to answer that question. 3.Critically appraise the evidence for its validity, impact, and applicability. 4.Integrate the evidence with our clinical expertise and our patient’s characteristics and values.

Strategies for Critical Appraisal of Studies of Harm Clinical Importance Validity Applicability

Strategies for Critical Appraisal of Studies of Harm Validity

Judging validity with just 4 questions! 1. Did investigators assemble clearly defined groups of patients similar in all important ways other than exposure? 2. Were exposures and outcomes measured in the same ways in both groups (objective/blinded)? 3. Was follow-up sufficiently long and complete (5% and 20% rule)? 4. Do the results of the harm study fulfill some of the tests for “causation”?

Types of Studies (in order of decreasing likelihood of being valid) Systematic reviews are ideal because individual RCTs seldom large enough to detect rare adverse events with precision - unfortunately, SR are uncommon. RCTs are difficult to conduct for most studies of harm. Cohort studies - exposed and unexposed followed for development of outcome of interest. Case-control studies - cases with outcome of interest compared with controls for “exposure”. Cross-sectional studies. Case reports. * *

Criteria for Inferring Causality Is it clear that the exposure preceded the onset of the outcome? Is there a dose-response relationship? Any positive evidence from a dechallenge- rechallenge study? Is the association consistent across studies? Does the association have biological plausability?

Strategies for Critical Appraisal of Studies of Harm Clinical Importance

Judging clinical importance with just 2 questions! 1. What is the magnitude of the treatment effect? RR = (Exposed ER - Unexposed ER)/Unexposed ER AR difference = Exposed ER - Unexposed ER NNH = 1/AR difference 2. How precise is this estimate of the treatment effect? 95% CI - range of values within which we can be 95% sure that the population value lies.

Calculating NNT/NNH 1. A randomized trial of new drug “Ligatite” reveals that 25% of World Cup skiers who take the drug for one year have ACL tears whereas 50% of World Cup skiers who take the placebo for the year have ACL tears. What is the NNT? NNT = 1/AR reduction = 1/( ) = 4 3. An advertisement for a new drug fails to mention that it increases the relative risk of myocardial infarction by 50 % over 5 years. You read a valid study describing this finding. What is the NNH? Unknown without knowing the event rate in the control population. 2. The study of the drug “Ligatite” also notes that 20% of athletes taking the drug develop clinical depression whereas 10% of athletes taking the placebo develop depression. What is the NNH? NNH = 1/AR increase = 1/( ) = 10

The Odds Ratio Used as an estimate of the risk ratio if the risk of the disease in a population is low. Is the principle measure of effect from case- control studies (cannot calculate event rates). Also used to report effect size in meta-analysis. Odds of exposure in the disease group divided by odds of exposure in non-diseased group. Disease PresentAbsent Risk Factor Presentab Absentcd OR = (a/c)/(b/d) = ad/cb

Converting OR to NNH Calculator available at: For OR greater than PEER

Avoiding TIV (table induced vertigo) OR’s greater than 1.5 produce NNH < 50 across most PEER’s Patient needs to be at risk (non-trivial PEER) in order to be concerned. for any OR, NNH greatest when PEER=0.5 Consider carefully nature of harm (are your patient’s values disrupted by the intervention and its sequelae)

Estimating Our Patient’s Expected Event Rates (PEER) 1. Assign our patient the overall control event rate from the study. 2. If there is a subgroup of patients in the study with similar characteristics assign the event rate for that subgroup. 3. If a validated clinical predication guide is available use it to assign an event rate. 4. Look for a different paper that describes the prognosis of untreated patients more similar to our patient and use its results to assign an event rate.

Clinical Tools for Estimating PEER Available at:

Strategies for Critical Appraisal of Studies of Harm Applicability

Applicable to Our Patient? 1. Is our patient so different from those in the study that its results cannot apply? 2. What is our patient’s risk of benefit and harm from agent? 3. What are our patient’s preferences, concerns, and expectations from this treatment? 4. What alternative treatments are available?

Returning to “Ligatite” The trial of the drug revealed that 25% of World Cup skiers who take the drug for 1 year have ACL tears whereas 50% of skiers who take the placebo have ACL tears. It also revealed that 20% of exposed skiers developed depression whereas 10% of unexposed skiers developed depression. Your patient reads about this in Ski Magazine and asks you to write a prescription. In discussing the medication with her you want to provide her with an estimate of the magnitude of risk reduction she would realize. Is her NNT = 4 and should she take this medication? Probably Not: 1. Her risk of an ACL tear is substantially less so you have to re-estimate her expected event rate. 2. She is unlikely to be skiing year round so NNT is at least 2 to 3 times as high. 3. There is risk of developing depression (NNH = 10 over 1 year). 4. Whether or not to take the drug should take into account the relative value to the patient of preventing an ACL tear faced with the probability of developing depression.

Harm Questions