Assessing Applicability Interactive Quiz Prepared for: The Agency for Healthcare Research and Quality (AHRQ) Training Modules for Systematic Reviews Methods Guide

Why might trials with run-in periods and high prerandomization exclusion rates negatively affect applicability? A.The adverse events seen in the intervention groups would likely be lower than in the clinical setting. B.The benefits seen in the intervention groups would likely be lower than in the clinical setting. C.Run-in periods should not negatively affect applicability. Effects of Run-in Periods and Prerandomization Exclusion Criteria

Why might the use of a composite end point (e.g., death, nonfatal myocardial infarction, ventricular fibrillation, stroke, or stable anginal pain) negatively affect applicability? A.Stroke is a surrogate end point. B.It is often difficult to determine the effect of an intervention on each component of the composite end point. C.Composite end points do not affect applicabilit y. Effects of Composite End Points

A study shows that the use of a new medication in male astronauts, 25 to 35 years of age, prevents bone mineral density loss during prolonged stays in outer space when compared with no therapy. Which factors would limit the applicability of this trial to patients with osteoporosis? A.Differences in gender and age between the study population and the general population B.Use of a surrogate end point C.Differences in setting between the study population and the general population D.All of the above Study Population

Why should grading of the strength of evidence and assessment of applicability be carried out separately? A.Strength of evidence grading and applicability assessment are essentially the same. B.Studies using strong methods can show that an intervention is superior to a control therapy in a study population, but it may or may not work in actual practice. C.Strength of evidence grading is primarily associated with assessing the external validity, whereas applicability assessment is concerned with internal validity. Strength of Evidence

TrialPICO & SComments Smith et al.. Heart failure population Mean age: 65 years NYHA class II or III: 83% Surgical debulking of myocardium Watchful waiting (ACE inhibitor use, 34%; ß- blocker use, 40%) Outcomes: hospitalizations and survival Median followup: 1 year Setting: single, large, tertiary care hospital An efficacy trial; limited standardization of intervention; comparator did not include optimal medical therapy; unclear how the benefits and harms would compare in a smaller community hospital Evidence Table Review the evidence table above. Which columns are completed in step 1 in an applicability assessment? A.Columns 1 through 5 B.Columns 1 through 6 C.Column 1 only ACE = angiotensin-converting enzyme inhibitor; NYHA = New York Heart Association; PICOS = population, intervention, comparators, outcomes, and setting

 The process used to assess applicability and to distinguish efficacy trials from effectiveness trials can have many pitfalls.  Using a three-step process to assess applicability can make the process easier.  Transparent reporting of results is an important component of comparative effectiveness reviews. Summary

 This quiz was prepared by C. Michael White, Pharm.D., FCP, FCCP, a member of the University of Connecticut/Hartford Hospital Evidence-based Practice Center.  This module is based on chapter 6 in version 1.0 of the Methods Reference Guide for Effectiveness and Comparative Effectiveness Reviews (available at: raft MethodsGuide.pdf). Author