1. Standards for Developing Trustworthy Clinical Practice Guidelines Standards for Developing Trustworthy Clinical Practice Guidelines Institute of Medicine January 11, 2010 Institute of Medicine January 11, 2010 Sandra Zelman Lewis, PhD Asst VP, Health & Science Policy American College of Chest Physicians Sandra Zelman Lewis, PhD Asst VP, Health & Science Policy American College of Chest Physicians

2. Most Important Challenge: Definition 1.Need for a universally accepted definition of evidence-based guidelines (EBGs)  The problem  Consensus statements and “even less structured” documents are often designated as “guidelines”  Physicians are misled into thinking that these are evidence-based and methodologically rigorous  Patient care can be adversely impacted

3. Most Important Challenge: Definition  What action should this committee take?  Set the bar high for methodological rigor  Require a minimal threshold of rigor or at least provide a rating scale that EBG users can understand  Do not allow consensus statements (and less evidence-based documents) to be titled “guidelines”

4. Most Important Challenge: Definition  Challenges  Within EBGs, should consensus-based recs be allowed? Reduce to areas of identified need for guidance Allow only when the evidence is weak, inconsistent, or nonexistent Downgrade appropriately Ensure panel: expertise, multidisciplinary, vetted for COIs Appropriate review and balanced viewpoints  Some guideline developers must adjust their processes Set a timetable to meet the new definition and standards Provide instruction on new standards and methods

5. Another Important Challenge: Funding 2.Funding guidelines is the biggest challenge that guidelines developers face.  The problem  EBGs are very expensive if done correctly  Little funding available, difficult to obtain  Charge to this committee  Identify sources for funding guidelines produced according to the established standards

6. When Evidence is Nonexistent or Poor Quality o What does ACCP do when the scientific evidence is absent or poor? Sets minimal threshold for evidence: Must be published in a peer-reviewed journal Downgrade recs when poor quality evidence Allow consideration of indirect evidence if justification (described in text) acceptable but downgrade appropriately If evidence is not sufficient, then discuss in text but do not provide recommendation

7. Disagreements in Interpretations of Evidence o How do you reconcile disagreements in evidence interpretation among guidelines? Assess rigor of methodology/adherence to the evidence Request invited associations to review and comment on our guideline recommendations Offer competing guideline organizations opportunity to appoint a member to our next edition or update panel (providing he/she passes COI vetting and approval process)

8. Accommodating Subgroups o How do guidelines accommodate subgroups (ex. older populations or persons with multi-morbidities) whose treatment outcomes may differ from the average patient? If data exist for subgroups, use it. If not, use indirect evidence from similar groups and downgrade appropriately Always describe patient population in each recommendation based on the patient population in the original studies A major challenge is multiple co-morbidities

9. Setting Standards o What topics and/or processes do you think the committee should consider in deriving quality standards for clinical practice guidelines? Establish a definition of evidence-based guidelines, possibly with rating scales Standard setting topics and criteria.PCPI (Physician Consortium for Performance Improvement) criteria for guidelines permitted as basis of performance measures.AGREE instrument (AGREE III is in development).COGS (Conference on Guideline Standardization).GLIA – for implementation purposes

10. Panel Composition o What should the composition of CPG development panels, in particular the balance of methodologists, topical experts, and consumers, look like? Depends on structure of panel but should have methodologists reviewing evidence and developing evidence tables or profiles Consumers must be educated in EBM Content experts provide credibility but all must go through rigorous review, including COIs Other considerations: health economists, frontline clinicians, patient preference consultants

11. Grading System o Is there an available assessment tool that adequately rates both the level of the scientific evidence and strength of clinical recommendations that should be used as standard practice in guideline development? The ACCP Grading System.A modification of GRADE (major difference: restricts evidence to approved threshold).Based on (1) assessment of the quality of the original studies and (2) a balance of the risks to benefits.Has been adopted by other guideline developers

12. Grading System Grade of rec Benefit versus risk and burdens Methodologic quality of supporting evidence 1A Benefits clearly outweigh risks/burdens, or vice versa (negative recs) Consistent evidence from randomized controlled trials without important limitations, or exceptionally strong evidence from observational studies 1B Benefits clearly outweigh risks/burdens, or vice versa (negative recs) Evidence from randomized controlled trials with important limitations (inconsistent results, methodologic flaws, indirect or imprecise), or very strong evidence from observational studies 1C Benefits clearly outweigh risks/burdens, or vice versa (negative recs) Evidence for at least one critical outcome from observational studies, case series, or from randomized, controlled trials with serious flaws or indirect evidence 2A Benefits closely balanced with risks/burdens Consistent evidence from randomized controlled trials without important limitations, or exceptionally strong evidence from observational studies 2B Benefits closely balanced with risks/burdens Evidence from randomized, controlled trials with important limitations (inconsistent results, methodologic flaws, indirect or imprecise), or very strong evidence from observational studies 2C Benefits closely balanced with risks/burdens Evidence for at least one critical outcome from observational studies, case series, or from randomized, controlled trials with serious flaws or indirect evidence

13. Recommendations for PMs o What methods might be developed for determining which recommendations among those in a guideline should be applied to quality measures or electronic medical record decision prompts? Evidence should dictate direction and strength of recs Suggest some 1A and 1B recs for PMs (although not all) 1C, 2A, 2B, and 2C recs should generally not be used However, all should be evaluated based on feasibility, usability, scientific importance, practicality, and applicability

14. QI and Harmonization o What administrative (eg, accreditation) or legal approaches might improve the quality of clinical practice guidelines? Published guideline quality rating scale Listed on NGC Web site Currency rating also listed in NGC PMs & CMS policies should be based on highly rated EBGs o What explicit approaches might harmonize guideline developers and increase guidelines convergence? Funding requires multiple societies to collaborate Require compliance with evidence-based standards, rating scale

15. EBG Promotion o What types of strategies might promote greater utilization of guidelines? Requirements for implementation into:.EMRs.Registries.PMs, including PQRI Education to allow developers to learn about new techniques and processes (see last slide) Healthcare providers need to know how to find good guidelines and good guidelines should address the needs of providers (ask frontline clinicians what they need!)

16. Other Considerations o Are there other characteristics of guideline standards you think are important for the committee to consider? Most important: address funding Also should move the field toward incorporating resource considerations into the EBG recs

17. Promotion of IOM Standards The IOM report and new standards should be presented to appropriate audiences: Guideline developers will be attending the Guidelines International Network conference and guideline methodology course

18. Guidelines International Network (G-I-N) 2010 Conference - Chicago August 26-28, 2010 – Conference Dates August 25, 2010 – Pre-meeting Course in Guideline Methodology Chicago, Illinois, USA www.GIN2010.org Host: American College of Chest Physicians