1. Workshop on Standards for Clinical Practice Guidelines Institute of Medicine January 11, 2010 Vivian H. Coates, Vice President, ECRI Project Director, NGC and NQMC

2. National Guideline Clearinghouse (NGC) NGC and its sister National Quality Measures Clearinghouse (NQMC) sponsored by U.S. Agency for Healthcare Research and Quality (AHRQ) NGC and its sister National Quality Measures Clearinghouse (NQMC) sponsored by U.S. Agency for Healthcare Research and Quality (AHRQ) Mary Nix, Clearinghouse Administrator Mary Nix, Clearinghouse Administrator Since 1997, ECRI contractor to AHRQ to create and maintain NGC; since 2001, contractor for NQMC Since 1997, ECRI contractor to AHRQ to create and maintain NGC; since 2001, contractor for NQMC ECRI EPC also provides methodology support to federal and private sector CPG developers ECRI EPC also provides methodology support to federal and private sector CPG developers

3. NGC Perspective - Overview of all CPGs Maintain relationships with hundreds of guideline developers from many countries Maintain relationships with hundreds of guideline developers from many countries Produce structured abstracts of thousands of guidelines, taking note of underlying methodology used to develop each guideline Produce structured abstracts of thousands of guidelines, taking note of underlying methodology used to develop each guideline Author guideline syntheses – in depth comparisons of areas of agreement and difference across multiple guidelines on same topic Author guideline syntheses – in depth comparisons of areas of agreement and difference across multiple guidelines on same topic Maintain relational database and website (www.guideline.gov) Maintain relational database and website (www.guideline.gov)www.guideline.gov Currently, NGC contains more than 2500 guidelines from more than 200 developers, and has reviewed nearly 8000 guidelines Currently, NGC contains more than 2500 guidelines from more than 200 developers, and has reviewed nearly 8000 guidelines

4. NGC Perspective - Overview of all CPGs NGC template of attributes a widely used tool for understanding important guideline characteristics NGC template of attributes a widely used tool for understanding important guideline characteristics Facilitates understanding and critical appraisal of guideline quality and rigor Facilitates understanding and critical appraisal of guideline quality and rigor Important attributes include: guideline scope, methods used to collect and analyze evidence, formulate and validate recommendations, panel characteristics, financial and other disclosures Important attributes include: guideline scope, methods used to collect and analyze evidence, formulate and validate recommendations, panel characteristics, financial and other disclosures

5. NGC Perspective - Overview of all CPGs New in 2009: research project underway by ECRI’s NGC team to explore feasibility of using Guideline Elements Model (GEM) to process clinical guidelines to facilitate their use in clinical decision support (CDS) information systems at the point of care New in 2009: research project underway by ECRI’s NGC team to explore feasibility of using Guideline Elements Model (GEM) to process clinical guidelines to facilitate their use in clinical decision support (CDS) information systems at the point of care Many guideline recommendations statements are vague and not “actionable” (decidable and executable), and therefore cannot readily be implemented into CDS or electronic health record (EHR) systems Many guideline recommendations statements are vague and not “actionable” (decidable and executable), and therefore cannot readily be implemented into CDS or electronic health record (EHR) systems In general, the 2500 guidelines in NGC vary widely in methodologic rigor: In general, the 2500 guidelines in NGC vary widely in methodologic rigor: Comprehensive, transparent, well-documented Comprehensive, transparent, well-documented “Evidence-light”, poorly documented “Evidence-light”, poorly documented

6. Challenges for CPG Developers and Users Recommendation statements that are vague or ambiguous may be due to deficiencies in underlying evidence – Developer may not be able to formulate explicit recommendations. Recommendation statements that are vague or ambiguous may be due to deficiencies in underlying evidence – Developer may not be able to formulate explicit recommendations. Guidelines often conflict: differences in evidence interpretation, but also because of differences in approaches to evidence synthesis ( eg, narrative review vs full systematic review),different developer perspectives (eg. primary care vs specialty; health plan vs provider). NGC identifies at least 25 different conditions for which conflicting guidelines exist. Guidelines often conflict: differences in evidence interpretation, but also because of differences in approaches to evidence synthesis ( eg, narrative review vs full systematic review),different developer perspectives (eg. primary care vs specialty; health plan vs provider). NGC identifies at least 25 different conditions for which conflicting guidelines exist. Complex patients with multiple co-morbidities not well addressed by most guidelines. Complex patients with multiple co-morbidities not well addressed by most guidelines.

7. Other Challenges for CPG Developers Resource and time constraints Resource and time constraints many orgs cannot afford to undertake an indepth evidence synthesis (eg, using full GRADE approach) many orgs cannot afford to undertake an indepth evidence synthesis (eg, using full GRADE approach) updating existing guidelines: many developers unable to do this in a timely way updating existing guidelines: many developers unable to do this in a timely way Need for education of less sophisticated developers in how best to conduct evidence reviews Need for education of less sophisticated developers in how best to conduct evidence reviews Available funding sources may lead to actual bias or perception of bias Available funding sources may lead to actual bias or perception of bias Guideline committees may be subject to undue influence by individuals (“eminence-based medicine”) Guideline committees may be subject to undue influence by individuals (“eminence-based medicine”)

8. Additional Challenges for Users of Guidelines Represented in NGC Lack of disclosure in CPGs about financial and other conflicts of interest is common Lack of disclosure in CPGs about financial and other conflicts of interest is common Lack of documentation in CPG about the details of methodologic approaches used is common Lack of documentation in CPG about the details of methodologic approaches used is common Lack of any detail about composition of guideline panel is common Lack of any detail about composition of guideline panel is common Lack of awareness on part of guideline developers of what is needed by quality measure developers (eg, what data sources will document whether a guideline recommendation was used for a specific patient) Lack of awareness on part of guideline developers of what is needed by quality measure developers (eg, what data sources will document whether a guideline recommendation was used for a specific patient)

9. Additional Challenges for Users of Guidelines Represented in NGC Lack of awareness by CPG developers of how to facilitate uptake of their guidelines into clinical decision support systems (CDS) and EHRs Lack of awareness by CPG developers of how to facilitate uptake of their guidelines into clinical decision support systems (CDS) and EHRs Delay/long time frames for release/publication of many guidelines means that the evidence on which they are based is outdated before implementation phase begins Delay/long time frames for release/publication of many guidelines means that the evidence on which they are based is outdated before implementation phase begins

10. Suggestions for Composition of Panel Guidelines for conditions requiring multidisciplinary approaches should have multidisciplinary panel composition Guidelines for conditions requiring multidisciplinary approaches should have multidisciplinary panel composition Nonphysician perspectives (esp. nursing) often helpful Nonphysician perspectives (esp. nursing) often helpful Include individual trained in EBM from a patient advocacy perspective, rather than simply a consumer rep Include individual trained in EBM from a patient advocacy perspective, rather than simply a consumer rep Include a quality improvement expert, if QM are to be developed from the guideline Include a quality improvement expert, if QM are to be developed from the guideline Include a clinical informatics expert, if CPG recommendations are to be actionable and implementable in CDS and EHR systems Include a clinical informatics expert, if CPG recommendations are to be actionable and implementable in CDS and EHR systems

11. Other Issues to Consider: There is currently no consensus on what constitutes a “good” guideline. A guideline may recommend interventions and practices that are based on good evidence, but be poorly worded so that the recommendations are unclear or not implementable in a CDS. The reverse is also true. There is currently no consensus on what constitutes a “good” guideline. A guideline may recommend interventions and practices that are based on good evidence, but be poorly worded so that the recommendations are unclear or not implementable in a CDS. The reverse is also true. NGC identifies more than 40 attributes of guidelines. Guidelines for which key information is “Not stated” may contain evidence-based, implementable recommendations, but lack transparency about the underlying methodology and processes. NGC identifies more than 40 attributes of guidelines. Guidelines for which key information is “Not stated” may contain evidence-based, implementable recommendations, but lack transparency about the underlying methodology and processes.

12. Other Issues to Consider: Most of the developers whose guidelines are represented in NGC (158 of 204; 77%) use some sort of rating scheme to grade the underlying evidence and/or strength of the recommendations. Most of the developers whose guidelines are represented in NGC (158 of 204; 77%) use some sort of rating scheme to grade the underlying evidence and/or strength of the recommendations. Of these, 10 developers report using GRADE or modified GRADE Of these, 10 developers report using GRADE or modified GRADE 6 report using the USPSTF approach, either as is, or modified 6 report using the USPSTF approach, either as is, or modified The great majority (142 developers) do not identify the origin of their rating schemes, and appear to be using schemes unique to their organizations. The great majority (142 developers) do not identify the origin of their rating schemes, and appear to be using schemes unique to their organizations. A more consistent approach implemented by CPG developers would be very helpful. A more consistent approach implemented by CPG developers would be very helpful.