Making Guidelines Actionable Richard Rosenfeld & Richard Shiffman E-GAPPS Breakout Session NY Academy of Medicine 12/12

Standards for Developing Trustworthy Clinical Practice Guidelines Updated IOM Definition of Clinical Practice Guidelines Guidelines are statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options

AAO-HNS Clinical Practice Guideline Development Process

Clinical Practice Guideline Development: A Quality- Driven Approach for Translating Evidence into Action Pragmatic, transparent approach to creating guidelines for performance assessment Pragmatic, transparent approach to creating guidelines for performance assessment Evidence-based, multidisciplinary process leading to publication in months Evidence-based, multidisciplinary process leading to publication in months Emphasizes a focused set of key action statements to promote quality improvement Emphasizes a focused set of key action statements to promote quality improvement Uses evidence profiles to summarize decisions and value judgments in recommendations Uses evidence profiles to summarize decisions and value judgments in recommendations Rosenfeld & Shiffman, Otolaryngol HNS 2009 Otolaryngol Head Neck Surg 2009; 140(Suppl):S1-43

Two Approaches to Evidence and Guidelines Evidence as Protagonist Model Development is driven by the literature search, which takes center stage with exhaustive evidence tables or textual discussions that rank and summarize citations. Product is a Practice Parameter, Evidence Report, or Evidence-Based Review Evidence as Supporting Cast Model Development is driven by a priori considerations of quality improvement, using the literature search as one of many factors that are used to translate evidence into action. Product is a Guideline with Actionable Statements

Generating Topics for Action Statements Ask “If we could only discuss a few aspects of this condition, what topics would we focus on most to improve quality of care?” Ask “What should we focus on to minimize harm?” Consider high level evidence from systematic review and the concept list generating when discussion scope. Remember: A quality-driven approach allows all important topics to be included, even if evidence is weak or limited. Action statements may still be possible based on the balance of benefit and harm. Rosenfeld & Shiffman, Otolaryngol HNS 2009 Developing key action statements begins with asking the group to suggest topics that are opportunities for quality improvement within the scope Otolaryngol Head Neck Surg 2009; 140(Suppl):S1-43

Ranked Topic List for Hoarseness Guideline

1. Promote appropriate care 2. Reduce inappropriate or harmful care 3. Reduce variations in delivery of care 4. Improve access to care 5. Facilitate ethical care 6. Educate & empower clinicians & patients 7. Facilitate coordination & continuity of care 8. Improve knowledge base across disciplines Quality Improvement Opportunities Eden J, Wheatley B, McNeil B, Sox H (eds).Washington, DC: Nat’l Academies Press a.k.a. Potential topics for guideline action statements

Standards for Developing Trustworthy Clinical Practice Guidelines Standard 6. Articulation of Recommendations 6.1 Recommendations should be articulated in a standardized form detailing precisely: what the recommended action is, and under what circumstances it should be performed. 6.2 Strong recommendations should be worded so that compliance with the recommendation(s) can be evaluated.

Statements of Fact vs. Action Clinicians should not routinely administer or prescribe perioperative antibiotics to children undergoing tonsillectomy. Antibiotic therapy does not improve recovery after tonsillectomy The management of acute otitis externa should include an assessment of pain. The clinician should recommend analgesic treatment based on the severity of pain. Acute otitis externa (swimmer’s ear) is associated with moderate to severe pain. Clinicians should advocate for voice therapy for patients diagnosed with hoarseness (dysphonia) that reduces voice-related quality of life. Voice therapy has been shown to improve quality of life for patients with hoarseness (dysphonia). Clinicians should use pneumatic otoscopy as the primary diagnostic method for otitis media with effusion. Pneumatic otoscopy is the most accurate test for otitis media with effusion. Statement of ActionStatement of Fact

Guidelines ARE NOT Review Articles! Guidelines contain key statements that are action-oriented prescriptions of specific behavior from a clinician Monitor Test GatherInterpretPerformDispose Action ConcludePrescribe Educate Document Procedure Consult Advocate Prepare Beware of the dreaded “Consider…”

Key Action Statements An ideal action statement describes: When (under what conditions) Who (specifically) Must, Should, or May (e.g., the level of obligation) do What (precisely) to Whom Anatomy of a Guideline Recommendation

Action Statement Profiles and Guideline Development 1.Encourage an explicit and transparent approach to guideline writing 2.Force guideline developers to discuss and document the decision making process 3.Create “organizational memory” to avoid re-discussing already agreed upon issues 4.Allow guideline users to rapidly understand how and why statements were developed 5.Facilitate identifying aspects of guideline best suited to performance assessment Key action statement with recommendation strength and justification Supporting text for key action statement Action statement profile:   Aggregate evidence quality:   Confidence in evidence:   Benefit:   Risk, harm, cost:   Benefit-harm assessment:   Value judgments:   Intentional vagueness:   Role of patient preferences:   Differences of opinion:   Exclusions:

1. Diagnosis of acute rhinosinusitis: Clinicians should distinguish presumed acute bacterial rhinosinusitis (ABRS) from acute rhinosinusitis caused by viral upper respiratory infections and non-infectious conditions. A clinician should diagnose ABRS when (a) symptoms or signs of acute rhinosinusitis are present 10 days or more beyond the onset of upper respiratory symptoms, or (b) symptoms or signs of acute rhinosinusitis worsen within 10 days after an initial improvement (double worsening). Strong recommendation based on diagnostic studies with minor limitations and a preponderance of benefit over harm. AAO-HNS Adult Sinusitis Clinical Practice Guideline Evidence profile (abbreviated):  Aggregate evidence quality: Grade B, diagnostic studies with minor limitations regarding signs and symptoms associated with ABRS  Benefits: decrease inappropriate use of antibiotics for non-bacterial illness; distinguish non-infectious conditions from rhinosinusitis  Harms: risk of misclassifying bacterial rhinosinusitis as viral, or vice-versa  Benefits-harm assessment: preponderance of benefit over harms  Value judgments: importance of avoiding inappropriate antibiotics for treatment of viral or non-bacterial illness; emphasis on clinical signs and symptoms for initial diagnosis; importance of avoiding unnecessary diagnostic tests Otolaryngol Head Neck Surg 2007; 137(Suppl):S1-S31

Classifying Recommendations for Practice Guidelines AAP Steering Committee on Quality Improvement and Management Pediatrics 2004; 114:

Action Statements as Behavior Constraints MAY SHOULD MUST or SHOULD Obligation level Be flexible in decision making regarding appropriate practice, although bounds may be set on alternatives Generally follow a recommendation, but remain alert to new information Follow unless a clear and compelling rationale for alternative approach exists Implication for clinicians Option Recommendation Strong recommendation Policy strength Cross-sectional survey of 1,332 registrants of the 2008 annual AHRQ conference given a clinical scenario with recommendations and asked to rate the level of obligation they believe the authors intended Lomotan E, et al. How “should” we write guideline recommendations? Interpretation of deontic terminology. Quality Safety Health Care 2009

Standards for Developing Trustworthy Clinical Practice Guidelines Standard 5. Recommendations For each recommendation provide: An explanation of the reasoning including: benefits, harms, evidence summary (quality, quantity, consistency), and the role of values, opinion and experience An explanation of the reasoning including: benefits, harms, evidence summary (quality, quantity, consistency), and the role of values, opinion and experience A rating of the level of confidence in (certainty regarding) the evidence A rating of the level of confidence in (certainty regarding) the evidence A rating of recommendation strength A rating of recommendation strength A description and explanation of any differences of opinion regarding the recommendation A description and explanation of any differences of opinion regarding the recommendation

Building Better Guidelines with BRIDGE-Wiz Description of a software assistant for structured action statement creation to promote clarity, transparency and implementability Shiffman…Rosenfeld et al, JAMIA 2012 J Am Med Inform Assoc 2012; 19: Choose an action type 2. Choose a verb 3. Define the object for the verb 4. Add actions 5. Check executability 6. Define conditions for the action 7. Check decidability 8. Describe benefits, risks, harms & costs 9. Judge the benefit-harms balance 10. Select aggregate evidence quality 11. Review proposed strength of recommendation and level of obligation 12. Define the actor 13. Choose recommendation style 14. Edit the final statement

8. Testing for allergy and immune function: Clinicians may obtain testing for allergy and immune function in evaluation a patient with chronic rhinosinusitis (CRS) or recurrent acute rhinosinusitis. Option based on observational studies with an unclear balance of benefit vs. harm. AAO-HNS Adult Sinusitis Clinical Practice Guideline Evidence profile:  Aggregate evidence quality: Grade C, observational studies  Benefits: identify allergies or immunodeficient states that are potential modifying factors for CRS or recurrent acute rhinosinusitis  Harms: procedural discomfort; instituting therapy based on test results with limited evidence of efficacy for CRS or recurrent acute rhinosinusitis; very rare chance of anaphylactic reactions during allergy testing  Cost: procedural and laboratory cost  Benefits-harm assessment: unclear balance of benefit vs. harm  Value judgments: need to balance detecting allergy in a population with high prevalence vs. limited evidence showing benefits of allergy management outcomes  Role of patient preferences: role for shared decision making Otolaryngol Head Neck Surg 2007; 137(Suppl):S1-S31

Classifying Recommendations for Practice Guidelines AAP Steering Committee on Quality Improvement and Management Pediatrics 2004; 114:

Clinicians and Options Evidence quality is suspect or well-designed studies have demonstrated little clear advantage to one approach vs. another Evidence quality is suspect or well-designed studies have demonstrated little clear advantage to one approach vs. another Options offer flexibility in decision making about appropriate practice, although they may set boundaries on alternatives Options offer flexibility in decision making about appropriate practice, although they may set boundaries on alternatives Hard to hold clinicians accountable (performance measures) Hard to hold clinicians accountable (performance measures) Patient preference should have a substantial role in influencing clinical decision making Patient preference should have a substantial role in influencing clinical decision making What Do They Mean?

…And Now It’s Your Turn…

Treatment & Prevention of the Common Cold Cochrane Systematic Reviews The Cochrane Library, 2010; John Wiley & Sons, Ltd Intervention (update)EvidenceConclusion Antibiotics (2009) 6 trials No benefits; more adverse events Non-steroidal anti- inflammatory drugs (2009) 9 trials Reduced headache, ear pain, muscle & joint pain; no effect on duration or adverse events Echinacea (2007) 16 trials Some early treatment benefit; no effect on prevention Heated, humidified air (2006) 6 trials Benefit for symptom relief in 3 studies; overall effects equivocal; minor discomfort, irritation, congestion Chinese medicinal herbs (2008) 17 trials Faster recovery 7 trials; no benefits in 10; problem with heterogeneity Vitamin C (2010) 29 trials Reduced duration and severity in prophylaxis trials (but not treatment trials); no benefit for prevention Garlic (2009) 1 trial Benefit for prevention in a single trial