Appraising Patient Management Recommendations: Clinical Practice Guidelines and Decision Analyses Updated for the third edition of the Users' Guides to the Medical Literature.

Objectives Be able to Distinguish practice recommendations from other secondary sources of evidence Evidence literacy Recognize steps in the development of clinical practice guidelines Apply Users’ Guides for patient management recommendations Interpret recommendation grading scales Evidence numeracy 2

Patient Five A’s of EBM Ask Act Acquire Apply Appraise EBM, evidence-based medicine. This slide returns to the evidence cycle first explained in the Education Guide “An Approach to Evidence-Based Medicine.” This Education Guide focuses on the “Appraise” step in the evidence cycle, as we appraise patient management recommendations and practice guidelines. Appraise

Clinical Practice Guidelines What are practice guidelines? How are guidelines developed? Assessing recommendations How should you use guidelines?

What Are Practice Guidelines? Users’ Guides definition A strategy for changing clinician behavior Systematically developed statements or recommendations to assist clinician and patient decisions about appropriate health care for specific clinical circumstances

What Are Practice Guidelines? Decision-making aid Decision guides, reasonable rules, advice, pathway, algorithm, comparisons of alternatives Statement of authority Official endorsement, statements of groups Instrument of control Costs, privileges, reimbursement

What Are Practice Guidelines? Set boundaries/limits to practice Clinical standards, appropriateness criteria, minimum acceptable practice, expected performance, practice parameters Synthesize evidence and expertise Comprehensive review of evidence, comprehensive analysis, experts assessment, consensus

Systematic Reviews vs Guidelines Unit of analysis = study Bias control = methods Synthesis = results Evidence Unit of analysis = problem Bias control = quality of evidence assessment Synthesis = decision Evidence and Values

Finding Guidelines Citation searching Electronic texts PubMed National Library of Medicine database Special markup since 1985 By publication type Use limits Publication type = practice guideline Electronic texts UpToDate and Dynamed

Finding Guidelines Repository searching Institutional policy collections Regional guideline collections National clearinghouses International registries

National Clearinghouses National Guidelines Clearinghouse Evidence-based clinical practice guidelines www.guideline.gov National Quality Measures Clearinghouse Evidence-based quality measures and measure sets www.qualitymeasures.ahrq.gov

National Clearinghouses National Patient Safety Network Clearinghouse of information about policies and practices affecting patient safety http://psnet.ahrq.gov National Institute for Clinical Excellence Evidence-based clinical practice guidelines Clinical practice and public health www.nice.org.uk

Outline What are practice guidelines? How are guidelines developed? Assessing recommendations How should you use guidelines?

Developing Recommendations Practice guidelines are ideally informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options To make a recommendation, guideline panelists ideally Define clinical questions Select the relevant outcome variables Retrieve and synthesize all of the relevant evidence Rate the confidence in the effect estimates Rely on a systematic approach and consensus to move from evidence to recommendations

Developing Recommendations In general, patient management recommendations are developed in context of clinical practice guidelines Clinicians may also find guidance originating from a decision analysis

Decision Analysis Systematic approach to decision making under conditions of uncertainty Involves Identifying all available alternatives and estimating the probabilities of potential outcomes associated with each alternative Valuing each outcome Arriving at a quantitative estimate of the relative merit of each alternative, on the basis of probabilities and values

Decision Trees Most clinical decision analyses are built as decision trees Decision analysis articles will usually include 1 or more diagrams showing the structure of the decision tree used for the analysis

Decision Tree: Example Abbreviation: LMWH, low-molecular-weight heparin. This figure shows a simplified decision tree for the scenario of the pregnant woman considering thromboprophylaxis. The patient has 2 options: to use or not use prophylaxis with LMWH. The decision is represented by a square, termed a “decision node.” The lines that emanate from the decision node represent the clinical strategies under consideration. Circles, called “chance nodes,” symbolize the different events that can occur after each clinical strategy. Patients may or may not develop a thrombotic or bleeding event, and the decision analysis requires estimates of the probability of both events. Triangles or rectangles identify outcome states. Learners can view this decision tree in more detail at http://jamaevidence.mhmedical.com/ViewLarge.aspx?figid=69034012&gbosContainerID=0&gbosid=0.

Outline What are practice guidelines? How are guidelines developed? Assessing recommendations How should you use guidelines?

Assessing Recommendations Is the clinical question clear and comprehensive? Were the recommendations based on current best evidence? Are values and preferences associated with the outcomes appropriately specified? Do the authors indicate the strength of their recommendations? Is the evidence supporting the recommendations easily understood? Was the influence of conflict of interests minimized?

Is the Question Clear? Is the recommended intervention clear and actionable? Is the alternative clear? Were all patient-important outcomes explicitly considered?

Patient-Important Outcomes Outcomes that patients value directly, in contrast to surrogate outcomes that clinicians may consider important (eg, lipid levels, bone density) Patient-important outcomes include mortality and morbidity (eg, hospital admission, acute exacerbation of a chronic disease), as well as outcomes such as quality of life and functional status

Recommendations and Evidence Ideally, recommendations are based on current or updated systematic reviews, preferably with meta-analyses Clinicians should check the date of the literature review and look for a description of the process used to identify and summarize the evidence and judge to what extent the process is credible Recommendations that do not use best current evidence risk promoting suboptimal or even harmful care

Recommendations and Evidence Systematic review: Identification, selection, appraisal, and summary of primary studies that address a focused clinical question using methods to reduce the likelihood of bias Meta-analysis: Statistical technique for quantitatively combining the results of multiple studies that measure the same outcome into a single pooled or summary estimate See also, The Process of a Systematic Review and Meta-analysis, at http://jamaevidence.mhmedical.com/content.aspx?bookid=847&sectionid=69031500.

Considering Values and Preferences In treatment recommendations, values and preferences should be appropriately specified for each outcome Clinicians should look for explicit statements regarding the values and preferences used to inform the recommendation

Values and Preferences This term refers to the goals, expectations, predispositions, and beliefs that individuals have for certain decisions and their potential outcomes Incorporation of patient values and preferences in decision making is central to evidence-based medicine

Strength of Recommendations Trustworthy recommendations specify the strength of the recommendations and also a rating of confidence in effect estimates that support the recommendations (also known as quality of evidence) Sensitivity analyses are used to explore strength of conclusions that arise from a decision analysis

Sensitivity Analysis Any test of stability of conclusions of a health care evaluation over a range of probability estimates, value judgments, and assumptions about the structure of the decisions to be made May involve repeated evaluation of a decision model in which one or more of the parameters of interest are varied

Grades of Recommendations There are 3 commonly used approaches to grading recommendations GRADE (Grading of Recommendations Assessment, Development and Evaluation) American Heart Association (AHA) US Preventive Services Task Force (USPSTF)

Grades of Recommendations Directions and strength of recommendations Abbreviations: AHA, American Heart Association; GRADE, Grading of Recommendations Assessment, Development and Evaluation; USPSTF, US Preventive Services Task Force. The 3 grading systems included in this figure feature a rating for confidence in effect estimates. Confidence in the effect estimates represents the extent to which the estimates are sufficiently credible to support a particular recommendation, as shown in the figure. The GRADE approach specifies 4 levels of confidence: high, moderate, low, and very low. The AHA and USPSTF systems specify 3 levels of confidence: A, B, and C in the AHA approach and high, moderate, and low in the USPSTF approach. The 3 systems share another critical feature: they differentiate between recommendations that should be applied (or avoided) in all, or almost all, patients (ie, strong recommendations) from those that require individualization to the patient’s values, preferences, and circumstances (ie, weak recommendations).

Is Supporting Evidence Easily Understood? For strong recommendations, is the strength appropriate? For weak recommendations, does the information provided facilitate shared decision making?

Strong Recommendations Message to clinicians is “just do it” Recommendations that are inappropriately graded as strong may therefore have undesirable consequences

Strong Recommendations High confidence in effect estimates will support strong recommendation if Desirable consequences considerably outweigh undesirable ones There is reasonable confidence and limited variability in patients’ values and preferences Benefits of proposed course of action justify its cost

Weak Recommendations Weak recommendations in particular should explicitly provide key information necessary to act on recommendation In guidelines, typically found in Remarks section Recommendation rationale Tables that accompany recommendation GRADE Working Group and Cochrane Collaboration designed summary-of-findings table specifically for this purpose

Summary-of-Findings Table In a practice guideline developed according to the GRADE method, the summary-of-findings table provides confidence ratings for all important outcomes and associated estimates of relative and absolute effects Summary-of-findings tables can facilitate shared decision making Learners can view a summary-of-findings table relevant to an example used in Chapter 26 of the Users’ Guides to the Medical Literature at http://jamaevidence.mhmedical.com/ViewLarge.aspx?figid=69034063&gbosContainerID=0&gbosid=0.

Appraising Conflicts of Interest Judgments involved in interpretation of evidence and the decision on the final recommendation may be vulnerable to conflicts of interest

Conflicts of Interest Exist when investigators, authors, institutions, reviewers, or editors have financial or nonfinancial relationships with other persons or organizations (such as study sponsors) that may inappropriately influence their interpretation or actions Conflicts of interest can lead to biased design, conduct, analysis, and interpretation of study results and to bias in review articles and opinion pieces

Appraising Conflicts of Interest Clinicians can check the conflict of interest statements of guideline panelists or decisions analysts, usually found at the beginning or end of a publication or in a supplementary file Clinicians should also check what strategies were implemented to manage these conflicts of interest

Outline What are practice guidelines? How are guidelines developed? Assessing recommendations How should you use guidelines?

Strong Recommendations If panel’s assessment is astute, clinicians can apply strong recommendations to all or almost all patients in all or almost all circumstances without a review of the underlying evidence and without a detailed discussion with the patient Also true for decision analysis when utility of one alternative is substantially greater than the other and this relative utility is robust to sensitivity analyses It is important to note that there will always be idiosyncratic circumstances in which clinicians should not adhere to even strong recommendations. For examples of such circumstances, please refer to Chapter 26 of the Users’ Guides to the Medical Literature, http://jamaevidence.mhmedical.com/content.aspx?bookid=847&sectionid=69031506.

Weak Recommendations Typically sensitive to a patient’s values and preferences A shared decision-making approach that involves a discussion addressing potential benefits and harms is the optimal way to ensure decisions reflect best evidence and the patient’s values and preferences A solid understanding of the evidence is necessary for clinicians when using weak recommendations

Original slides created by Robert Hayward, MD, Centre for Health Evidence Updated by Gordon Guyatt, MD, Kate Pezalla, MA, and Annette Flanagin, RN, MA

Terms of Use: Users Guides to the Medical Literature Education Guides PowerPoint Usage Guidelines JAMAevidence users may display, download, or print out PowerPoint slides and images associated with the site for personal and educational use only. Educational use refers to classroom teaching, lectures, presentations, rounds, and other instructional activities, such as displaying, linking to, downloading, printing, and making and distributing multiple copies of said isolated materials in both print and electronic format. Users will only display, distribute, or otherwise make such PowerPoint slides and images from the applicable JAMAevidence materials available to students or other persons attending in-person presentations, lectures, rounds, or other similar instructional activities presented or given by User. Commercial use of the PowerPoint slides and images are not permitted under this agreement. Users may modify the content of downloaded PowerPoint slides only for educational (non-commercial) use; however, the source and attribution may not be modified. Users may not otherwise copy, print, transmit, rent, lend, sell, or modify any images from JAMAevidence or modify or remove any proprietary notices contained therein, or create derivative works based on materials therefrom. They also may not disseminate any portion of the applicable JAMAevidence site subscribed to hereunder through electronic means except as outlined above, including mail lists or electronic bulletin boards.