Critical Appraisal of Clinical Practice Guidelines November 6, 2012 Mary H. Palmer, PhD, RN,C Helen W. & Thomas L. Umphlet Professor in Aging University of North Carolina at Chapel Hill

Outline Quiz Focused discussion of revised PICOT questions Powerpoint review of critical appraisal of clinical practice guidelines Brief report of homework critical appraisals, reporting relevance and noteworthiness Small group activity – Critical appraisal and recommendations re. CPGs

Need for Clinical Practice Guidelines Increased volume of research-based evidence is a barrier to better use of knowledge. Lack of time to search, appraise and apply knowledge in clinical decision-making. Some erroneously equate lack of evidence for treatment effectiveness as the treatment being ineffective. Information should be reliable, relevant, and readable. Source: Straus & Haynes, 2009 CMAJ 180(9), 942.

Developmental Approach to Clinical Practice Guideline Source: Morris et al. Orthopaedic Nursing, (5):

CPGs as Tools Definition: “systematically developed statements based on the best available evidence, including syntheses, make recommendations for specific clinical conditions across a broad array of clinical diagnoses and situations.” (Tricoci, et al, 2009). Reduce unnecessary variations in clinical practice designed with flexibility for individual patients who fall outside the scope of the guideline.

Clinical Practice Guidelines Outcomes meaningful to patients Inclusive in considering all reasonable options CPGs are a snapshot of evidence at given point in time – need updating Rating scheme – Quality and strength of studies – Level of evidence – Class or grade of recommendations

Accessing Guidelines  Googling “practice guideline” will yield numerous sources, including  National Guideline Clearinghouse  Centre for Health Evidence  Canadian Medical Association  Registered Nurses’ Association of Ontario (RNAO)  The term “practice guideline” can be used as a limit to define a publication type in PubMed  Guidelines International Network (G-I-N) has the world’s largest guideline library

Accessing Guidelines  Googling “practice guideline” will yield numerous sources, including  National Guideline Clearinghouse  Centre for Health Evidence  Canadian Medical Association  Registered Nurses’ Association of Ontario (RNAO)  The term “practice guideline” can be used as a limit to define a publication type in PubMed  Guidelines International Network (G-I-N) has the world’s largest guideline library

National Guideline Clearinghouse (NGC)  Developed in partnership with the American Medical Association and the American Association of Health Plans  Of all the guidelines sources, the NGC contains the most descriptive information about guidelines

National Library of Medicine (NLM) Gateway  Allows users to put in a search term that is then sent out to eight different NLM databases  Health Services/Health Technology Assessment Text (HSTAT)  Takes large guidelines, systematic reviews, and technology assessments and enables their texts to be searchable on the Internet

Question  Which of the following online databases is the most likely source of a CPG to guide a nurse’s care? a.CINAHL b.EBSCO c.PubMed d.PsycInfo

Answer  c. PubMed  Rationale: PubMed can be searched for CPGs by setting publication limits on the query to “practice guideline.”

Finding the Right Guideline  Locating and reviewing current guidelines on a particular subject is often overwhelming  CPG should specify information such as:  Who developed and funded it  Who was on the panel  How the guideline was developed  What dates the literature review covered

Finding the Right Guideline (cont’d)  Need to keep in mind that “one size does not fit all”  Assess their application to the right person at the right time and in the right way  Ask  What are the guideline recommendations?  Are the guideline recommendations valid?  How useful are the recommendations?

Reading Guidelines  Recommendations should be as unambiguous as possible  Consider the developers’ values  Should specify the process used to systematically search and review the evidence that underlies the guideline  Evidence should be graded using a recognized format  Recommendations themselves should be graded  Updates are important  Consider whether a particular guideline will help your patients

Evaluating Guidelines  Consider  Validity  Reliability and reproducibility  Clinical applicability  Clinical flexibility  Clarity  Documentation  Development by a multidisciplinary process  Plans for review

Evaluating Guidelines (cont’d)  Rapid critical appraisal (RCA) checklist – see Box 8.4  AGREE instrument  NGC summaries

Developing Guidelines  There is a need for a CPG that addresses a topic when:  The topic is clinically important  The topic is complex and requires clarity  There is evidence of a gap between actual and optimal care  There are no existing valid or relevant guidelines available  There is evidence available to support guideline development  The topic is central to healthy public policy

Developing Guidelines (cont’d)  Processes and panels  Both must be identified  Review questions must be created  Conducting the literature search and review  Drafting recommendations  Peer review and dissemination

Implementing CPGs  Requires multifaceted and sustained interventions  Practitioners’ commitment and organizational leadership are keys  Use of best practice champions  EBP mentors  Guideline implementation must be sustained over time  Context must be considered

Question  Which of the following factors has the greatest bearing on the success or failure of CPG implementation? a.The strength of the evidence that underlies the guideline b.The validity of the process that was used to develop the guideline c.The education level of the nurses who will implement the guideline d.The commitment of the caregivers who will put the guideline into practice

Answer  d. The commitment of the caregivers who will put the guideline into practice  Rationale: While CPGs must be based on strong evidence and must be developed in a valid manner, these factors do not determine the success of their clinical implementation. Successful implementation is dependent on the commitment of practitioners to the process. Education level is not a key determinant of success.

Use of innovative technology to promote EBP Clinical decision support system – Access data, interventions and evidence-based recommendations focused on changing the system and/or patient policies – Integrate research – Evaluate interventions – Provide feedback to providers Source: Williamson et al. (2011). Worldviews on Evidence- based Nursing, fourth quarter

Discussion of Homework Critical Appraisal As you report on each article you critically appraised for your homework assignment, rate each article for Relevance and Noteworthiness as part of your evidence summary

McMaster Online Rating of Evidence (MORE) Source: Straus & Haynes (2009), CMAJ:180(9) Relevance of individual studies 7- directly and highly relevant 6 - definitely relevant 5 - probably relevant 4 -possibly relevant; likely of indirect of peripheral relevance at best 3- possibly not relevant 2 - probably not relevant; content only remotely related 1 - definitely not relevant; completely unrelated content area

McMaster Online Rating of Evidence (MORE) Source: Straus & Haynes (2009), CMAJ:180(9) Newsworthiness of individual studies 7 – useful information; most practitioners in my specialty definitely don’t know this 6 – useful information; most practitioners in my specialty probably don’t know this 5 – useful information; most practitioners in my specialty possibly don’t know this 4 – useful information; most practitioners in my specialty possibly already know this 3- useful information; most practitioners in my specialty probably already know this 2 – it probably doesn’t matter whether they know this or not 1 – not of direct clinical interest

Critical Appraisal Rapid critical appraisal checklist AGREE instrument – Scope and purpose – Stakeholder involvement (bias) – Rigor of development – Clarity and presentation – Application – Editorial independence

Class Exercise Randomly assign to two groups – Group 1: Schuster et al paper – Group 2: Moyer et al paper Use rapid critical appraisal checklist and then use AGREE instrument Report your findings and respond: Were there any differences in your findings and conclusion about the CPG?