1. CAN-IMPLEMENT © for Guideline Adaptation and Implementation Part 2: The Methodology

3. 3 Phase Methodology PHASE 1 Identification and Clarification of Issue/Problem PHASE 2 Solution Building PHASE 3 Implementation, Evaluation and Sustainability

5. PHASE 1: Identification & Clarification of Issue/Problem

6. PHASE 1, Step 1: Call-to-Action 1.1 Clarify the motivation, purpose and scope of the proposed initiative. Consider:  What is the agency/ institutional mandate and infrastructure supporting evidence-informed practice?  Is this a response to a specific practice challenge?  Is a guideline the most appropriate solution to the challenge?  Who (person/group) will lead, implement and maintain these recommendations?  What is the intended practice jurisdiction (local, regional, national??

7. PHASE 1, Step 1: Call-to-Action A critical, strategic element requiring strong facilitation and leadership skills; establishes legitimate guideline development mandate and infrastructure; especially important for new groups  Plan an orientation session for participants  Discuss: What is a “guideline”?

8. Tools PHASE 1 : Evidence-informed practice Training Resources

9. PHASE 1, Step 2: Plan 2.1 Establish scope of guideline and articulate Health Question(s). 2.2 Determine feasibility of adaptation. 2.3 Form steering committee and working panel(s) and determine key stakeholders and necessary resources 2.4 Determine consensus process. 2.5 Write the Adaptation Work Plan.

10. PHASE 1, Step 2: Plan  Use the PIPOH instrument to develop health questions  Determine required expertise and resources; (using the Skills Assessment Checklist)  Understand Facilitation and the role of the Coordinator  Draft Adaptation Work Plan – an essential document outlining:  Scope of topic and health questions  Terms of reference (steering committee and working panel(s)  Funding and resource commitments  Consensus process  Conflicts of interest  Projected timeline  Meeting arrangements

11. Tools PHASE 1 : PIPOH Checklist ► Patient population: average risk women ► Intervention: screening ► Professionals: family physicians/ GPs/nurses ► Outcomes: screening interval/modality ► Healthcare setting: family practice Example Question: What is appropriate cervical cancer screening (CCS) for average risk women seen in family practice?

12. Tools PHASE 1 : Search Sources

13. Tools PHASE 1 : Skills Assessment Checklist

14. Tools PHASE 1 : Sample Declaration of Conflict of Interest CONFLICT OF INTEREST DISCLOSURE DECLARATION

15. Tools PHASE 1 : Sample Adaptation Work Plan

16. PHASE 1, Step 3: Search and Screen 3.1 Search existing guidelines, systematic reviews, and new or emerging areas of evidence; confirm if guideline is de novo, adaptation or mixed initiative. 3.2 Screen search results to develop short list for full appraisal; document selection.

17. PHASE 1, Step 3: Search and Screen  Designing and executing the search - engaging services of a health science librarian or information specialist  Managing citations: Developing a screening protocol and documenting selection decisions  Library Science Supplement and Toolkit resources

18. Designing the search: Choosing inclusion/exclusion criteria  Selecting only evidence-based guidelines (guideline must include a report on systematic literature searches and explicit links between individual recommendations and their supporting evidence  Selecting only national and/or international guidelines; selecting guidelines written in a particular language Fr/Eng?)  Specifying a range of dates for publication; selecting only those published since an important review was published  Selecting peer reviewed publications only; excluding guidelines written by a single author not on behalf of an organization – ideally has multidisciplinary input  Excluding guidelines published without references – panel must have access to the evidence

19. Potential papers found (n=599) Potential papers found (n=599) Rejected – not CPGs (n=537) Rejected – not CPGs (n=537) Remaining more Carefully reviewed (n=62) Remaining more Carefully reviewed (n=62) Rejected – not population screening (n=10) Rejected – not population screening (n=10) Duplicates searched for in remaining (n=52) Duplicates searched for in remaining (n=52) Duplicates (n=32) Duplicates (n=32) Updates searched for in remaining (n=20) Updates searched for in remaining (n=20) Updated (n=2) Updated (n=2) Cervical cancer population screening CPGs (n=18) Cervical cancer population screening CPGs (n=18) 8 from Medline 9 from Web 1 from CCOHTA 8 from Medline 9 from Web 1 from CCOHTA Evidence/ Literature Search and Screen

20. PHASE 1, Step 4: Assess and Select 4.1 Assess shortlisted guidelines (recommendations and supporting evidence) in detail for: quality (e.g. AGREE), currency, content, coherence between evidence and recommendations, and applicability and acceptability to local context. 4.2 Decision and Selection: review all assessments and achieve consensus with respect to Selecting, Rejecting or Modifying specific

21. PHASE 1, Step 4: Assess and Select  Assessing Quality of guidelines  Using the AGREE II Instrument  Summarizing and displaying AGREE scores  Assessing Quality of Systematic Reviews  Assessing guideline Currency  Assessing guideline Content  Preparing the ‘Recommendations Matrix’  The evidence: type and level; classification systems  Critical appraisal (interpretation and Consistency of evidence)  Assessing Acceptability and Applicability  Making Decision to Accept, Reject or Modify  Achieving and documenting consensus

22. www.agreetrust.org

23. Tools PHASE 1: AGREEII Instrument 23 items 7 point Scale Overall Assessment User Guide & Manual 1. Scope & purpose (3) 2. Stakeholder involvement (3) 3. Rigour of development (8) 4. Clarity of presentation (3) 5. Applicability (4) 6. Editorial independence (2) SIX Domains

24. CURRENCY Assessment  Is there any new evidence relevant to guideline?  Does new evidence invalidate any of the recommendations?  Are there any plans to update the guideline in the near future?  When was the guideline last updated?

25. CONSISTENCY Assessment  Quality of source guideline search strategy and study selection ( ADAPTE Tool 13)  Was the search for evidence comprehensive?  Is there any bias in the selection of articles?  Consistency between evidence and interpretations; between interpretations and recommendations ( ADAPTE Tool 14)  Is the evidence valid, overall?  Are the recommendations based on data and interpretations?

26. Tools PHASE 1 : Evaluation: Search and Selection of Evidence

27. Tools PHASE 1 : Evaluation Sheet: Scientific Validity of Guidelines C onsistency between evidence, its interpretation and recommendations)

28. Tools PHASE 1 : Recommendations Matrix (template)

29. APPLICABILITY Assessment  Review of each of the recommendations with respect to 2 main questions (ADAPTE Tool 15)  Can the recommendation be put into practice? Consider patient similarity, interventions, outcomes, patient preferences, availability of equipment, availability of expertise, any constraints?  Is the benefit from this recommendation worth implementing?

30. Tools PHASE 1 : Evaluation Sheet: Acceptability/Applicability (Process is repeated as needed for additional health questions)

31. Decision and Selection; you might …  Use process of elimination based on overall AGREE scores  Eliminate any outdated CPGs  Look at recommendations of top 3 CPGs and decide which you could adopt in entirety; which you cannot reach agreement upon; which individual recommendations you can agree to adopt …

32. Decision and Selection Options 1. ACCEPT a whole guideline and all of its recommendations After reviewing all of the assessments, the panel accepts the guideline as is. 2. REJECT a whole guideline and all of its recommendations After reviewing all of the assessments, the panel decides to reject the complete guideline. The decision will be based on how the panel weighs the assessments (e.g., poor AGREE scores, guideline is out-of-date, the recommendations do not apply to the panel’s context). 3. ACCEPT the evidence summary of the guideline After reviewing all of the assessments, the panel decides to accept the description of the evidence (or parts) but to reject the interpretation and the recommendations. 4. ACCEPT a single recommendation After reviewing the recommendations from the guideline or guidelines, the panel decides which to accept and which to reject which may be from one or more guidelines. 5. MODIFY single recommendations After reviewing all of the recommendations from the guideline(s), the panel decides which are acceptable but need to be modified.

33. PHASE 1, Step 5: Draft, Review, and Endorse Recommendations 5.1 Draft Customized Guideline 5.2 Conduct internal review and make revisions 5.3 Conduct external review and obtain endorsement 5.4 Prepare final documents, including any practitioner and patient information, records or application tools, and appropriate source acknowledgments 5.5 Establish a Renewal Plan

34. PHASE 1, Step 5: Draft, Review, and Endorse Recommendations  Customizing recommendations:  Using a template for structure and content  Authors, acknowledgements, permissions and copyright issues from source developers  Using brief, unambiguous, actionable language  Including application tools, algorithms, patient information  Including a short preface summarizing recommendations, and methodology; appendices and possible web links to documents Important aspect: Transparency of all decision making (e.g., consensus process is described, how decisions were arrived at and resolved; if recommendations were modified, how and why they were modified);

35. External Review  External review with those affected by uptake, e.g., experts (practitioners, patients) and users (policy makers, managers)  Consultation with endorsement bodies  Inclusion of representative on panel throughout process?  Consultation with source guideline developers  Acknowledgement of source document s

36. Tools PHASE 1: Sample External Review Survey

37. Sustainability/Planning for renewal  Guideline maintenance  specifying in the guideline when, how and by whom the recommendations will be updated

38. PHASE 2: Solution Building

39. PHASE 2, Step 1: Align Knowledge to Local Context (Practice and System) Conduct a gap analysis To focus an implementation action plan, it is necessary to first understand the difference between current practices and the new recommendations. The objective of the gap analysis is to clarify exactly what and how much will need to change in the prevailing practice and system.

40. Gathering information Information about the local population, organization, and providers can be obtained from many sources using a variety of formal and informal methods, including:  Guideline documentation from PHASE 1 Steps 1-5, e.g. previous environmental scans or other baseline measurements, and meeting notes from discussions on re: acceptability and feasibility of recommendations  Clinical/Chart Audit  Interviews, focus groups, town hall sessions, surveys and questionnaires

41. PHASE 2, Step 2: Assess the Innovation, Adopters and Practice Environment for “Barriers and Supports” Innovation (examples):  Attributes of the evidence-informed recommendations  The rigour/credibility of the guideline development process  Perceived compatibility of recommendations with existing routines (e.g. complexity, convenience, relative advantages or costs

42. PHASE 2, Step 2: Assess the Innovation, Adopters and Practice Environment for “Barriers and Supports” Adopters (examples):  Awareness  Knowledge and Skills  Attitudes, expectations, motivation  Current behaviours or routines  Concerns, preferences NOTE: Adopters include allied health practitioners (the care “providers”), administrative and organizational staff and leaders – and patients and their families!

43. PHASE 2, Step 2: Assess the Innovation, Adopters and Practice Environment for “Barriers and Supports” Practice Environment/Organizational Context (examples):  Organizational culture and “readiness” for evidence-based practice  Infrastructure; levels of authority and decision-making processes  Resources; economic constraints  Leadership; availability of Champions  Work Load; levels of job stress  Quality Improvement mechanisms

44. (Tools) RNAO “Environmental Readiness Assessment” (worksheet) Available at http://www.rnao.org/Page.asp?PageID=924&ContentID=823 http://www.rnao.org/Page.asp?PageID=924&ContentID=823

45. PHASE 2, Step 3: Select and Tailor Implementation Interventions Best practice Implementation strategies should address barriers related to the individual practitioner, social context and organizational and environmental context - and should be tailored to different groups of stakeholders (i.e. nursing staff, project leaders and administrators.”) Ploeg, 2007

46. Implementation Strategies/Tactics  Educational Outreach  Reminders: manual or computerized  Introduction of computers in practice; computerized decision support  Multi-faceted interventions  Interactive small group meetings  Multi-professional collaboration  Mass media campaigns  Financial interventions (fund-holding/budgets)  Use of opinion leaders/champions  Audit and feedback  Conferences and courses  Educational materials  Patient-mediated interventions  Dissemination of printed educational material  Didactic educational sessions (lectures)  Total quality improvement/continuous quality improvement

47. Usability Testing Barrier management and knowledge transfer are iterative processes. Consider the value of testing new procedures, decision making algorithms, training information and other support tools with both patients and providers before proceeding with a full scale implementation.

48. Tools PHASE 2 : Implementation Action Planning (template)

49. PHASE 3: Implementation, Evaluation & Sustainability

50. Evaluation Planning: Monitoring Uptake and Assessing Outcomes In PHASE 3, groups evaluate both the level of evidence uptake and the impact of the new recommendations for care on the intended:  patient population  health care providers and  organization

51. Evaluation Planning: Monitoring Uptake and Assessing Outcomes* *NOTE (June 2011): Canadian Partnership Against Cancer Study Cases are in the process of evaluating their guideline use and outcomes. Findings will be updated in CAN-IMPLEMENT © Resource, Version 3.0 Key Concepts  Evaluating use:  Defining evaluation period including date of “knowledge activation” and “targeted threshold” of guideline use  Evaluating implementation process  Revisiting and augmenting/modifying implementation “interventions”  Evaluating impact:  Defining measurement “indicators”  Establishing tracking measures (and resources)  Assessing Patient, Practice (provider) and System impacts

52. Using the CAN-IMPLEMENT © Resource Navigation  Field Notes  Tips  Checklists  Thinking about Implementation  Progress Checks - Direction - Outputs - Documents - Tools