1. Session 4: Frameworks used in Clinical Settings, Part 2 Janet Myers, PhD, MPH Janet.Myers@ucsf.edu415-597-8168 Session 2 ● September 27, 2012

2. Session 4 Overview  Review the rationale for frameworks  Fidelity Frameworks Fidelity of Implementation Framework for Implementation Fidelity  PARIHS Framework Promoting Action on Research Implementation in Health Services  ORCA Framework Organizational Readiness to Change  Frameworks exercise

3. Review – Why frameworks? 1. Planning: To guide the selection and tailoring of programs or interventions. 2. Implementation: To understand program “theory,” which can improve implementation 1. Can guide timing/stages of implementation 3. Evaluation: As a guide to evaluation 1. Suggest formative evaluation/diagnostic analysis 2. Guide the development of hypotheses to test. 3. Facilitate interpretation of process and outcomes and the relationship between the two

4. FOI: Fidelity of Implementation  To explain the degree to which evidence-based interventions succeed or fail.  FOI occurs between context and program effectiveness

5. Carroll et al “A conceptual framework for implementation fidelity” Implementation Science 2007 2:40. Framework for Implementation Fidelity

6. SmartSTEPS ATSM Model

7. Conceptual Framework for Evaluating Intervention Fidelity of SMARTsteps

8. PARIHS

9. PARiHS Framework Elements Evidence. Context. Facilitation. Weak to strong support for implementation

10. Evidence Sub-elements: Research evidence. Weak: Anecdotal evidence, descriptive. Strong: RCTs, evidence-based guidelines. Clinical experience. Weak: Expert opinion divided. Strong : Consensus. Patient preferences and experiences. Weak: Patients not involved. Strong : Partnership with patients. Local information.

11. Context Sub-elements: Culture. Weak: Task driven, low morale. Strong : Learning organization, patient-centered. Leadership. Weak: Poor organization, diffuse roles. Strong : Clear roles, effective organization. Evaluation. Weak: Absence of audit and feedback Strong : Routine audit and feedback.

12. Facilitation Sub-elements: Characteristics (of the facilitator). Weak: Low respect, credibility, empathy. Strong: High respect, credibility, empathy. Role. Weak: Lack of role clarity. Strong: Clear roles. Style. Weak: Inflexible, sporadic. Strong: Flexible, consistent.

13. PARiHS Framework: Elements and Sub-elements Evidence. Research Clinical experience Patient experience Local knowledge Context. Culture Leadership Evaluation Facilitation. Characteristics Role Style

14. PARIHS Causal Pathways (from Kitson et al)

15. Limitations  Used post-hoc  Study designs: Only cross-sectional or retrospective  Lack of conceptual clarity

16. ORCA  Assesses 3 major scales corresponding to the core elements of PARIHS  Evidence: published research, professional knowledge/competence, patient prefs, local context  Context: Org culture, leadership, evaluation/feedback.  Facilitation: Internal and external factors

17. Patient Exit Interviews HCW Interviews Data abstraction from EMR PREDICTOR VARIABLES (PARIHS Core elements): Evidence: the strength and nature of the evidence as perceived by multiple stakeholders ORCA-adapted measures of patient preferences ORCA measures on clinical experience and professional knowledge and perception of PP evidence Routine information derived from local practice context Context: the quality of the context of environment in which the research is implemented Patient ratings of types of information (eg. risk assessment) collected and how it is used in the setting. ORCA measures of HCW perception of organizational culture and leadership (requires multiple levels of HCW input); ratings of types of data collected and how it is used in the setting. Use of risk assessment fields in EMR; feedback to clinical team members regarding data completeness and clinical progress. Facilitation: Processes by which implementation is facilitated. ORCA measures regarding the enabling and interactive features of internal and external facilitation. OUTCOME VARIABLES (Composite FOI Score): Fidelity of Implementation: HCW satisfaction with the intervention; intervention delivery quality and consistency Quality: Reports of the quality of the intervention delivery Satisfaction: HCW reports of satisfaction with the intervention. Consistency: Proportion of patients seen who received intervention messages Table 2: Planned Data Collection for each element of the Conceptual Framework (PARIHS) and Fidelity of Implementation (FOI)