Julie C. Lowery, PhD, MHSA Associate Director, VA CCMR; Co-Implementation Research Coordinator, VA Diabetes QUERI HSR&D Center for Clinical Management Research VA Ann Arbor Healthcare System

 A comprehensive framework to promote consistent use of constructs, terminology, and definitions  Consolidates existing models and frameworks  Comprehensive in scope  Can tailor use to each project 2 Damschroder L, Aron D, Keith R, Kirsh S, Alexander J, Lowery J: Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. 2009, 4:50.

 Intervention  8 Constructs (e.g., evidence strength & quality, complexity)  Outer Setting  4 Constructs (e.g., patient needs & resources)  Inner Setting  14 constructs (e.g., leadership engagement, available resources)  Individuals Involved  5 Constructs (e.g., knowledge, self-efficacy)  Process  8 Constructs (e.g., plan, engage, champions) 3

4 Ref: Additional Resources:

 Embraces, consolidates, and standardizes key constructs from other models  Agnostic to specific models and theories  Provides a pragmatic structure for evaluating complex implementations  Helps to organize findings across disparate implementations  Paves the way for cross-study synthesis 5

6  Consists of 39 individual constructs  Cannot use them all in every study  Not all will apply  Conduct a priori assessment of which constructs to include  Interviews  Survey  Develop interview guide based on key constructs

 Intervention Source (Perception of key stakeholders about whether the Mini-Residency program is externally or internally developed) {Choose one} ( ) Very Unimportant ( ) Unimportant ( ) Neutral ( ) Important ( ) Very Important  Evidence Strength & Quality (Stakeholders' perceptions of the quality and validity of evidence in the published literature, based on clinical experience, or other local evidence or experience supporting the belief that the Mini- Residency program will have desired outcomes)

 Purposive sample of low & high uptake sites  Semi-structured interviews with key stakeholders  Rapid analysis  Analyze notes instead of verbatim transcripts  Use a prescribed coding template: CFIR  Analyze data on an ongoing basis 8

Two pairs of analysts do coding and analysis. For each respondent:  Each analyst independently codes  Meet to compare and achieve consensus on coding 9

For each respondent:  Two analysts independently code  Meet to compare and achieve consensus on coding 10

For each respondent, each construct:  Two analysts independently rate  Meet to compare and achieve consensus on rating  Create summary memo with supporting quotes and recommendations 11

 Think of CFIR constructs as independent variables  Construct 1 + construct 2 + … = f(implementation effectiveness)  Ratings  ordinal values of independent variables  Is the construct positive or negative force in the organization?  Does it manifest strongly or weakly?  Is the construct present but neutral? 12

Positive: Facilitating influence on implementation Negative: Impeding influence on implementation Strong+2: Specific statements, concrete examples -2: Specific statements, concrete examples Weak+1: General statements without specific examples -1: General statements without specific examples NeutralPresent but no influence (0); mixed remarks (* - these are important at the organizational level) MissingNo information obtained or respondent not knowledgeable 13

CFIR Construct Rating (-2, -1, 0, +1, +2) plus Summary of Rationale* Recommendation(s)** Reflecting & Evaluating No, no, I really haven’t received any data. They haven’t included e-consults on any reports. Would be good to have a website to go for any info but I am not aware of anything at this point. Certainly resource tool available would be helpful. Patient Needs & Resources How are patients involved? At this point, not that involved, to be frank. I think they should be more involved. They are not used to that when Patient asks to see spec, has to explain, they get electronic apt. I would think that specialist would give Patient a call but they give back to PC and give advice. Patient should be. We should get a Patient satisfaction survey, are they satisfied are not involved as they, I

 For each construct, review ratings across respondents. Consider:  Involvement  Role  Knowledge  Decide on a facility-level rating for each respondent (“weighted average”)  Pull the most important quotes from the respondent memos to justify each rating at the facility level  Add recommendations 15

 Identify constructs that appear to correlate with implementation success/uptake  Returning to the facility-level memos (or the respondent memos, if necessary), describe how these constructs are manifested at high implementation sites (facilitators) and low implementation sites (barriers)  Develop recommendations for dissemination based on these findings 16

CFIR ConstructsLowHigh I. INTERVENTION CHARACTERISTICS Relative advantage-2122 II. OUTER SETTING Patient needs & resources-2022 External Policy & Incentives-201 III. INNER SETTING Networks and communications-2 22 Implementation Climate Tension for change0011 Relative priority-212 Goals and feedback-212 Learning climateN/A12 Readiness for Implementation Leadership Engagement-222 Available resources-2 1 V. PROCESS PlanningN/A11 Executing-2122 Reflecting & Evaluating

 “Chief of Staff as well has recognized the value. Not cracking the whip, but making sure there’s motivation…Especially at executive meetings showing his wide support.”  “It’s exciting to see when people are excited about an initiative and embrace it. Had all the chiefs sitting in the room to sell the system…Leadership provided FTE after they saw the additional workload generated.”  “The support of leadership/administration moving forward and encouraging specialists to go forward implementing E-consults. Primarily this is the Chief of Staff…The medical director is always looking at improving access, so this is a natural involvement for him.”

Sites could not identify a local champion and/or perceived little support from top leadership. Another common negative comment was that leadership was too focused on the numbers, and not the quality of care.  “More focused on meeting targets than quality.”  In response to question of leadership involvement: “Not really. Their involvement is more on the side of the statistics of what we are using.”

 Leadership’s focus on achieving a certain number of E- Consults is not viewed as supportive. If leadership wants to demonstrate their interest, they need to derive better metrics for tracking implementation, preferably ones that focus on the appropriateness and quality of the consults, and patient and provider satisfaction.  Leadership needs to recognize the additional time that may be required for PCPs to collect additional data needed to provide to PCPs and implement specialists’ recommendations.  Leadership should keep the program in the forefront of meetings/discussions with clinicians and clinical leadership, linking the success of the program to VHA and medical center goals of improving access to quality care for our Veterans.

 Continue to refine CFIR constructs & rating process  Link to quantitative measures to increase efficiency of process  Build a repository of findings and continue to add study findings  Tool for researchers & practitioners  Use repository to conduct synthesis across studies  Use more sophisticated and rigorous analyses (e.g., QCA) to develop higher-order theories 21

Linking CFIR constructs to quantitative measures 22

ProgramModeDoseContent MOVE! ® Weight Management On-site Weekly hr in- person group sessions 6-14 weeks Weight Loss DPP-inspired Multi-disciplinary team TeleMOVEIn-home devices 1 message/day for 82 days Daily workbook lesson 3 x min monthly calls Daily psycho-ed content Safety checks Motivational and problem-solving support Telephone Lifestyle Coaching (TLC) Telephone 10 x 20 min sessions 6 months Unlmt inbound calls 6 topics MI coaching Coach continuity 23

Study:MOVE!TeleMOVETLC Structural Characteristics Networks & Communications Tension for Change Compatibility Relative Priority Goals & Feedback Learning Climate Leadership Engagement Available Resources 24 Strongly Distinguishes Weakly Distinguishes Not assessed

 The CFIR Wiki will promote:  Shared definitions  Operationalization of definitions  Repository of findings  Predictive modeling  Site-specific “System-change likelihood Indices” 1  Which will result in…  … more reliable implementation strategies  …more generalized knowledge about what works where and why 1. Davidoff F: Heterogeneity is not always noise: lessons from improvement. JAMA 2009, 302:

Visit the CFIR Wiki: 26