Identifying Necessary and Sufficient Conditions for Sustainment of Evidence-Based Substance Abuse and Mental Health Programs Sapna J. Mendon, Lawrence A. Palinkas, Suzanne Spear, Juan Villamar, and C. Hendricks Brown

Acknowledgements

Acknowledgements Center Dir. Frances Harding, CSAP Center for Substance Abuse Prevention (CSAP) Division of Community Programs Division of State Programs Center for Mental Health Services Administration (CMHS) Division of Prevention, Traumatic Stress, and Special Programs

Disclaimer The views, opinions, and content expressed in this presentation do not necessarily reflect the views, opinions, or policies of the Center for Substance Abuse Prevention (CSAP), the Center for Mental Health Services (CMHS), the Substance Abuse Mental Health Services Administration (SAMHSA), or the U.S. Department of Health and Human Services (HHS).

Acknowledgements Michael Davies Leanne Giordono

Consoldiated Framework for Implementation Research (CFIR) Widely used to study implementation in healthcare settings Gilmer 2006: Housing First study Green 2014: pharma treatment How all of these characteristics work together is still not well understood. Damschroder et al., 2009

How Has the CFIR Been Used? Int: Knowledge & Beliefs about the program (Gilmer et al., 2006); Relative advantage (Damschroder and Lowery, 2013) Outer: patient needs & resources (Gilmer et al., 2006; Damschroder and Lowery, 2013), external policies & incentives (Gilmer et al., 2006) Ind. Characteristics: Personal Attributes (Gilmer et al., 2006); barriers: clinician’s perception of effectiveness (Green et al., 2014) Inner: readiness, org. culture, communication, compatability (Gilmer et al. 2006); barriers: structural/communication issues, clinician availability (Green et al., 2014); communications, tension for change, relative priority, goals/feedback, learning climate, (Damschroder & Lowery, 2013) Process: Program champions, integration of services (Green et al., 2014); Planning for implementation (Damschroder and Lowery, 2013). CFIR: Widely used to study implementation in healthcare settings Guiding implementation work, for example, barriers and facilitators Theory building Gilmer 2006: Housing First study Green 2014: pharma treatment Damschroder and Lowery 2013: MOVE! weight management program (at the time, used in nearly 1,000 hospitals and outpatient clinics)-- study identifying high and low implementation facilities to study significant factors How all of these characteristics work together is still not well understood.

Results from parent NIDA R34 Top 5 highest rated: needs and resources of the communities being served (97.4%); program champions (94.9%); assessment of progress made towards sustainment (94.7%); access to knowledge and information about the program (92.3%); knowledge and beliefs about the program (91.4%) Lowest rated: pressures to implement from other states, tribes and communities (21.1%) and organizational incentives and rewards for implementing program (45.9%) 15 characteristics in the 51-75% quartile Why QCA? To help make further sense of how these different characteristics work together, and which ones are deserving of greater focus (from what’s sufficient and necessary)

Current Study Research Question: Based on CFIR characteristics, hat are the necessary and sufficient important to sustainment? Sample 44 key staff member interviews from 4 different SAMHSA-funded mental health and prevention programs: GLS, PPS, SPF-SIG, Stop Act) Missing data Total sample for current study: 39 Current stage: Before we can build a system designed to measure sustainment across a variety of mental health and substance abuse prevention programs, as a field, we must be able to define it. An important part of this process includes not only identifying which factors may be important to sustainment, but also begs an understanding of how these factors interact and work together. Outcome: Based on the Consolidated Framework for Implementation Research, what characteristics are sufficient and necessary for the importance of sustainment Missing data: 5 cases deleted for 100% missing data N = 39

Methods: Selection of causal conditions

QCA Process Data Set Calibration Sufficiency Analysis Necessity Analysis Causal condition must (almost always) be present for outcome to occur Outcome (almost) always occurs when causal condition is present ** causal pathway Truth table gives us…. Standard analysis… Necessity Analysis… **Results of each** Ragin, 1987

Methods: Thresholds and Calibration Causal conditions: Rating of 0 = no membership Rating of 2 or higher and support from qualitative information = full membership Outcome: At least 60% of causal conditions rated as important = full membership (on outcome) Full membership approx.79%

Truth Table Decision points Results: 19 circumstances where some configuration of these conditions were applicable Frequency of cases with consistency on outcome: > 0.8 Of those 19, 12 had a consistency score of 1 (20 cases) Remaining cases were given a 0 on outcome (based on raw consistency) All remaining cases with zero membership are deleted Truth table gives us…. Standard analysis… Necessity Analysis… **Results of each**

Sufficiency Analysis fsQCA will give you 3 solutions: Complex, Parsimonious, and Intermediate. Per Ragin, Intermediate Solution should be used generally.

Causal (sufficient) Pathway Important for Sustainment INT: idea OUT: network IN: structure IN: fit of values IND: identification w/ org IND: personal attributes PROC: external change agents Consistency = 1 Coverage = .45 * * * Causal (sufficient) Pathway Important for Sustainment * * * Program Freq. SPF-SIG 4 GLS 3 PPS 1 Stop Act = logical and Conjuncture of relationships = causal pathway *

Causal (sufficient) Pathway Important for Sustainment INT: idea INT: costs IN: structure IN: fit of values IND: personal attributes OUT: network PROC: executing to plan Consistency = 1 Coverage = .45 * * * Causal (sufficient) Pathway Important for Sustainment * * * Program Freq. GLS 3 PPS 1 SPF-SIG Stop Act = logical and Conjuncture of relationships = causal pathway *

Causal (sufficient) Pathway Important for Sustainment INT: idea OUT: network IN: structure IN: fit of values IND: identification w/ org IND: personal attributes PROC: executing to plan Consistency = 1 Coverage = .45 * * * Causal (sufficient) Pathway Important for Sustainment * * * Program Freq. GLS 3 PPS 1 SPF-SIG Stop Act = logical and Conjuncture of relationships = causal pathway *

Necessity Analysis: Results Independent analyses 2B, 3A, 3D2 – Ragin states the rule of thumb is 90% for consistency, but this can be relaxed (lowest .8) **These items are NECESSARY for the importance of sustainment**

Necessity Analysis (conditions) Threshold (consistency) OUT: Degree of Network > 90% IN: structural characteristics IN: Fit of values INT: Where idea came from 80-89% IND: Other personal attributes PROC: External change agents PROC: Executing program according to plan

Discussion and Implications Network and potential mechanisms of network Age and maturity are important to sustainment, but…. Alignment of values– of program to individuals, to existing workflows; presumably stronger alignment Intervention components and process are necessary- How is information (int.) and approach to implementation being discussed? Personal attributes: are there ways leadership can promote motivation, learning, ability to tolerate ambiguity? When *** is necessary, this can be a focus for intervention developers, organization leaders, and mid-level supervisors *Limitation: need further information to determine directionality; example: structural characteristics– what are the benchmarks, if any

Next Steps Run as fuzzy set analysis (accounting for partial membership) Develop complex causal pathways (from sufficiency analysis) Identify necessary pathways

THANK YOU! smendon@usc.edu