1. Identifying Necessary and Sufficient Conditions for Sustainment of Evidence-Based Substance Abuse and Mental Health Programs
Sapna J. Mendon, Lawrence A. Palinkas, Suzanne Spear, Juan Villamar, and C. Hendricks Brown

2. Acknowledgements

3. Acknowledgements Center Dir. Frances Harding, CSAP
Center for Substance Abuse Prevention (CSAP)
Division of Community Programs
Division of State Programs
Center for Mental Health Services Administration (CMHS)
Division of Prevention, Traumatic Stress, and Special Programs

4. Disclaimer
The views, opinions, and content expressed in this presentation do not necessarily reflect the views, opinions, or policies of the Center for Substance Abuse Prevention (CSAP), the Center for Mental Health Services (CMHS), the Substance Abuse Mental Health Services Administration (SAMHSA), or the U.S. Department of Health and Human Services (HHS).

5. Acknowledgements
Michael Davies
Leanne Giordono

6. Consoldiated Framework for Implementation Research (CFIR)
Widely used to study implementation in healthcare settings
Gilmer 2006: Housing First study
Green 2014: pharma treatment
How all of these characteristics work together is still not well understood.
Damschroder et al., 2009

7. How Has the CFIR Been Used?
Int: Knowledge & Beliefs about the program (Gilmer et al., 2006); Relative advantage (Damschroder and Lowery, 2013)
Outer: patient needs & resources (Gilmer et al., 2006; Damschroder and Lowery, 2013), external policies & incentives (Gilmer et al., 2006)
Ind. Characteristics: Personal Attributes (Gilmer et al., 2006); barriers: clinician’s perception of effectiveness (Green et al., 2014)
Inner: readiness, org. culture, communication, compatability (Gilmer et al. 2006); barriers: structural/communication issues, clinician availability (Green et al., 2014); communications, tension for change, relative priority, goals/feedback, learning climate, (Damschroder & Lowery, 2013)
Process: Program champions, integration of services (Green et al., 2014); Planning for implementation (Damschroder and Lowery, 2013).
CFIR:
Widely used to study implementation in healthcare settings
Guiding implementation work, for example, barriers and facilitators
Theory building
Gilmer 2006: Housing First study
Green 2014: pharma treatment
Damschroder and Lowery 2013: MOVE! weight management program (at the time, used in nearly 1,000 hospitals and outpatient clinics)-- study identifying high and low implementation facilities to study significant factors
How all of these characteristics work together is still not well understood.

8. Results from parent NIDA R34
Top 5 highest rated:
needs and resources of the communities being served (97.4%);
program champions (94.9%);
assessment of progress made towards sustainment (94.7%);
access to knowledge and information about the program (92.3%);
knowledge and beliefs about the program (91.4%)
Lowest rated:
pressures to implement from other states, tribes and communities (21.1%) and organizational incentives and rewards for implementing program (45.9%)
15 characteristics in the 51-75% quartile
Why QCA? To help make further sense of how these different characteristics work together, and which ones are deserving of greater focus (from what’s sufficient and necessary)

9. Current Study
Research Question: Based on CFIR characteristics, hat are the necessary and sufficient important to sustainment?
Sample
44 key staff member interviews from 4 different SAMHSA-funded mental health and prevention programs: GLS, PPS, SPF-SIG, Stop Act)
Missing data
Total sample for current study: 39
Current stage:
Before we can build a system designed to measure sustainment across a variety of mental health and substance abuse prevention programs, as a field, we must be able to define it.
An important part of this process includes not only identifying which factors may be important to sustainment, but also begs an understanding of how these factors interact and work together.
Outcome:
Based on the Consolidated Framework for Implementation Research, what characteristics are sufficient and necessary for the importance of sustainment
Missing data:
5 cases deleted for 100% missing data
N = 39

10. Methods: Selection of causal conditions

11. QCA Process
Data Set Calibration
Sufficiency Analysis
Necessity Analysis
Causal condition must (almost always) be present for outcome to occur
Outcome (almost) always occurs when causal condition is present ** causal pathway
Truth table gives us….
Standard analysis…
Necessity Analysis…
**Results of each**
Ragin, 1987

12. Methods: Thresholds and Calibration
Causal conditions:
Rating of 0 = no membership
Rating of 2 or higher and support from qualitative information = full membership
Outcome:
At least 60% of causal conditions rated as important = full membership (on outcome)
Full membership approx.79%

13. Truth Table Decision points Results:
19 circumstances where some configuration of these conditions were applicable
Frequency of cases with consistency on outcome: > 0.8
Of those 19, 12 had a consistency score of 1 (20 cases)
Remaining cases were given a 0 on outcome (based on raw consistency)
All remaining cases with zero membership are deleted
Truth table gives us….
Standard analysis…
Necessity Analysis…
**Results of each**

14. Sufficiency Analysis
fsQCA will give you 3 solutions: Complex, Parsimonious, and Intermediate. Per Ragin, Intermediate Solution should be used generally.

15. Causal (sufficient) Pathway Important for Sustainment
INT: idea
OUT: network
IN: structure
IN: fit of values
IND: identification w/ org
IND: personal attributes
PROC: external change agents
Consistency = 1
Coverage = .45 * * *
Causal (sufficient) Pathway Important for Sustainment * * *
Program
Freq.
SPF-SIG 4
GLS 3
PPS 1
Stop Act
= logical and
Conjuncture of relationships = causal pathway *

16. Causal (sufficient) Pathway Important for Sustainment
INT: idea
INT: costs
IN: structure
IN: fit of values
IND: personal attributes
OUT: network
PROC: executing to plan
Consistency = 1
Coverage = .45 * * *
Causal (sufficient) Pathway Important for Sustainment * * *
Program
Freq.
GLS 3
PPS 1
SPF-SIG
Stop Act
= logical and
Conjuncture of relationships = causal pathway *

17. Causal (sufficient) Pathway Important for Sustainment
INT: idea
OUT: network
IN: structure
IN: fit of values
IND: identification w/ org
IND: personal attributes
PROC: executing to plan
Consistency = 1
Coverage = .45 * * *
Causal (sufficient) Pathway Important for Sustainment * * *
Program
Freq.
GLS 3
PPS 1
SPF-SIG
Stop Act
= logical and
Conjuncture of relationships = causal pathway *

18. Necessity Analysis: Results
Independent analyses
2B, 3A, 3D2 – Ragin states the rule of thumb is 90% for consistency, but this can be relaxed (lowest .8)
**These items are NECESSARY for the importance of sustainment**

19. Necessity Analysis (conditions)
Threshold (consistency)
OUT: Degree of Network
> 90%
IN: structural characteristics
IN: Fit of values
INT: Where idea came from
80-89%
IND: Other personal attributes
PROC: External change agents
PROC: Executing program according to plan

20. Discussion and Implications
Network and potential mechanisms of network
Age and maturity are important to sustainment, but….
Alignment of values– of program to individuals, to existing workflows; presumably stronger alignment
Intervention components and process are necessary- How is information (int.) and approach to implementation being discussed?
Personal attributes: are there ways leadership can promote motivation, learning, ability to tolerate ambiguity?
When *** is necessary, this can be a focus for intervention developers, organization leaders, and mid-level supervisors
*Limitation: need further information to determine directionality; example: structural characteristics– what are the benchmarks, if any

21. Next Steps
Run as fuzzy set analysis (accounting for partial membership)
Develop complex causal pathways (from sufficiency analysis)
Identify necessary pathways

22. THANK YOU! smendon@usc.edu