FINDING THE OPTIMAL BALANCE BETWEEN FIDELITY AND FIT: LOCAL ADAPTATIONS OF CRIME PREVENTION PROGRAMS Melanie Bania Benjamin Roebuck Beth O’Halloran Vanessa Chase 2017 Prepared for Public Safety Canada

Overview of Project Components Finding the Optimal Balance Between Fidelity and Fit: Local Adaptations of Crime Prevention Programs Literature Review Recommendations, Framework & Practical Tools Key Effective Elements of Crime Prevention Programming Illustrative Case Examples Summary Toolkit Overview of Project Components

Overview & Key Findings Literature Review Overview & Key Findings

How can evidence-based crime prevention initiatives be adapted from one successful program into new contexts with different people, cultures, and geographies, while remaining effective? Research Question

Literature Review Outline Situating the Concept of Program Adaptation Scaling Effective Initiatives Replication, Fidelity, & Implementation Science Program Adaptation: Balancing Fidelity & Fit Specific Frameworks for Program Adaptation Types of Program Modifications Green Light, Yellow Light, & Red Light adaptations Dynamic Process Adaptation (DAP) ADAPT-ITT Community-Based Participatory Research (CBPR) Empowerment Implementation Cultural Adaptation Literature Review Outline

Methodology Extensive search of multiple databases and libraries: Mendeley National Implementation Research Network uOttawa Library, including databases of online journals Google.com Retained 257 publications for reading Retained 96 publications for inclusion in the review Methodology

Scaling Effective Interventions Program replication and program adaptation are both forms of ‘scaling out’ – trying to expand the reach of an evidence-based program to other individuals in other settings Program replication is an exact copy (Stith et. al., 2006) Program adaptation modifies an evidence-based program to meet the unique needs of a specific population (Child & Family Research Institute) Scaling Effective Interventions

Replication, Fidelity, & Implementation Science Implementation fidelity The degree to which the implementation of an evidence-based program adheres to the exact elements, components, activities and tools developed and tested by its original developers (Stith et al., 2006) Better fidelity generally suggests better outcomes (Metz, 2016) Implementation drivers (Metz, 2016): Leadership Drivers – effective leadership Competency Drivers – competent and well-trained staff Organization Drivers – organizational support, project management, data monitoring, and ongoing improvement Enabling contexts (Fixsen et al., 2005; Metz, 2016; Moore et al., 2013): Dynamic environments open and amenable to change Site capacity and readiness (Bory & Franks, 2016) Replication, Fidelity, & Implementation Science

Program Adaptation: Balancing Fidelity & Fit Tension Between (Castro, Barrera, & Martinez, 2004): Fidelity: scientific, universally applicable, empirically-supported programs replicated with high implementation fidelity Fit: responding adequately to local contexts and needs Potential conflict between those who prioritize fidelity by focusing on empirical aspects, and those who prioritize fit by emphasizing community knowledge and processes that seek to engage local people in meaningful ways (Bania, 2012) Program Adaptation: Balancing Fidelity & Fit

Fidelity & Fit: Evidence-Based Decision Making Components of Evidence-Based Decision Making (CDC, nd) Fidelity & Fit: Evidence-Based Decision Making

Program Adaptation: Balancing Fidelity & Fit Ad Hoc Adaption: Leads to program ‘drift’ and a loss of benefits to participants (Aarons et al., 2012). Program adaptation must be a carefully planned and intentional process where modifications are made through a series of assessments and decisions amongst program developers, researchers, funders and services providers. Balance of fidelity and fit is best addressed through a planned, organized , and systematic approach (Castro, Barrera, & Martinez, 2004; Solomon, Card & Malow, 2006). Benefits of program adaptation (Solomon, Card & Malow, 2006): Enhances community support Enhances client participation Enhances program satisfaction Enhances outcomes Promotes sustainability Program Adaptation: Balancing Fidelity & Fit

Program Adaptation on the Ground Types of Program Modification (adapted from Stirman et al., 2013): By whom is the modification made? What is modified? How is it modified? Green Light, Yellow Light, & Red Light Adaptations (adapted from O’Conner et al., 2007; & U.S. Department of Health and Human Services, 2012, p.4-5). Green Light Adaptations: Go for it! Yellow Light Adaptations: Proceed with caution! Red Light Adaptations: Stop! Program Adaptation on the Ground

List of Adaptations to Consider or Avoid

Specific Frameworks for Program Adaptation Dynamic Process Adaptation (DAP) ADAPT-ITT Community-Based Participatory Research (CBPR) Empowerment Implementation Cultural Adaptation Specific Frameworks for Program Adaptation

Dynamic Adaptation Process (DAP) Six stages of implementation for the replication of an evidence- based program (Aarons et al., 2012; Savignac & Dunbar, 2014). Exploration and adoption of an evidence-based program Preparation and installation in the site Initial implementation in the site Full implementation in the site Sustainability / continuity Innovation (adjustments) Also includes (Aarons et al., 2012): Multi-level assessment of ‘fit’ at the exploration phase Implementation resource team Formal feedback loops between stages of implementation Dynamic Adaptation Process (DAP)

ADAPT-ITT Phase Consideration Assessment Who is the new target population and what are the risk and protective factors? Decision What evidence-based program is going to be selected and is it going to be adopted or adapted? Administration What in the original evidence-based program needs to be adapted, and how should it be done? Production How do you produce draft 1 and document adaptations? 5. Topical Experts Who can help adapt the evidence-based program? Integration What will be included in the pilot adaptation? Training Who needs to be trained? Testing Was the adaptation successful, and did it enhance sort-term outcomes? ADAPT-ITT (adapted from Wingood & DiClemente 2008, Table 1, p.542).

Community-Based Participatory Research (CBPR) A Continuum of Participatory Research in Practice (Lee, 2008) Community-Based Participatory Research (CBPR) Research ON communities Research FOR communities Research WITH communities Less Community Involvement More

Community-Based Participatory Research (CBPR) Cyclical Process of Community-Based Participatory Research Community-Based Participatory Research (CBPR) Engagement Research Questions/ Needs Assessment Research Plan/ Action Plan Implementation Analysis, Interpretation, Dissemination

Empowerment Implementation Four Steps (Rabin, 2016; Van Daele, 2012): Develop theoretically sound core program components, test them in a controlled setting, identify elements that are key to efficiency. Select skilled partners to be responsible for implementation. Adapt program by involving partners to assess the fidelity/adaptation concerns, decide on logistics and practical applications, and work on content adaptations with partners and researchers as equal contributors. Develop implementation plan with the role of partners outlined. Researchers monitor variations to essential program elements and work with partners to address deviations and avoid future issues. Empowerment Implementation

Cultural Adaptation Considerations for Cultural Adaptation Integrating cultural variables in community initiatives Multilevel cultural framework / Cultural sensitivity model (surface culture and deep culture adaptations) Dimensions of culturally sensitive interventions / Ecological validity model Guiding principles for cultural adaptation Cultural Adaptation

Recommendations, Framework & Tools Planning for Success

Recommendations For Funders & Policy Makers Invest in systematic development and implementation of adaptations Support rigorous mixed methods evaluation of adaptations For Local Adaptations of Crime Prevention Programming on the Ground Support key leadership, competency, and organization drivers Balance empirical evidence with context and experience Rely on acceptable motives Avoid fundamental change to key elements of intervention Respect and reflect the surface structure and deep structure of culture Use a strengths-based and capacity-building lens Use mixed methods and community based participatory research to guide assessment and evaluation Use an intentional and systematic approach to balance fidelity & fit Recommendations

Build a strong foundation through strong leadership and effective partnership building Conduct program exploration and participatory assessment to select an appropriate evidence-based intervention and decide on its necessary adaptations; create adaptation plan Conduct training and pilot testing of adaptation Refine program adaptation and begin implementation Evaluate and maximize program quality (process) Evaluate and maximize program impact (outcomes) Disseminate information about your program and its results ~ Ensure continuous learning and improvement ~ Framework Phases

Tools Provided The VSP Tool (Values, Structure, Processes) The Community Toolbox Hexagon Exploration Tool Pentagon Tool Green Light, Yellow Light, & Red Light Adaptations Program Fidelity & Adaptation Plan Template Experiential Learning Tips & Tools Intelligent Failure & Learning Loops Worksheet Tools Provided

Framework Phases & Steps to Follow

Key Effective Elements of Crime Prevention Programming A Review of 31 Evidence-Based Programs

Review of 31 Evidence-Based Programs Programs in Indigenous Communities (N=3) Programs for Children Ages 6 to 11 (N=13) Programs for Youth Ages 12 to 17 (N=12) Programs for Addressing Youth Gang Involvement (N=3) Programs for Offenders (N=1) Review of 31 Evidence-Based Programs

Coding Framework 1 Main objectives of the intervention 2 Target risk and protective factors 3 Definition & description 4 Universal vs. targeted program 5 Target population 6 Setting 7 Level of intervention 8 Components 9 Specifics of delivery 10 Dosage 11 Duration 12 Evaluations methods and findings 13 References 14 Existing adaptations Coding Framework

Key Effective Elements Review of findings are presented - a comprehensive table of each evidence-based program reviewed is provided Broader themes in crime prevention literature are highlighted – importance of target population (the right audience), dosage (the right amount), duration (for the right period of time) Recent trends are discussed: from prevention science to strength-based approaches and trauma-informed services Key Effective Elements

Case Studies Applying the Model

Case Example 1:  Local Adaptation of the OJJDP Comprehensive Gang Model (Spergel Model) in a Canadian City Case Example 2:  Local Adaptation of Multisystemic Therapy (MST) for Indigenous Youth Case Studies

Putting Knowledge into Practice Final Toolkit Putting Knowledge into Practice

Melanie Bania, PhD Community-Engaged Consulting mel@melaniebania.ca Contact Information