Sustainability Through the Looking Glass: Shifting Contingencies Across Levels of a System Jack States Randy Keyworth Ronnie Detrich

Sustainable Practice A practice that has been adopted, implemented, and reliably maintained over time and generations of practitioners and decision makers. aaverage life of an education innovation is months (Latham, 1988)

Why Initiatives Fail? Political support Funding Faculty commitment Insufficient training Leadership Stability Faculty turnover Sustained professional development Competing reforms Model specificity Positive student outcomes … no one risk is statistically significant … combinations of risk factors “ Sustainability: Examining the Survival of Schools ’ Comprehensive School Reform Efforts ” American Institute for Research Schools ’ past & current performance

What do we know about the science of “ sustainable ” implementation? aMaximize the variables that support implementation of the intervention aMinimize the variables that oppose implementation of the intervention aAlign the contingencies at all levels of the system aRecognize and manage the different contingencies that exist at the different stages of implementation

What do we know about maximizing and minimizing the variables? What do we know about maximizing and minimizing the variables? Manage Performance Implementation is a social / cultural change process  As a social process, success often has very little to do with the details or merits of the actual innovation … aWhich is why poor innovations are adopted more frequently than good innovations

How do we Manage Performance? OBM is Demonstrating That Managing the Contingencies for Groups is Much Like Managing Contingencies for Individuals Contingencies: a contingent relationship between an individual’s behavior and the outcome of that behavior that affects the practice’s subsequent probability. Example: aBehavior: Student A failing reading aIntervention: Individual tutoring after school aOutcome: Student A achieves grade level on reading test Meta-contingencies: a contingent relationship between a cultural practice and an outcome of that practice that affects the practice’s subsequent probability. Example: aPerformance: High Stakes test results reveal students from School A are not meeting the minimum standards in reading aIntervention: Align the schools curriculum with the test standards aOutcome: Students enrolled in School A improve reading performance to minimum state standards

Achieving Sustainable Implementation for the Great Numbers Requires System Change “Build and manage an effective organization” “Hire and manage people who will be effective in doing jobs within the organization.”

CORE COMPONENTS OF IMPLEMENTATION Establish OUTCOMES, GOALS, AND MEASURES aSelect the relevant goals (program, practice, fiscal, staffing) aEstablish objective and measurable outcomes and align levels Employ and Align PERFORMANCE MANAGEMENT strategies aBuild activities and systems –Recruitment and hiring –Performance expectations –Training –Consultation and coaching –Feedback and evaluation aManage contingencies Conduct frequent and on-going MONITORING aOutcome and process aAssure program fidelity (program level) and treatment integrity (practice level) Be prepared to ADAPT AND INNOVATE aEvidence-based practices aData-based decision making

Monitoring Monitoring involves observing a behavior for any changes that my occur over time, or for effects an intervention may have on the observed behavior. aEvaluate success of the program or intervention against goals aAssess program fidelity and treatment integrity Monitoring generates information that is essential when making data- based decisions. aOutcome measures aProcess measures Monitoring should occur at all levels of implementation: aOrganizational (implementation) aPractitioner (intervention) aConsumer

Cultural Practices School A: Requires ten new vocabulary words to be introduced each week, to be used and spelled correctly in writing assignments. Individual Behavior Teacher A: Has parents review spelling test results with child Contingency Alignment Desired Outcome School exceeds state expectations for student spelling Individual Behavior and Cultural Practices

Teacher School District StateFederal Student Education Outcomes Achieved Contingency Alignment Levels of Implementation

Teacher School District State Federal Desired Outcomes Unlikely RtI Is implemented piece meal and sporadically across the state Non-Alignment IDIEA permits use of RtI California has no mandate or requirements for RtI A committee to study RtI has been formed - no policy School special ed staff form team to review evidence based practices Does not believe that science is best means to judge effectiveness of practices Level of RtI Implementation California

Alignment of Core Implementation Components Across Levels Example: Improved math scores FederalStateDistrictSchoolClassroom Outcomes and Goals Performance Management Policy & Regulations: funding for schools meeting minimum standards Regulations: Teacher credential require math training Hiring: Recruitment of teachers with math credentials to teach math Training: New math teachers receive training from “master math teachers” Feedback: Master teacher provides coaching feedback on teacher performance Monitoring Federal Govt. collects data from states on student math scores Teacher credential programs include math requirements Teachers hired meet the credential standards Teachers receive training and demonstrate competency Monitor treatment integrity and test scores Adaptation

Why Do So Many Programs Fail? They do not plan for and address the different contingencies required of each stage

Full Operation Program Installation Initial Implementation Adoption & Exploration Long Term Operation Desired Outcome Sustainability Stages Of Implementation Over Time and Over Generations

Stages Of Implementation 1.Adoption & Exploration aAssess the fit aUtilize data based decision-making / consensus building aAchieve cultural integration 2.Program Installation (start-up) aCreate new outcome expectations aEstablish and/or redesign the infrastructure aConstruct or redesign reporting systems (monitoring) aEnsure funding streams aDevelop or modify policies aHR strategic development aTrain and/or retrain personnel 3.Initial Implementation (performance change) aInitiate new practices and performance aTroubleshoot obstacles aPerformance feedback systems operational aAdapt systems and apply to novel situations

Stages of Implementation 4.Full Operation (Integration) aHR functions operating (staffing and re-staffing) aTraining and re-training aTreatment provided proficiently and with integrity aManagers supports and facilitates the new practice aPolicies and procedures adapted to experience aPerformance management systems support outcomes aReporting systems functioning (monitoring) 5.Long-Term Operation aPractice refined and treatment integrity of practice maintained aUndesirable drift controlled aInnovations adopted and incorporated into the practice aCore practices and outcomes monitored and effectiveness sustained over time

Types of Change Planned and Formal 1.Adaptation: Changes to an intervention that effectively address issues unique to the operating environment including assuring a cultural fit. These changes do not alter the core components of the practice and do not modify targeted outcome(s). Adoption, Program Installation, and/or Initial Implementation. 2.Innovation: Changes that offer opportunities that improve and expand upon an intervention above what has been achieved by current practices and procedures. Innovations to a practice should not be attempted before treatment integrity is first attained. Full operation and/or Long-term operation Unplanned and Informal 1.Drift: Undesirable changes that are identified as threats to the treatment integrity of the practice as defined by the core practices and outcome(s). Full operation and/or Long-term operation

Core Components CORE COMPONENTS: The essential elements necessary to achieve the desired outcomes for the consumer. aNot knowing the core components leads to time and resources wasted on attempting to implement a variety of non-essential elements. aKnowing the core components are essential to answering critical questions required for adaptation. Examples 1.Differential reinforcement 2.Phonics instruction 3.Reading fluency 4.Systematic de-sensitization 5.Coaching 6.Use of evidence-based practices

Practices DEFINITION PRACTICES: Skills, techniques, and strategies, that can be used by practitioners for interventions that benefit consumers. Such practices reliably produce desirable effects and can be used individually or in combination to form more complex practices and/or programs. Examples 1.Direct Instruction (DI) 2.Positive Behavior Interventions (PBI) 3.Good Behavior Game 4.RtI 5.New Chance, (for young welfare mothers) 6.Teaching family model

Programs DEFINITION PROGRAMS: Collections of practices for a defined consumer base implemented through an organizational structure using a specific philosophy, value system, service delivery structure, particular funding sources, and core practice components. Programs represent an efficient method to translate practices into effective treatment outcomes. 1.Boys Town 2.Morningside Academy 3.Institute for Effective Education 4.Spectrum Center 5.Oakland Unified School District

Programs - Practices - Components ProgramMorningside Academy Institute for Effective Education Bethel School District PracticeSopris West's Educational Series Direct Instruction PBIS ComponentFluencyPhonicsSchool-wide rules

Sustainable Implementation Requires Successful Aligning and Management of the Contingencies 1.Across levels of the system (Fed, State, district, school, classroom) 2.Across the core components of implementation (goals, performance management systems, monitoring, and innovations) 3.Across the stages of implementation (adoption, installation, initial implementation, full operation, long-term) 4.Across the Intervention (program, practices and core components)