Treatment Integrity and Program Fidelity: Necessary but Not Sufficient to Sustain Programs Ronnie Detrich Randy Keyworth Jack States Wing Institute.

Slides:

Advertisements

Similar presentations

Vs. Attending a Different Training as a Site Team.

Advertisements

Using Student Data as a Basis for Feedback to Teachers Ronnie Detrich Wing Institute Cal-ABA, 2011.

Instructional Technology vs. Educational Technology

Teacher Preparation and Education Reform: A Behavioral Systems Perspective Çhair: Ronnie Detrich, Wing Institute Discussant: Chuck Salzberg, Utah State.

School-wide Positive Behavioral Interventions and Supports & Students with Autism Jointly developed by the above organizations with funding from the U.S.

School Leadership: A Lynchpin for Improving Instructional Outcomes Ronnie Detrich, Wing Institute: Chair Trina Spencer, Northern Arizona University: Discussant.

Schoolwide Positive Behavior Interventions and Support -SWPBIS- Mitchell L. Yell, Ph.D. University of South Carolina

1 Implementing a Three-Tiered State Evaluation Structure Bob Putnam The May Institute Karen Childs University of South Florida 2009 National PBIS Leadership.

July 2007 IDEA Partnership 1 RTI Process What is it?

Treatment Integrity: A Fundamental Component of PBS Ronnie Detrich Wing Institute.

Evaluation is a professional and ethical responsibility and is a core part of PHN professional practice Commitment to evaluation helps build the PHN intelligence.

Positive Behavioral Intervention and Support (PBIS) A New Way to Manage By: Annie Schave and Emily Smith.

SAFA- IFAC Regional SMP Forum

National Center on Response to Intervention Developed by the National Center on Response to Intervention and RMC Research RTI Integrity Framework: A Tool.

Problem Solving Model Problem Solving Model NC DPI Summer Preparation Preparation & Implementation Implementation North Carolina.

Oregon Response to Intervention Vision: Every child in every district receives the instruction that they need and deserve…every day. Instructional Coaching:

1 A Behavior and Reading Improvement Center Presentation Integrating Academic and Behavior Support Richard White and Bob Algozzine Integrated Systems for.

Moving from Development to Efficacy & Intervention Fidelity Topics National Center for Special Education Research Grantee Meeting: June 28, 2010.

Sustainability Through the Looking Glass: Shifting Contingencies Across Levels of a System Jack States Randy Keyworth Ronnie Detrich 34th Annual Convention.

Response to Intervention USING RTI FOR NONACADEMIC INTERVENTIONS: PART II.

Association for Behavior Analysis Conference Sustainable Programs: In Search of the Elusive Randy Keyworth Ronnie Detrich Jack States.

Supporting and Evaluating Broad Scale Implementation of Positive Behavior Support Teri Lewis-Palmer University of Oregon.

Specific Learning Disability: Accurate, Defensible, & Compliant Identification Mississippi Department of Education.

The Seventh Annual Hawaii International Conference on Education Sustainability: Implementing Programs That Survive and Thrive Randy Keyworth Jack States.

Implementation Science Vision 21: Linking Systems of Care June 2015 Lyman Legters.

Association for Behavior Analysis International 36 th Annual Convention Teacher Induction: Where the Rubber Meets the Road Randy Keyworth.

Coaches Training Introduction Data Systems and Fidelity.

Toward a Technology of Treatment Integrity Ronnie Detrich Wing Institute APBS, Denver, Colorado 2011.

INTERNATIONAL SOCIETY FOR TECHNOLOGY IN EDUCATION working together to improve education with technology Using Evidence for Educational Technology Success.

Positive Behavioral Interventions & Supports (PBIS) Core Behavioral Component The Response to Intervention Best Practices Institute Wrightsville Beach,

Randomized Clinical Trials: The Versatility and Malleability of the “Gold Standard” Wing Institute Jack States Ronnie Detrich Randy Keyworth “Get your.

MiBLSi Schools’ Implementation Process and Student Outcomes Anna L. Harms Michigan State University MiBLSi State Conference

Critical Elements Effective teaching Alignment of curriculum and instruction to standards Instructional program coherence Fidelity of implementation Evaluation.

PBIS Meeting for BCPS Team Leaders and Coaches March 14, 2008 Oregon Ridge.

Educable Mental Retardation as a Disability: Accurate, Defensible, & Compliant Identification Mississippi Department of Education.

Getting There from Here: Creating an Evidence- Based Culture Within Special Education Ronnie Detrich Randy Keyworth Jack States.

Positive Behavior Intervention and Supports: A Brief Introduction.

Evidence-based Education and the Culture of Special Education Chair: Jack States, Wing Institute Discussant: Teri Palmer, University of Oregon.

Suggested Components of a Schoolwide Reading Plan Part 1: Introduction Provides an overview of key components of reading plan. Part 2: Component details.

Spartan Expectations Be Responsible  Return promptly from breaks  Be an active participant  Use the law of two feet Be Respectful  Maintain cell phone.

Data-based Decision Making: Basics OSEP Center on Positive Behavioral Interventions & Supports February 2006

Data-Based Decision Making: Using Data to Improve Implementation Fidelity & Outcomes.

Treatment Integrity: Necessary but Not Sufficient for Improving Outcomes Ronnie Detrich Wing Institute.

THOUGHTFUL SUSTAINABILITY Teri Lewis Oregon State University Ronnie Detrich Wing Institute David StandifordOregon State University ABAI, San Antonio, Texas.

An Expanded Model of Evidence-based Practice in Special Education Randy Keyworth Jack States Ronnie Detrich Wing Institute.

Introduction to School-wide Positive Behavior Support.

Treatment Integrity Degree to which something is implemented as designed, intended, planned: –Delivery of instruction/intervention –Formative evaluation.

Effective Behavior & Instructional Support. Implementing RTI through Effective Behavior & Instructional Support.

Bridging the Research to Practice Gap: Perspectives from the Practice Side Ronnie Detrich Wing Institute.

Evaluation Research Dr. Guerette. Introduction Evaluation Research – Evaluation Research – The purpose is to evaluate the impact of policies The purpose.

A Descriptive Approach to Measuring a School Culture Ronnie Detrich Wing Institute Cal-ABA, 2012.

Using Student Data as a Basis for Feedback to Teachers Ronnie Detrich Wing Institute ABAI, 2011.

Care Enough to Count: Measuring Teacher Performance Ronnie Detrich Wing Institute.

EDF 593: Tier 3 RTI Instruction Rachel Brown-Chidsey, Ph.D

Cal-ABA 27th Annual Western Regional Conference Building a Data-based Decision Making Culture through Performance Management Randy Keyworth Ronnie Detrich.

Cal-ABA 26th Annual Western Regional Conference What We Know About Sustaining Programs? Randy Keyworth Ronnie Detrich Jack States.

Introduction to School-wide Positive Behavior Support.

V 2.1 Version 2.1 School-wide PBIS Tiered Fidelity Inventory.

Implementation and Sustainability in the US National EBP Project Gary R. Bond Dartmouth Psychiatric Research Center Lebanon, NH, USA May 27, 2014 CORE.

4.0 Introduction to Positive Behavioral Interventions and Supports (PBIS) District Cohort 1

Specific Learning Disability: Accurate, Defensible, & Compliant Identification Mississippi Department of Education.

SCHOOL-WIDE POSITIVE BEHAVIORAL INTERVENTIONS AND SUPPORT: ADDRESSING THE BEHAVIOR OF ALL STUDENTS Tier 2/3 Advanced Behavior Supports: Session 3 KENTUCKY.

Data-based Decision Making: More than the Data Ronnie Detrich Randy Keyworth Jack States Wing Institute Cal-ABA, March 2009.

School Climate Transformation Grants SEA Session October

Introduction to Promoting Positive Behavior in Schools:

RESEARCH IMPLEMENTATION PRACTICE

Response to Intervention (RtI) What is a Teacher’s Role?

Introduction to Coaching

Effect combined IMPACT on achieving outcomes Organizational OUTPUTS

Tier 2/Tier 3 Refresher Small Group.

Presentation transcript:

Treatment Integrity and Program Fidelity: Necessary but Not Sufficient to Sustain Programs Ronnie Detrich Randy Keyworth Jack States Wing Institute

Goals Describe the relationship between treatment integrity, program fidelity, and sustainability. Review what we know about treatment integrity and program fidelity. Review what we do not know about treatment integrity and program fidelity.

Relationship Between Integrity/Fidelity and Sustainability Programs and practices are a set of complex behaviors that are prescribed by policy and protocols. To make claims that a particular program or practice has sustained, it is necessary to know to what extent policy and protocols were followed. By default, failure to implement with integrity/fidelity means that the program or practice has not sustained.

What We Know about Treatment Integrity/Program Fidelity Most of what we know has been developed at the level of individuals implementing a specific behavior intervention or teaching plan.  The most common method for increasing and maintaining high treatment integrity is direct observation and feedback about implementation. Observation and feedback sessions are frequently completed by individuals who are external to the system. May be too intensive to be accomplished in many service settings.

What We Know about Treatment Integrity  If integrity checks are discontinued then integrity suffers. Apparently integrity checks are always necessary. Implies that it is necessary to have a system to assess the integrity of assessing integrity of implementation.

What We Do Not Know About Treatment Integrity How much integrity is enough to impact performance?  Some studies suggest that integrity can be “degraded” and still have positive impact on behavior (Northup, Fisher, Kahang, Harrel, & Kurtz,1997; Vollmer, Roane, Ringdahl, & Marcus,1999; Wilder, Atwell, & Wine, 2006). Does everyone have to be at criterion or is it sufficient that overall implementers are at criterion?

What We Do Not Know About Treatment Integrity What are relatively easy methods for increasing and maintaining integrity?  Performance feedback systems are most common but may be too resource and time intensive for many large systems. How often do we need to assess integrity to maintain adequate levels? How to incorporate routine treatment integrity checks into an organization?

What We May Know about Program Fidelity Program fidelity is used to describe when large “packages” are implemented.  Laws.  Policy.  Practices such as school-wide intervention.

What We May Know about Program Fidelity We know much less about fidelity of implementation at the large systems level.  Much of what we know is descriptive rather than experimental.  Measures are usually at large unit (school-wide).  Fidelity measures usually taken less often than with treatment integrity.

What We May Know About Program Fidelity We have many “hints” about what is important to assure fidelity.  Most is drawn from retrospective interviews with individuals from successful, sustainable programs.  Much of the quantitative data is correlational.

Two Examples Kovaleski, Gickling, Morrow, & Swank (1999).  Evaluated high vs low implementation of Instructional Support Teams (IST). School-wide organizational change. Students benefited from IST processes only when implemented with high fidelity. Implementing with low fidelity resulted in no better outcomes for students than control group not exposed to IST processes. Having structures in place was not sufficient to assure high fidelity. Fidelity assessed one time per year.

Two Examples Horner (2005).  Effect of high fidelity vs low fidelity on office discipline referrals. Schools that had high fidelity had 25% fewer office referrals for major rule violations than schools that did not meet fidelity criterion. Fidelity measures taken 2 times per year.

What We Do Not Know About Program Fidelity With broad packages such as school-wide positive behavior support what are the core components of a program that have to be in place to have effect? What is the necessary level of fidelity to effect performance? How often is it necessary to assess program fidelity?  Can we decrease the frequency of checks over time?

Conclusions Integrity/fidelity is necessary to achieve sustainable programs but not sufficient. There is much more that we do not know than we do know about both treatment integrity and program fidelity, especially as it relates to sustainable programs. Viable, sustainable programs have to adapt to changing circumstances while maintaining integrity and fidelity.

Thank you Copies of these presentations can be downloaded from