Treatment Integrity

Treatment Integrity Implementing treatment requires human judgments Example: A teacher is supposed to give a token to a student every 5 min (IV) if the child does not engage in problem behavior (DV). Does the teacher actually give the tokens on time and only when the child behaves correctly? Example: A therapist is supposed to deliver verbal praise (IV) to a child every time the child correctly matches words to pictures (DV). Does the therapist deliver praise every time? What exactly does “praise” consist of?

Treatment Integrity Treatment integrity: the extent to which the IV was implemented as the experimenter intended AKA procedural integrity, independent variable integrity, procedural fidelity (and other variations) Failure to assess treatment integrity poses threats to: Internal validity If an effect is observed, how do we know it is due to the IV? If an effect is not observed, it may have been because tx was implemented incorrectly External validity Treatments that are not described specifically cannot be accurately replicated But if we did not demo exp control—so the tx appears ineffective, we can’t determine or analyze failure of the tx—was it that the tx was not effective or that the IV was not implemented correctly? If our IV was not implemented correctly, others who replicate the study and implement it correctly are likely to get very different results

Treatment Integrity The “curious double standard” in ABA research: We operationally define and measure the DV but not the IV Gresham et al. (1993) found that Only 1/3 of the studies in JABA between 1980 and 1990 provided op defs of the IV Only 15% reported measuring the IV Why? Normally, IV is not measured—people are trained and the DV is measured. So, for example…we say that we implemented habit reversal We’re so careful about the measurement of DV, but we infer about the IV. In beh analysis, we want to make sure that changes in target behaviors are due soley to the treatment we implement. To do this we need to precisely measure the DV as we’ve talked about this semester, but we also need to demo precise control of the IV. Both of these are necessary to conclude that there is a relationship between the 2.

How do we ensure treatment integrity? Measure the IV as we do the DV Sometimes called “manipulation checks” Operationally define the IV in terms of When Where What is said What is done Collect data on correct implementation of tx How often? 25% of sessions is good standard Collect IOA on tx integrity during 25% of these Provide scripts and easy-to-follow protocols Train therapists well! Record IVs-sim to collecting data on DV, collect on IV Create op defs for the tx components. Come up with a way to objectively measure the components of tx. Could just record tx behs right along w/ Dep measure—show FA and TD Train-the people implementing tx must be trained well.

Example (Mace, Page, Ivancic, and O’Brien, 1986) Immediately following the occurrence of a target behavior… The therapist said, “No, go to time out”… Led the child by the arm to a pre-positioned time-out chair Seated the child facing the corner Tx drift-just like observers drift from the op def, therapists may also drift from the def of the IV—how they implement the tx. Exp bias-observer may…instead of what actually was done Reactivity-If checking for correct implementation of the IV, the therapist might… Same procedures for preventing these: Spot check!, observe from unobtrusive location, reinforce for accurate recording, booster training for therapists throughout.

Tx drift-just like observers drift from the op def, therapists may also drift from the def of the IV—how they implement the tx. Exp bias-observer may…instead of what actually was done Reactivity-If checking for correct implementation of the IV, the therapist might… Same procedures for preventing these: Spot check!, observe from unobtrusive location, reinforce for accurate recording, booster training for therapists throughout.

Tx drift-just like observers drift from the op def, therapists may also drift from the def of the IV—how they implement the tx. Exp bias-observer may…instead of what actually was done Reactivity-If checking for correct implementation of the IV, the therapist might… Same procedures for preventing these: Spot check!, observe from unobtrusive location, reinforce for accurate recording, booster training for therapists throughout.

Tx drift-just like observers drift from the op def, therapists may also drift from the def of the IV—how they implement the tx. Exp bias-observer may…instead of what actually was done Reactivity-If checking for correct implementation of the IV, the therapist might… Same procedures for preventing these: Spot check!, observe from unobtrusive location, reinforce for accurate recording, booster training for therapists throughout.

Tx drift-just like observers drift from the op def, therapists may also drift from the def of the IV—how they implement the tx. Exp bias-observer may…instead of what actually was done Reactivity-If checking for correct implementation of the IV, the therapist might… Same procedures for preventing these: Spot check!, observe from unobtrusive location, reinforce for accurate recording, booster training for therapists throughout.

Tx drift-just like observers drift from the op def, therapists may also drift from the def of the IV—how they implement the tx. Exp bias-observer may…instead of what actually was done Reactivity-If checking for correct implementation of the IV, the therapist might… Same procedures for preventing these: Spot check!, observe from unobtrusive location, reinforce for accurate recording, booster training for therapists throughout.

Tx drift-just like observers drift from the op def, therapists may also drift from the def of the IV—how they implement the tx. Exp bias-observer may…instead of what actually was done Reactivity-If checking for correct implementation of the IV, the therapist might… Same procedures for preventing these: Spot check!, observe from unobtrusive location, reinforce for accurate recording, booster training for therapists throughout.