1. Treatment Integrity

2. Treatment Integrity Implementing treatment requires human judgments
Example: A teacher is supposed to give a token to a student every 5 min (IV) if the child does not engage in problem behavior (DV). Does the teacher actually give the tokens on time and only when the child behaves correctly?
Example: A therapist is supposed to deliver verbal praise (IV) to a child every time the child correctly matches words to pictures (DV). Does the therapist deliver praise every time? What exactly does “praise” consist of?

3. Treatment Integrity
Treatment integrity: the extent to which the IV was implemented as the experimenter intended
AKA procedural integrity, independent variable integrity, procedural fidelity (and other variations)
Failure to assess treatment integrity poses threats to:
Internal validity
If an effect is observed, how do we know it is due to the IV?
If an effect is not observed, it may have been because tx was implemented incorrectly
External validity
Treatments that are not described specifically cannot be accurately replicated
But if we did not demo exp control—so the tx appears ineffective, we can’t determine or analyze failure of the tx—was it that the tx was not effective or that the IV was not implemented correctly?
If our IV was not implemented correctly, others who replicate the study and implement it correctly are likely to get very different results

4. Treatment Integrity The “curious double standard” in ABA research:
We operationally define and measure the DV but not the IV
Gresham et al. (1993) found that
Only 1/3 of the studies in JABA between 1980 and 1990 provided op defs of the IV
Only 15% reported measuring the IV
Why?
Normally, IV is not measured—people are trained and the DV is measured. So, for example…we say that we implemented habit reversal
We’re so careful about the measurement of DV, but we infer about the IV.
In beh analysis, we want to make sure that changes in target behaviors are due soley to the treatment we implement. To do this we need to precisely measure the DV as we’ve talked about this semester, but we also need to demo precise control of the IV. Both of these are necessary to conclude that there is a relationship between the 2.

5. How do we ensure treatment integrity?
Measure the IV as we do the DV
Sometimes called “manipulation checks”
Operationally define the IV in terms of
When
Where
What is said
What is done
Collect data on correct implementation of tx
How often? 25% of sessions is good standard
Collect IOA on tx integrity during 25% of these
Provide scripts and easy-to-follow protocols
Train therapists well!
Record IVs-sim to collecting data on DV, collect on IV
Create op defs for the tx components. Come up with a way to objectively measure the components of tx. Could just record tx behs right along w/ Dep measure—show FA and TD
Train-the people implementing tx must be trained well.

6. Example (Mace, Page, Ivancic, and O’Brien, 1986)
Immediately following the occurrence of a target behavior…
The therapist said, “No, go to time out”…
Led the child by the arm to a pre-positioned time-out chair
Seated the child facing the corner
Tx drift-just like observers drift from the op def, therapists may also drift from the def of the IV—how they implement the tx.
Exp bias-observer may…instead of what actually was done
Reactivity-If checking for correct implementation of the IV, the therapist might…
Same procedures for preventing these: Spot check!, observe from unobtrusive location, reinforce for accurate recording, booster training for therapists throughout.

9. Tx drift-just like observers drift from the op def, therapists may also drift from the def of the IV—how they implement the tx.
Exp bias-observer may…instead of what actually was done
Reactivity-If checking for correct implementation of the IV, the therapist might…
Same procedures for preventing these: Spot check!, observe from unobtrusive location, reinforce for accurate recording, booster training for therapists throughout.

10. Tx drift-just like observers drift from the op def, therapists may also drift from the def of the IV—how they implement the tx.
Exp bias-observer may…instead of what actually was done
Reactivity-If checking for correct implementation of the IV, the therapist might…
Same procedures for preventing these: Spot check!, observe from unobtrusive location, reinforce for accurate recording, booster training for therapists throughout.

12. Tx drift-just like observers drift from the op def, therapists may also drift from the def of the IV—how they implement the tx.
Exp bias-observer may…instead of what actually was done
Reactivity-If checking for correct implementation of the IV, the therapist might…
Same procedures for preventing these: Spot check!, observe from unobtrusive location, reinforce for accurate recording, booster training for therapists throughout.

13. Tx drift-just like observers drift from the op def, therapists may also drift from the def of the IV—how they implement the tx.
Exp bias-observer may…instead of what actually was done
Reactivity-If checking for correct implementation of the IV, the therapist might…
Same procedures for preventing these: Spot check!, observe from unobtrusive location, reinforce for accurate recording, booster training for therapists throughout.

14. Tx drift-just like observers drift from the op def, therapists may also drift from the def of the IV—how they implement the tx.
Exp bias-observer may…instead of what actually was done
Reactivity-If checking for correct implementation of the IV, the therapist might…
Same procedures for preventing these: Spot check!, observe from unobtrusive location, reinforce for accurate recording, booster training for therapists throughout.

15. Tx drift-just like observers drift from the op def, therapists may also drift from the def of the IV—how they implement the tx.
Exp bias-observer may…instead of what actually was done
Reactivity-If checking for correct implementation of the IV, the therapist might…
Same procedures for preventing these: Spot check!, observe from unobtrusive location, reinforce for accurate recording, booster training for therapists throughout.