1. Evaluating Intervention Effects
Chapter Six
Evaluating Intervention Effects

2. OBJECTIVES
Summarize and graph or chart data using techniques appropriate for the data.
Visually analyze graphed data and write data decision rules.
Identify the major types of single subject research designs and give the uses and limitations of each.

3. Graphing and Charting Data
Why?
Continuous monitoring of performance, facilitate decision making
Formative evaluation of the effectiveness of instruction

4. Types
Graph
One or two dependent variables Bar, cumulative, frequency polygon, equal interval, equal ratio
Progress
Chart
Several variables monitored
Performance

5. Components of Graphs The effects of IV on DV for student Ordinate
Descriptive Title
Ordinate
Baseline
Intervention 1
Intervention 2
Condition label
Data path
Data Points
dependent variable
Condition Line
time
Abscissa
Legend
Billy =
Sally =

6. Graphing Conventions Do not connect non-consecutive points
Baseline
Reinforcement
Extinction
Label all phases and axes
dependent variable
Connect consecutive
data points
Use different symbols and plot lines for different data 0-
Do not connect across phase lines
time
Separate 0 from abscissa
Billy =
Sally =

7. Lines of Progress Trend lines Aim lines
Visual estimates of future performance
Aim lines
Visual representation of performance
Based on criteria and allotted time of STO
Line starts from last 3 days of baseline data
Accelerating and decelerating

8. Middle School Math: Revisited
Mean:
The average of a set of numbers
Mode:
The most frequent of a group of numbers
Median:
The mid point of a set of sequenced numbers 2
What is the Median of these numbers? 1, 2, 5
7.5
What is the Median of these numbers? 4, 7, 8, 10 1
What is the Median of these numbers? 1, 1, 998

9. Trend lines represent the trend of data within each condition
How to generate trend lines:
Count total data points
Draw vertical line to divide points in half
Mid-date: draw vertical line at mid-date point. (repeat on each side of vertical line)
Mid-rate: draw horizontal line at mid-rate
Connect intersections of mid-rates/dates in each phase

10. TREND LINE: STEP 1

11. TREND LINE: STEP 2

12. TREND LINE: STEP 3

13. TREND LINE: STEP 4

14. TREND LINE EXERCISE A

15. TREND LINE EXERCISE A - Solution

16. TREND LINE EXERCISE B

17. TREND LINE EXERCISE B - Solution

18. TREND LINE EXERCISE C

19. TREND LINE EXERCISE C - Solution

20. TREND LINE EXERCISE D

21. TREND LINE EXERCISE D - Solution

22. TREND LINE EXERCISE E

23. TREND LINE EXERCISE E - Solution

24. TREND LINE EXERCISE F

25. TREND LINE EXERCISE F - Solution

26. How to generate aim lines:
Aim lines represent the goal or objective and keep us on track toward that goal
How to generate aim lines:
Determine the aim date and aim rate based on the criteria expressed in the student’s long-term objective
Draw an aim star (an A right side up for accelerating target, an A upside down for decelerating target) at the desired rate and date intersection.

27. Determining Aim Lines
Baseline
Number Hand raises A • • • • • •
Time

28. Determining Aim Lines
Determine the mid-date and mid-rate of the LAST THREE DAYS OF BASELINE DATA POINTS.
Mid-date and mid-rate are the median or middlemost points. Don’t average the points, simply count the # and take the middle. Mid-date: count left to right. Mid-rate: count bottom to top.
Baseline
# hand raises A • • • • • •
Time

29. Determining Aim Lines
Draw an aim line through the mid-date and the mid-rate intersection to the aim star.
Baseline
# hand
raises A • • • • • •
Time

30. AIM LINE EXERCISE A
Baseline • • • •
Behavior •
Time

31. A AIM LINE EXERCISE A - Solution Baseline • • • • • Behavior • •
Time

32. AIM LINE EXERCISE B
Baseline
Behavior • • • A
Time

33. AIM LINE EXERCISE B - Solution
Baseline A
Behavior • • •
Time

34. AIM LINE EXERCISE C
Baseline
Behavior • • • • • •
Time

35. AIM LINE EXERCISE C - Solution
Baseline A
Behavior • • • • • •
Time

36. Objectives & Monitoring
Given the prompt, “point to the…,” Richie will point to the designated object within 5 secs for 10 of 10 trials over 10 consecutive sessions by the 7th day of intervention.
How will we measure progress toward this objective?
Event recording--controlled presentations (correct response w/in 5 secs).

37. Ritchie’s Progress 2 4 8 6 10 A
Correct Trials
Days

38. Using Data During Intervention
Data patterns (within and across conditions)
Variability
more stable = more predictive
look for cyclical patterns (e.g., only on Monday, when reading, etc.)
Level changes
indicate possible change in functional relationships or influencing factors
Trend changes
predict future performance
indicate possible change in functional relationship

39. NO CHANGE

40. CHANGE IN TREND

41. CHANGE IN LEVEL

42. CHANGE IN LEVEL AND TREND

43. Data-Based Decision Making

44. Data Decision Rules
Define adequate progress and dictate when changes are to be made
Determined before you intervene
Basic “three-day rule”

45. Intervention Data Patterns & Decisions
Decisions made
by comparing data with Aim Line
Make no change (data at or better than aim)
Change goal or aim date (break ddr)
Slice back (slight - misrule during instruction)
Step back (large - teach prerequisite)
Move to new procedure (add/fade prompts)
Move to new skill (next skill in hierarchy)
Begin compliance training (R+ and error correction)
Move to new phase of learning

46. Use Data to Analyze Errors
Need to know why errors occur to plan for future instruction or intervention
Random errors: irregular pattern, large discrepancy
Systematic errors: consistent, misrule
Compliance errors: variable, sharp turn-down
Unlearned prerequisites: no progress

47. DATA SHOWING NEED FOR COMPLIANCE TRAINING

48. DATA SHOWING NEED TO
MOVE TO NEW SKILL

49. DATA SHOWING THE NEED TO
SLICE BACK - RETEACH

50. DATA SHOWING THE NEED TO
STEP BACK - PRETEACH

51. Using Data After Intervention (post-maintenance)
Data used to:
Evaluate intervention effectiveness
Identify functional relationships between Independent Variable (IV) and Dependent Variable (DV)

52. Single Subject Research Designs

53. Single Subject Research Designs
Advantages
Student serves as his/her own control
High internal validity Clear demonstration of experimental control Control for threats to internal validity
Small n
Formative evaluation
Disadvantage
Low external validity-replications required

54. The Logic of Single Subject Research Designs
Baseline data are an assessment of the current level of performance, and provide a basis for predicting future performance
Data collected during intervention condition are an assessment of the effect of the intervention and a prediction of future performance
However, an A and a B condition do not demonstrate experimental control (a functional relationship)--competing explanations for the observed effects cannot be ruled out
Therefore, a control condition (in this case, baseline) is implemented to demonstrate that intervention was responsible for observed effects of IV on DV

55. Types of Designs Selection
Based upon the questions and the conditions of the setting
The most valid designs may not lend themselves to specific situations/settings
Symbols
A = baseline condition
B = a first intervention condition: manipulation of IV
C - Zx = successive manipulations of IV(s)

56. Case Study Designs - AB Baseline followed by intervention Example: “everyday teaching with data”
Advantages
Good for hypothesis/question development
Fits teaching model
Disadvantages
Cannot identify functional relationship
Many threats to internal validity
Cannot reverse if learning occurred

57. AB Design Baseline Intervention 180 150 120 Number of Seconds 90 60 30 
150        
120 
Number of Seconds 90 60 30         
Sessions

58. Changing Conditions (ABCD) Each phase contains a completely different Independent Variable
Advantages
Evaluate effects of several different interventions
Disadvantages
Doesn’t identify functional relationships
Ordering effects
Requires withdrawal of potentially effective intervention
Confounded by learning

59. Changing Conditions
100 . . . 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 . . . . . . . . . . . . . .
Goal . . . . . . . . . . . .
% Time on Task . . .
Assigned .
seats
Praise-
Rules, chair
arrangement,
Non-contingent
free time
Ignoring
Office
Threat
Jim tutors
& models
praise
Contingent Free Time
"No tutoring"
Day 28: Jim stops tutoring
(+ a party contingency on day 21)
Day 29: Complete reversal
continued
Fading or transfer
of data collection
Baseline
to teacher A B C D A
D/E
Observation Days (12:00-1:00p.m.)

60. Withdrawal (ABAB) Put IV in and take it away (repeat) -Most frequently used SSD in behavior mod. -Aka “replication” or “reversal” design
Advantages
Identifies functional relationships
Can be a teaching design
Disadvantages
Requires return to baseline
Confounded by learning

61. Withdrawal
Baseline
(A)
Time-Out
(B)
Baseline2
(A)2
Time-Out2
(B)2 20 . . .
. . 15 . . . . . . .
Number Per Day . . . 10 . . . . . . . . . . .
May skip first baseline if clear history of behavior or behavior is extremely dangerous 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Days

62. Changing Criterion -An AB design with the B sub-phases -Used to evaluate shaping programs of behavior toward a terminal objective
Advantages
No withdrawal or reversal required
Fits will into instructional program
Can establish a functional relationship
Disadvantages
Not for behaviors requiring immediate change
ID of appropriate criterion steps may be difficult

63. Changing Criterion Criterion Baseline Intervention 10 9 8 7 6 5 9    8     7 6       5
Criterion
Total Latency
(in minutes)  4     3
Vary length of condition and criterion level to demonstrate functional relationship   2              1
Sessions

64. Multiple Baseline Identify 3 distinct students, settings, or behaviors -collect data on each while implementing on only one at a time, using the others as control
Advantages
Not confounded by learning
Return to baseline not required
Can also be used as “multiple probe”
Disadvantages
Prolonged baselines
Don’t use if behavior can’t be tolerated

65. Multiple Baseline (across students)
Intervention
100 80 60 40 20   
Subject 1                            
100 80 60 40 20 
Percent Correct
Subject 2                     
100 80 60 40 20                    
Subject 3            

66. Multiple Baseline (across behaviors)
100 80 60 40 20 A B 
Rate Per Minute
Throwing
Hitting
Spitting
Sessions

67. Multiple Baseline (across settings)
Intervention
Latency
(in minutes)
Condition 1
Condition 2
Condition 3
Sessions 16 12 8 4 

68. Multiple Probe Across Behaviors
100 80 60 40 20      
(5)
(4) 
Word Group 1
(3)     
100 80 60 40 20      
(5)
(4)
PERCENT CORRECT
Word Group 2 
(1)
(3)    
100 80 60 40 20     
(5)
(4)
Word Group 3  
(1)
(2)
(3)     
DAYS

69. Alternating Treatments -Alternate number of IV through multiple sessions -Can follow with reversal design for one IV
Advantages
Demonstrate relative effectiveness of treatments
Minimize sequencing effects
Functional relationship if reversed with one variable
Disadvantages
Does not identify functional relationship
Treatments must be independent and not interactive

70. Alternating Treatments15 20 30 25 35 45 40 50 60 55 5 10   
= Intervention 1
= Intervention 2
= Intervention 3
Baseline
Intervention
Number of Events
Sessions