1. overview of visual analysis of single-case intervention data
Research on visual analysis, Advantages and limitations
Wendy Machalicek

2. Agenda
Origins of visual analysis in applied behavior analysis
Strengths and limitations of visual analysis
Suggestions for use

3. Why not pie charts, bar charts, a table?
Machalicek et al. in
preparation

4. Historical use of graphical displays
Statistical thinking rise in 18th and 19th century associated with rise in use of visuals
Early research on how humans perceive statistical graphics (Spence & Lewandowsky, 1990)
Preference for line graphs emerged as easy to read quickly and accurately

5. It began with skinner… “Where a reasonable degree of smoothness
and reproducibility can be obtained with a few
cases….there is little reason…to consider
larger numbers” (p 442, Skinner, 1938)
“The statistical approach is characterized by
relatively unrefined methods of measurement
and a general neglect of the problem of
direct description; [whereas] the non-statistical
approach confines itself to specific instances of
behavior and to the development of methods of direct
measurement and analysis (p. 443, Skinner, 1938)

6. Sidman, 1960

7. Defining Elements of Applied Behavior Analysis Baer, Wolf, & Risley, 1968
Applied (important dependent variable)
Behavioral (valid & reliable measurement)
Analytic (experimental design)
Technological (replicable procedures)
Conceptual (principles of behavior)
Effective (socially important effect sizes)
General (generalization and maintenance)
Social Validity (continued use)
If the application of
behavioral techniques
does not produce large
enough effects for
practical value,
then application has failed.

8. Advantages to use of visual analysis
Easy to create with current technology
Data intimacy. Transformed as little as possible.
Ideal for visually assessing interaction effects between two or more dimensions of behavior
Quick visualization of results over time when paired with x and y axis
When published, accessible
Parsonson, B. S., Baer, D. M., Kratochwill, T. R., & Levin, J. R. (1992). The visual analysis of data, and current research into the stimuli controlling it. Single-case research design and analysis: New directions for psychology and education, 15-4

9. Issues with visual analysis
Use of humans, rather than cumulative recorders to take data
Observer drift, complexity of data collection (Kazdin, 1977)
Percentage agreement insensitive to some target behavior (Kratochwill & Wetzel, 1977)
Inductive process can be obscured w/statistical analysis
Autocorrelation of data-inflated Type 1 errors (Busk & Marascuilo, 1988)
Type II errors-especially when degree of change is small

10. Contextual variables affecting visual analysis
Treatment acceptance (Spirrison & Maundy, 1994)
Participant characteristics (Grigg, Snell, & Loyd, 1989)
Social significance of effects (DeProspero & Cohen, 1979)
Charting and scaling of y-axis (Furlong & Wampold, 1982)

11. McDuffie, A. , Bullard, L. , Nelson, S. , Machalicek, W
McDuffie, A., Bullard, L., Nelson, S., Machalicek, W., & Abbeduto, L. (2016). A spoken language intervention for school-aged boys with fragile X syndrome. American Journal on Intellectual and Developmental Disabilities, 121(3),

13. Hansen, S. , Raulston, T. , Machalicek, W. , & Frantz, R. (In press)
Hansen, S., Raulston, T., Machalicek, W., & Frantz, R. (In press). Caregiver-mediated joint attention intervention. Behavioral Interventions.

14. Machalicek et al. in
preparation

15. How are we really doing?
IRA = .58 Ottenbacher et al. (1993)
Overall weighted mean proportion IRA = .76 for all 32 effects (Ninci et al, 2015)
Ninci only included peer reviewed studies, scales reduced for comparison, proportional effect sizes only. Potential moderators evaluated included (a) design families, (b) rater expertise, (c) the provision of contextual information for graphs, (d) the use of visual aids, (e) the provision of an operational definition of the construct being rated, and (f) rating scale ranges.
Potential moderators evaluated included (a) design families, (b) rater expertise, (c) the provision of contextual information for graphs, (d) the use of visual aids, (e) the provision of an operational definition of the construct being rated, and (f) rating scale ranges.

16. What impacts interrater reliability of visual analysis?

17. What do we know about accuracy & interrater agreement (IRA)?
Novice raters may have difficulty with variable data, trend or intervention effect is small and with extreme values (Nelson et al., 2017)
Raters may be more accurate with positive versus negative trends (Christ et al., 2014)
The use of visual aids may be the most concrete beneficial strategy we have thus far for improving IRA (e.g., Roane et al., 2013) (Ninci et al., 2015)
Ninci and colleagues suggested that experienced individuals may have variety of backgrounds and training in SCR with different areas of emphasis. It will be interesting to see the impact of additional training (e.g. WWC models) on IRA.

18. Risley, 2001

19. References
Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1(1),
Busk,PL.,&Marascuilo,L.A.(1988).Autocorrelation in single-subject research: A counterargument to the myth of no autocorrelation. Behavioral Assessment, 10,
Fisch, G.S. (1998). Visual inspection of data revisited: Do the eyes still have it? The Behavior Analyst, 21,
Hollands, J.G., & Spence, I. (1992). Judgments of change and proportion in graphical perception. Human Factors: The Journal of the Human Factors and Ergonomics Society, 34, 313.
Kratochwill, T. R., & Wetzel, R. J. (1977). Observer agreement, credibility, and judgement: Some considerations in presenting observer agreement data. Journal of Applied Behavior Analysis, 10,
Parsonson, B. S., Baer, D. M., Kratochwill, T. R., & Levin, J. R. (1992). The visual analysis of data, and current research into the stimuli controlling it. Single-case research design and analysis: New directions for psychology and education, 15-4
Spence, I., and Lewandowsky, S. (1991). Displaying proportions and percentages. Applied Cognitive Psychology, 5,61-77.