1. A Meta-Analysis of Video Modeling Interventions that Teach Employment Related Skills to Individuals with Autism Carol Sparber, M.Ed. Intervention Specialist Kent State University
Abstract
Results
Strength of Intervention
This meta-analysis examines single subject studies that used video modeling (VM) interventions for teaching employment related skills to adolescents with Autism. Three indices of non-overlapping data (PND, PEM, and IRD) were used to examine each study for effectiveness of intervention and maintained skill outcomes. With trends in special education moving toward inclusion, there is a need to provide practitioners with practical and effective evidence based instructional practices.
Analysis of Video Modeling Interventions
Citation
Intervention
PND
PEM
IRD
Strength of
effect rated
(max. pts. possible: 7)
Kellems & Morningstar (2012)
Video modeling w/ iPod
1.00
(3pts)
(2pts)
7 pts
Smith, Ayres, Mechling, & Smith (2013)
Video modeling w/ iPhone
0.90
0.77
Allen, Wallace, Greene, Bowen, & Burke (2010)
Video modeling
0.81
0.93
0.65
(1pts)
5 pts
Allen, Wallace, Renes, Bowen, & Burke (2010)
0.39
(0pts)
*floor effect
0.78
(1pt)
0.68
2 pts
Goh & Bambara (2013)
0.00
0.22
0.08
0pts
Overall effect size for video modeling as the sole intervention.
* PND with floor effect not calculated into aggregate
0.90*
0.79
0.64
5pts
Yakubova & Taber-Doughty (2013)
Video modeling + prompting
7pts
Burke, Andersen, Bowen, Howard, & Allen (2010)
Video modeling + BSTraining & prompting
Bereznak, Ayres, Mechling, & Alexander (2012)
Video modeling + prompting on iPhone
Gog & Bambara (2013)
Video modeling + feedback & practice
0.76
0.63
Taber-Doughty, Miller, Shurr, & Wiles (2013)
Video modeling w/ iPad + least prompts
0.74
0.56
3pts
Overall effect size for Video Modeling + additional treatment
0.88
0.95
0.83
6pts
Criterion Measures Defined: 6-7 pts. large effect, 4-5 pts. moderate effect, 3 points or less, not effective
Percentage of Non-overlapping Data (PND) A calculation of non-overlap be-tween baseline & successive intervention phases (Scruggs, Mastropieri, & Casto, 1987)
PND > 90% highly effective (3pts)
PND >70% effective (2pts)
PND 50% - 70% questionable effectiveness (1pt)
PND < 50% ineffective (0pts)
Introduction
Percentage of Data Points Exceeding the Median (PEM) A calculation of percentage of data points exceeding the median of baseline phase (Ma, 2006)
PEM .90 to 1.0, highly effective (2pts)
PEM .70 to .90 moderately effective (1pt)
PEM Less than .70 questionable or not effective (0pt)
Employment is low for individuals with autism and they face many challenges in the work environment (Burke et al., 2010; Howlin, 2000; Wagner et al., 2006).
Video modeling can be effectively used to develop and strengthen academic performance, social, communication, and self-help skills and has strong potential for job skill training in employment settings (Bellini & Akullian, 2007; Goh & Bambara, 2013; Van Laahoven et al., 2009).
Video modeling combines observational learning and the unique strengths of individuals with ASD to respond to visual instruction resulting in strong maintenance effects and good generalization across novel tasks and settings (Bellini & Akullian, 2007; Buggey, 2005; Cihak, 2008)
Video modeling offers an efficient and effective intervention thus reducing time needed for on the job training resulting in quicker rates of skill acquisition, increased generalization and maintenance (Charlop-Christy, Le, & Freeman, 2000; Van Laahoven et al., 2012)
Effect sizes are necessary for single case studies in order to identify evidence-based interventions (Parker, Vannest, & Brown, 2009)
There is a need for improved rigor to support evidence-based interventions, with an “effect size” for more objective and reliable measures (Kazdin, 2008)
Few studies have been conducted to assess vocational interventions for adolescents and young adults with ASD so there is very little evidence available for specific vocational treatment approaches as individuals transition to adulthood (Taylor, 2012)
Improvement Rate Difference (IRD) Calculation of improvement rate differences between baseline and intervention phases (Parker, Vannest, & Brown, 2009)
IRD .70 to 1.0, large effect size (2pts)
IRD .50 to .70, moderate effect size (1pt)
IRD Less than .50, small or questionable effects, chance level improvement (0pts)
Conclusion
Results support the effectiveness of video modeling in teaching new job tasks.
Analysis of PND, PEM and IRD across all studies show the effect of video modeling interventions are not equally strong
PND in some studies were affected by floor and/or ceiling effects in baseline phase therefore PEM can provide a better interpretation of effect size
Video modeling interventions accompanied by additional treatments (i.e., prompting, practice, and/or feedback) generally resulted in a larger effect size
60% of the studies measured maintenance, of these, all reported high effect sizes
Overall metrics suggest integrating additional intervention treatments to improve the effectiveness of video modeling
Materials
Methods
Practical Implications & Future Studies
An electronic search was conducted for studies from 2003 to 2013 using the Educational Resources Information Center (ERIC), Ebsco, and PsycINFO.
Search terms included: ‘video modeling’, ‘video’, ‘ASD’, ‘autism’, ‘employment tasks’, ‘vocational settings’, and ‘employment skills’.
Inclusion Criteria:
Single case research with a multiple baseline design that demonstrated experimental control (Horner, et al., 2005)
Participants had a diagnosis of autism according to DSM-IV
Outcome measure targeted improved employment related skills
Independent variable was a form of video modeling or video modeling combined with additional treatment
Graphs for each study were examined and hand calculated for Percentage of Non-overlapping Data (PND), Percentage of Data Points Exceeding the Mean (PEM), and the Improvement Rate Difference (IRD) resulting in 202 calculations across 51 interventions with 29 participants.
Basic data unit of analysis was the comparison between baseline and intervention
Effect size for each study were recorded and rated to determine individual as well as overall effectiveness of video modeling interventions.
Interrater agreement 100%
Nonparametric data for each study was rated for effect size based on predetermined criteria (Scruggs et al., 1987; Ma, 2006; Parker et al., 2011)
Video modeling combined with other treatments are more likely to increase acquisition and maintenance of employment skills
Video modeling provides practitioners with an efficient, effective, evidence based intervention for improving job task outcomes for individuals with autism
Future single case studies should report non-overlapping calculations to inform practitioners of the relative strength or effectiveness of the intervention method.
Future research should investigate the instructional benefits of hand held devices (e.g., iPods and iPhones) for teaching employment skills to individuals with autism.