A Meta-Analysis of Video Modeling Interventions that Teach Employment Related Skills to Individuals with Autism Carol Sparber, M.Ed. Intervention Specialist Kent State University Abstract Results Strength of Intervention This meta-analysis examines single subject studies that used video modeling (VM) interventions for teaching employment related skills to adolescents with Autism. Three indices of non-overlapping data (PND, PEM, and IRD) were used to examine each study for effectiveness of intervention and maintained skill outcomes. With trends in special education moving toward inclusion, there is a need to provide practitioners with practical and effective evidence based instructional practices. Analysis of Video Modeling Interventions Citation Intervention PND PEM IRD Strength of effect rated (max. pts. possible: 7) Kellems & Morningstar (2012) Video modeling w/ iPod 1.00 (3pts) (2pts) 7 pts Smith, Ayres, Mechling, & Smith (2013) Video modeling w/ iPhone 0.90 0.77 Allen, Wallace, Greene, Bowen, & Burke (2010) Video modeling 0.81 0.93 0.65 (1pts) 5 pts Allen, Wallace, Renes, Bowen, & Burke (2010) 0.39 (0pts) *floor effect 0.78 (1pt) 0.68 2 pts Goh & Bambara (2013) 0.00 0.22 0.08 0pts Overall effect size for video modeling as the sole intervention. * PND with floor effect not calculated into aggregate 0.90* 0.79 0.64 5pts Yakubova & Taber-Doughty (2013) Video modeling + prompting 7pts Burke, Andersen, Bowen, Howard, & Allen (2010) Video modeling + BSTraining & prompting Bereznak, Ayres, Mechling, & Alexander (2012) Video modeling + prompting on iPhone Gog & Bambara (2013) Video modeling + feedback & practice 0.76 0.63 Taber-Doughty, Miller, Shurr, & Wiles (2013) Video modeling w/ iPad + least prompts 0.74 0.56 3pts Overall effect size for Video Modeling + additional treatment 0.88 0.95 0.83 6pts Criterion Measures Defined: 6-7 pts. large effect, 4-5 pts. moderate effect, 3 points or less, not effective Percentage of Non-overlapping Data (PND) A calculation of non-overlap be-tween baseline & successive intervention phases (Scruggs, Mastropieri, & Casto, 1987) PND > 90% highly effective (3pts) PND >70% effective (2pts) PND 50% - 70% questionable effectiveness (1pt) PND < 50% ineffective (0pts) Introduction Percentage of Data Points Exceeding the Median (PEM) A calculation of percentage of data points exceeding the median of baseline phase (Ma, 2006) PEM .90 to 1.0, highly effective (2pts) PEM .70 to .90 moderately effective (1pt) PEM Less than .70 questionable or not effective (0pt) Employment is low for individuals with autism and they face many challenges in the work environment (Burke et al., 2010; Howlin, 2000; Wagner et al., 2006). Video modeling can be effectively used to develop and strengthen academic performance, social, communication, and self-help skills and has strong potential for job skill training in employment settings (Bellini & Akullian, 2007; Goh & Bambara, 2013; Van Laahoven et al., 2009). Video modeling combines observational learning and the unique strengths of individuals with ASD to respond to visual instruction resulting in strong maintenance effects and good generalization across novel tasks and settings (Bellini & Akullian, 2007; Buggey, 2005; Cihak, 2008) Video modeling offers an efficient and effective intervention thus reducing time needed for on the job training resulting in quicker rates of skill acquisition, increased generalization and maintenance (Charlop-Christy, Le, & Freeman, 2000; Van Laahoven et al., 2012) Effect sizes are necessary for single case studies in order to identify evidence-based interventions (Parker, Vannest, & Brown, 2009) There is a need for improved rigor to support evidence-based interventions, with an “effect size” for more objective and reliable measures (Kazdin, 2008) Few studies have been conducted to assess vocational interventions for adolescents and young adults with ASD so there is very little evidence available for specific vocational treatment approaches as individuals transition to adulthood (Taylor, 2012) Improvement Rate Difference (IRD) Calculation of improvement rate differences between baseline and intervention phases (Parker, Vannest, & Brown, 2009) IRD .70 to 1.0, large effect size (2pts) IRD .50 to .70, moderate effect size (1pt) IRD Less than .50, small or questionable effects, chance level improvement (0pts) Conclusion Results support the effectiveness of video modeling in teaching new job tasks. Analysis of PND, PEM and IRD across all studies show the effect of video modeling interventions are not equally strong PND in some studies were affected by floor and/or ceiling effects in baseline phase therefore PEM can provide a better interpretation of effect size Video modeling interventions accompanied by additional treatments (i.e., prompting, practice, and/or feedback) generally resulted in a larger effect size 60% of the studies measured maintenance, of these, all reported high effect sizes Overall metrics suggest integrating additional intervention treatments to improve the effectiveness of video modeling Materials Methods Practical Implications & Future Studies An electronic search was conducted for studies from 2003 to 2013 using the Educational Resources Information Center (ERIC), Ebsco, and PsycINFO. Search terms included: ‘video modeling’, ‘video’, ‘ASD’, ‘autism’, ‘employment tasks’, ‘vocational settings’, and ‘employment skills’. Inclusion Criteria: Single case research with a multiple baseline design that demonstrated experimental control (Horner, et al., 2005) Participants had a diagnosis of autism according to DSM-IV Outcome measure targeted improved employment related skills Independent variable was a form of video modeling or video modeling combined with additional treatment Graphs for each study were examined and hand calculated for Percentage of Non-overlapping Data (PND), Percentage of Data Points Exceeding the Mean (PEM), and the Improvement Rate Difference (IRD) resulting in 202 calculations across 51 interventions with 29 participants. Basic data unit of analysis was the comparison between baseline and intervention Effect size for each study were recorded and rated to determine individual as well as overall effectiveness of video modeling interventions. Interrater agreement 100% Nonparametric data for each study was rated for effect size based on predetermined criteria (Scruggs et al., 1987; Ma, 2006; Parker et al., 2011) Video modeling combined with other treatments are more likely to increase acquisition and maintenance of employment skills Video modeling provides practitioners with an efficient, effective, evidence based intervention for improving job task outcomes for individuals with autism Future single case studies should report non-overlapping calculations to inform practitioners of the relative strength or effectiveness of the intervention method. Future research should investigate the instructional benefits of hand held devices (e.g., iPods and iPhones) for teaching employment skills to individuals with autism.