© 2016 Dr Mark A Stokes A meta-analysis of education interventions in children with High Functioning ASD Dr Mark Stokes Associate Professor Deakin University Authors: Stokes, M 1 ; Thomson, M 2 ; Dymond, S 2 ; Macmillan, C 1 ; Pecora, L 1 & Donaldson, E 2 1 Deakin University 2 Autism Teaching Institute

© 2016 Dr Mark A Stokes Background Four large US based systematic reviews undertaken: – NSP1 National Autism Center. (2009). National standards project findings and conclusions. – NSP2 Rue et al., (2015). National Standards Project: addressing the need for evidence-based practice guidelines for autism spectrum disorder – National Autism Center; Randolph, Massachusetts – EBP National Professional Development Center – EBP2 Wong et al. (2014). Evidence-Based Practices for Children, Youth, and Young Adults with Autism Spectrum Disorder – Autism Evidence-Based Practice Review Group, Frank Porter Graham Child Development Institute, University of North Carolina at Chapel Hill Other Reviews – i.e.: Odom et al. (2010). Evidence-Based Practices in Interventions for Children and Youth with Autism Spectrum Disorders, Preventing School Failure: Alternative Education for Children and Youth, 54,

© 2016 Dr Mark A Stokes Background – Evidence Based Practices EBP1 – Reviewed 175 papers EBP2 – Included 29,105 papers and reviewed 456 papers 159 reviewers included 542 participants

© 2016 Dr Mark A Stokes Background – EBP2 Exclusion criteria – Dx of ASD was unclear – Participants over 22 y/o – Participants with co-morbidities – Intervention was biomedical – Papers lacked empirical data – Papers focused on moderating or mediating variables Inclusion criteria – English papers published in between 1991 – 2011 in a peer-reviewed journal – Intervention was behavioral, developmental, or academic in nature – Intervention could be implemented in typical educational setting (school, home, community) – Full spectrum ID & HF ASD

© 2016 Dr Mark A Stokes Background – National Standards Project 1 7,115 abstracts – searches and literature reviews, & nominated by experts 5,978 removed 1137 reviewed 724 retained N unclear

© 2016 Dr Mark A Stokes Background – NSP1 Exclusion criteria – Dx of ASD was unclear – Participants over 22 y/o – Participants with co-morbidities – Intervention was biomedical – Papers lacked empirical data – Papers focused on moderating or mediating variables Inclusion criteria – English papers published in between 1992 – 2007 in a peer-reviewed journal – Intervention could be implemented by schools, or early intervention, home-, hospital-, vocational- or community-based programs or in clinic settings – Included ID & HF ASD

© 2016 Dr Mark A Stokes Background – National Standards Project 2 Included 2,705 abstracts and included 351 papers (+27 focusing on adults) – 27 expert reviewers – N unclear

© 2016 Dr Mark A Stokes Background – NSP2 Exclusion criteria – Dx of ASD was unclear – Participants over 22 y/o – Participants with co-morbidities – Intervention was biomedical – Papers lacked empirical data – Papers focused on moderating or mediating variables Inclusion criteria – English papers published in between 2007 – 2012 in a peer-reviewed journal – Intervention could be implemented by schools, or early intervention, home-, hospital-, vocational- or community-based programs or in clinic settings – Included ID & HF ASD

© 2016 Dr Mark A Stokes Background – Reliability EBP1 – Not available EBP2 – Inter Rater Agreement collected for only 41% of papers – 74% for group design 77% for single case design NSP1 & 2 – Pilot team determined Inter Rater Agreement standard on two papers >80%. All reviewers were trained on these papers & required to meet criteria on training paper. No further assessment of reliability undertaken.

© 2016 Dr Mark A Stokes Background - Findings EBP1 found 24 interventions had some merit NSP1 found 11 interventions had some merit Considerable crossover between the two

© 2016 Dr Mark A Stokes Background – Findings EBP2 Effect sizes and sample sizes for each study not reported SCD – Single Case Design

© 2016 Dr Mark A Stokes Background – Findings NSP2 As for EBP studies, effect sizes not reported Majority of studies were behavioural

© 2016 Dr Mark A Stokes Background – Limitations Both NSP2 & EBP studies not focused upon HF-ASD – HF-ASD in general school settings Both included studies that could be implemented in non-school settings Raters Reliability was not above 80% in EBP & not assessed in NSP2 NSP1 & 2 do not report N NSP1 & 2 do not make it clear which papers support which results EBP1 & 2 studies had to have comparable evidence to be included EBP1 & 2, NSP1 & 2 don’t evaluate generalisability nor ecological validity None provide meta-analytic evaluation of findings

© 2016 Dr Mark A Stokes Method All literature (peer-reviewed journal articles, theses, and books) published from 2000 till January 2015 – Examined the impact of education-based interventions on students who have ASD (AS, AD, PDD-NOS, ASD) A measurable school-based intervention Formal diagnosis of HF ASD evidenced by IQ (>70) If study included non-ASD cases, and extraction was not possible, study was removed Pilot studies or theses later replicated or published in peer- reviewed publications were removed

© 2016 Dr Mark A Stokes Method 14,328 studies retrieved 13,754 studies excluded following title & abstract screening 444 studies excluded after full text screening 130 studies included in analysis 514 participants included, – 412 males, 66 female (6.24:1), 36 unspecified 734 participants excluded on criteria grounds Inter-rater reliability 83.7% for articles and 78.3% theses

© 2016 Dr Mark A Stokes Results – Academic Skills 36 investigations from 27 publications – 85 participants, M:F 6.7:1, mean IQ 10 pts lower (86.89) – Positive results from 32 investigations – Mean effect size Cohen’s d=1.28 with heterogeneity I 2 =26% – Modifying task demands to student competency – handwriting (Blakeley) – Self-regulated strategy development – planning (Leakan) – Adapted social story reading – reading comprehension (O’Connor)

© 2016 Dr Mark A Stokes Results – Social Cognition 31 investigations derived from 29 publications – 217 participants, M:F 7:1, mean IQ 4 pts higher (100.52) – 21 +ve investigations, 4 mixed, 6 no change investigations – Mean effect size Cohen’s d=1.41 with heterogeneity I 2 =67% – Three tiered social skills training – social competency (Copeland) – Teacher prompting (questions) – mental state inference (Kaland) – Social Competence Intervention – emotion recognition (Schmidt) – Cog. Beh. intervention – complex emotion recognition (Bauminger 2002)

© 2016 Dr Mark A Stokes Results – Verbal Skills 21 investigations derived from 16 publications – 36 participants (M:F 11:1), mean IQ 4.7 pts higher (101.2) – 15 +ve investigations, 5 mixed, 1 –ve investigations – Mean effect size Cohen’s d=1.69 with heterogeneity I 2 =0% – Disability awareness training – verbal interaction(Cook) – Video modelling – greeting (Kagohara) – Video modelling – verbalisations (Lydon) – Social Competence Intervention – Social communication (Schmidt)

© 2016 Dr Mark A Stokes Results On task behaviour – 39 investigations, derived from 34 publications – 83 participants (M:F 6.5:1) mean IQ 7.4 pts lower (89.2) – 34 +ve investigations, 4 mixed, 1 no change Play – 13 investigations, derived from 12 publications – 25 participants (M:F 7.3:1) mean IQ 6.2 pts lower (90.4) – 8 +ve investigations, 4 mixed, 1 no change Social interaction – 40 investigations, derived from 31 publications – 92 participants (M:F 6.7:1) mean IQ 5.6 pts higher (102.2) – 31 +ve investigations, 2 mixed, 7 no change

© 2016 Dr Mark A Stokes Discussion Improvements in – Academic skills depended upon tayloring the task to the child – Social Cognition depended upon targeted social interventions – Verbal skills depended upon modelling and improving others awareness – Most improvement in play depended upon modelling & behavioural reinforcement – Most improvement in social interaction depended on video modelling and social skills development – Many techniques resulted in maintenance, but little generalisation