1. Negative Results in Single Case Research Erin E. Barton, PhD, BCBA-D

2. Review SCRD allows for an examination of what works, for whom, under what conditions Examination of negative results essential for identifying evidence based practices Rigor of study needs to be strong

3. Sensory-based Interventions Sensory-based interventions have gained popularity and are the most frequently requested intervention for children with autism, despite having limited empirical support and particularly common with children with autism (Green, Pituch, Itchon, Choi, O’Reilly, & Sigafoos, 2006; Olson & Moulton, 2004). The driving principle for the use of sensory-based interventions is to improve sensory processing and increase adaptive functioning in individuals with sensory dysfunction (Ayers, 1979) This principle is based on sensory integration theory, which purports that controlled sensory experiences will help children appropriately respond to sensory input (Ayers, 1972).

4. What are sensory-based interventions? Common focal techniques: Wilbarger (brushing & joint compression) Weighted vests, blankets, pressure vests Special seating Vestibular stimulation (e.g., swings) Comprehensive sensory integration: “sensory diet” “sensory room”

5. Issues No protocols exist for their use Different uses observed across practitioners Does not appear to be conceptually systematic Seem to be aversive & reinforcing Passive intervention for reducing challenging behavior Lack of empirical support

6. AAP (2012) recently reported that: “Because there is no universally accepted framework for diagnosis, sensory processing disorder generally should not be diagnosed…. parents should be informed that the amount of research regarding the effectiveness of sensory integration therapy is limited and inconclusive.”

7. Examples of Research 1.Study from an OT journal 2.Study from Behavioral-focused journal/researchers 3.Study from an ASD focused journal 4.My research

8. Example #1: Study from an OT Journal Fertel-Daly, D., Bedell, G., & Hinojosa, J. (2001). Effects of a weighted vest on attention to task and self-stimulatory behaviors in preschoolers with pervasive developmental disorders. American Journal of Occupational Therapy, 55, 629- 640. Effect of weighted vest for increasing attention and decreasing self- stimulatory behaviors to a fine motor task in children with PDD

9. Example #1: Study from an OT Journal ABA Design “reversal” Duration recording IOA – 3 times during baseline Condition changes occurred at the same time for all participants No Procedural Fidelity Mention a published protocol for use of WV is not available Part of an “ABA” program Results: positive?

10. Fertel-Daly, Bedell, & Hinojosa, 2001)

11. P1 P2

12. Example #2: Study from Behavioral- focused journal/researchers Davis TN, Dacus S, Strickland E, et al. (2013). The effects of a weighted vest on aggressive and self-injurious behavior in a child with autism. Developmental Neurorehabilitation, 16, 210-215. Examined the effects of a weighted vest on the SIB of a boy with ASD Wore the vest for 4 hours prior to measurement and during

13. Example #2: Study from Behavioral- focused journal/researchers  9 year old boy with ASD  DV: Biting (self or others)  IOA & PF – met design standards  ABAB with multielement Davis et al. (2013)

15. Example #2: Study from Behavioral- focused journal/researchers “With the legal and ethical responsibility to implement evidence-based interventions educators and related service personnel have the responsibility to use protocols that are supported by empirical, quantitative-based research. However, the results of the body of research on weighted vests do not indicate that it is an effective treatment for individuals with developmental disabilities.” (p. 5). Davis et al. (2013)

16. Example #3: Study from an Autism- specific Journal Cox AL, Gast DL, Luscre D, & Ayres KM. (2009). The Effects of Weighted Vests on Appropriate In-Seat Behaviors of Elementary-Age Students With Autism and Severe to Profound Intellectual Disabilities. Focus Autism Devel Disabil. 24, 17-26.  Disclaimer: one of my all time favorite studies  Effects of weighted vests on in-seat behavior in children with disabilities Cox AL, Gast DL, Luscre D, & Ayres KM. (2009).

17. Example #3: Study from an Autism- specific Journal Study 1: All participants had ASD Ages 5, 6, 9 DV: in seat behavior Setting: daily circle time at a table IOA & PF met design standards Cox AL, Gast DL, Luscre D, & Ayres KM. (2009).

20. Example #3: Study from an Autism- specific Journal Study 2: Same setting & DV IV: Non-contingent reinforcment Choice of a preferred item prior to circle time Cox AL, Gast DL, Luscre D, & Ayres KM. (2009).

23. Example #4: My Research Reichow, B., Barton, E. E., Neely, J., Good, L., & Wolery, M. (2010). Effects of weighted vests on the engagement of children with developmental delays. Focus on Autism and Developmental Disabilities, 25, 3-11. Examined the effects of weighted vests on engagement, problem behaviors, small group table activities (e.g., art, matching, sorting)with preschool children with disabilities

24. Example #4: My Research 3 conditions: Weighted vest, unweighted vest, no vest Alternating treatments design Randomized order of Tx IOA & PF measured according to design standards and was >80% Reichow, Barton, Neely, Good, & Wolery (2010).

25. Tommy – Engagement weighted vest No vest unweighted vest

26. Tommy – Stereotypic Behavior weighted vest No vestunweighted vest

27. Tommy – Problem Behavior weighted vest No vest unweighted vest

28. Bert – Engagement BaselineTreatment No vest unweighted vest weighted vest

29. Bert – Stereotypic Behavior BaselineTreatment

30. Bert – Problem Behavior BaselineTreatment

31. Sam – Engagement BaselineTreatment No vest unweighted vest weighted vest

32. Sam – Stereotypic Behavior BaselineTreatment

33. Sam – Problem Behavior BaselineTreatment

34. Brief report: Effects of pressure vest usage on engagement and problem behaviors of a young child with developmental delays Reichow, Barton, Good, & Wolery (2009)

35. Methods Participant: –Michael, 57 mos., developmental delays in cognitive, language, and fine motor skills Conditions: –No vest –Pressure vest DV: engagement, non-engagement, and problem behavior Design: ABA

36. Results

37. Thoughts regarding weighted/pressure vests??

38. Systematic Review of Sensory-based Interventions for children with disabilities Purpose: To conduct a methodologically rigorous review of sensory-based treatments. Inclusion Criteria: IV: sensory-based intervention Experimental design > 9 years of age Peer-reviewed journals Identified Studies: 30 (Barton, Reichow, Schnitz, Smith, & Sherlock, 2014, under review)

39. Systematic Review Results Participants 856 children 631 males & 225 females 236 children with ASD mean age: 6 years 11 months 20 of 30 measured sensory processing 21 of 30 reported children having problem behaviors or stereotypies (Barton, Reichow, Schnitz, Smith, & Sherlock, 2014, under review)

40. Systematic Review Results Study Methodology: 15 = group 15 = SCRD (Barton, Reichow, Schnitz, Smith, & Sherlock, 2014, under review)

41. Systematic Review Results What Works Clearinghouse Design Standards: Group: 6 met, 5 met with reservations, and 4 did not meet SCRD: (19 analyses) 6 met, 7 met with reservations, 6 did not meet (Barton, Reichow, Schnitz, Smith, & Sherlock, 2014, under review)

42. Systematic Review: Results Results A vast majority of the studies did not identify an effect for SI All studies examining the effects of weighted vests noted no effects for engagement, problem behavior, joint attention, etc. Comparison studies found behavioral treatments to be more effective (e.g., Addison et al., 2012; Devlin et al., 2012). (Barton, Reichow, Schnitz, Smith, & Sherlock, 2014, under review)

43. Systematic Review: Results Risk of Bias Analysis –Risk of Bias assessment tool of the Cochrane Collaboration is often considered one of the best tools for appraisal of RCTs –Uses a domain-based evaluation system so that critical assessments are made separately for different domains in regards to the systematic errors with the internal validity of the study –We adapted the tool for SCRD (Barton, Reichow, Schnitz, Smith, & Sherlock, 2014, under review)

44. Systematic Review: Results Risk of Bias Analysis: SCRD Selection bias –Sequence generation –Participant selection Performance bias –Blinding of participants –Procedural fidelity Detection bias –Blinding of outcome assessment –Dependent variable reliability –Data sampling (Barton, Reichow, Schnitz, Smith, & Sherlock, 2014, under review)

46. Risk of Bias by Study for Single Case Research Design Studies

47. Risk of Bias by Study for group studies

48. Systematic Review: Results WWC Design Standards = SCRD 19 analyses across 15 studies 6 met design standards 7 met with reservations 6 did not meet (Barton, Reichow, Schnitz, Smith, & Sherlock, 2014, under review)

49. Systematic Review: Results WWC Evidence Criteria = SCRD Each tier = 33 analyses 7 demonstrations of strong evidence across 4 SCRD studies 4 were in comparative studies and all 4 favored behavioral over sensory-based interventions 3 were in use of a therapy ball or linear swinging (Barton, Reichow, Schnitz, Smith, & Sherlock, 2014, under review)

50. Systematic Review: Results WWC Evidence Criteria = SCRD Each tier = 33 analyses 6 demonstrations of moderate evidence across 3 SCRD studies 20 of 33 provided no evidence of a functional relation for sensory-based interventions (Barton, Reichow, Schnitz, Smith, & Sherlock, 2014, under review)

51. Systematic Review: Results Ruling out alternative explanations: Van Rie and colleagues (2009) documented increased levels of correct responding for two participants when they were swung immediately prior to instruction and for one participant when he was bounced on a ball. The swing or therapy ball might have simply been more reinforcing to the participants rather than related to improved processing of sensory information. (Barton, Reichow, Schnitz, Smith, & Sherlock, 2014, under review)

52. Systematic Review: Results Ruling out alternative explanations Schaaf and colleagues (2013) found positive outcomes for children receiving sensory integration treatment when compared to a control group. Parents provided ratings on primary outcomes, were not blind to the study purpose or groups Increased expectations? The amount of treatment received was not equal between groups (i.e., children in the sensory integration treatment group received more time in treatment). Increased time in treatment? (Barton, Reichow, Schnitz, Smith, & Sherlock, 2014, under review)

53. Systematic Review: Results Ruling out alternative explanations Schilling and Schwartz (2004) documented increased engagement and in seat behavior using a therapy ball as seating. Improved sensory processing or reinforcement contingencies. That is, the therapy ball might have simply been more reinforcing to the participants rather than related to improved processing of sensory information. (Barton, Reichow, Schnitz, Smith, & Sherlock, 2014, under review)

54. Systematic Review: Summary Based on our analysis of this literature, sensory-based treatments are more likely to be ineffective than effective for children with disabilities. Thus, it might be time to reexamine sensory integration theory and sensory processing disorder. (Barton, Reichow, Schnitz, Smith, & Sherlock, 2014, under review)

55. Systematic Review  30 experimental studies – a significant literature  There are studies with negative findings that meet rigor/design standards and have limited biases  At best: Inconsistent findings  Positive findings: did not allow for the exclusion of alternative explanations or did not meet design standards, had significant biases (Barton, Reichow, Schnitz, Smith, & Sherlock, 2014, under review)

56. Systematic Review  Lack of treatment fidelity data  only 10 of 30 (33%) studies measured sensory- based outcomes.  Lack of a standard outcome battery  No studies to date have directly measured neurological processing of sensory information, which is the purported mechanism by which sensory-based treatments are effective. (Barton, Reichow, Schnitz, Smith, & Sherlock, 2014, under review)

57. “More likely to be ineffective than effective”

58. Implications Reporting of negative results is essential for identifying evidence-based practices Other important issues to consider: –Selective outcome reporting –Selective participant reporting –Use of unpublished studies in reviews (dissertations, theses) –Registration of studies

59. References American Academy of Pediatrics (2012). Sensory integration therapies for children with developmental and behavioral disorders. Pediatrics, 129, 1186-1189. doi: 10.1542/peds.2012-0876 Cox, A. L., Gast, D. L., Luscre, D., & Ayres, K. M. (2009). The effects of weighted vests on appropriate in- seat behaviors of elementary-age students with autism and severe to profound intellectual disabilities. Focus on Autism and Other Developmental Disabilities, 24, 17-26. doi: 10.1177/1088357608330753 Davis TN, Dacus S, Strickland E, et al. (2013). The effects of a weighted vest on aggressive and self-injurious behavior in a child with autism. Developmental Neurorehabilitation, 16, 210-215. Reichow, B., Barton, E. E., Sewell, J. N., Good, L., & Wolery, M. (2009). Effects of Weighted Vests on the Engagement of Children With Developmental Delays and Autism. Focus on Autism and Other Developmental Disabilities, 25(1), 3-11. doi: 10.1177/1088357609353751 Reichow, B., Barton, E. E., & Maggin, D. (2013). Risk of bias assessment for single subject experimental designs. Unpublished Manuscript.