1. Statistical Randomization Techniques for Single-Case Intervention Data Statistical Randomization Techniques for Single-Case Intervention Data Joel R. Levin University of Arizona

2. Purpose of This Presentation (Dejà Vu?) To broaden your minds by introducing you to new and exciting, scientifically credible single-case intervention design-and-analysis possibilities. To let you know that these procedures are becoming increasingly acceptable to single-case intervention researchers and are beginning to appear in the SCD research literature. Whether YOU ever choose to adopt them in your own SCD research (after this week!) is entirely up to you.

3. A Permutation-Test Primer (Based on Levin, 2007) Rationale and assumptions Samples and populations Individuals and groups Scores and ranks –exact probabilities and sampling approximations for scores Levin, J. R. (2007). Randomization tests: Statistical tools for assessing the effects of educational interventions when resources are scarce. In S. Sawilowsky (Ed.), Real data analysis (pp. 115-123). Greenwich, CT: Information Age.

4. Example 1: Very Small Samples From a total of 6 elementary school classrooms, 3 are randomly assigned to receive an instructional intervention that is designed to boost students’ academic performance (intervention classrooms), while students in the other 3 classrooms continue to receive their regular instruction (control classrooms). Following the instructional phase of the experiment, students in all classrooms are administered a 50-point achievement test and the average test performance within each classroom is calculated. Of interest in this study is the mean achievement-test difference between the 3 intervention classrooms and the 3 control classrooms.

5. Example 1: Data-Analysis Rationale The obtained mean difference is examined in the context of the distribution of all possible mean differences that can be generated by assigning the 6 obtained classroom means to two instructional conditions, assuming that 3 classroom means must be assigned to each condition. A statistical test is then conducted by addressing the question: How (un)likely or (im)probable is what actually occurred (i.e., the obtained intervention-control mean difference) in relation to everything that could have occurred (i.e., the distribution of all possible intervention- control mean differences, given the study's design structure and the set of means produced)? Should the result of the foregoing test be deemed statistically improbable (e.g., p <.05), then the researcher would conclude that the two instructional methods differ with respect to students' average achievement-test performance.

6. In how many different ways can 6 scores be assigned to 2 groups, if 3 scores must end up in each group? That is the same thing as asking how many different combinations of 3 objects are there if selected from a total of 6 objects. So as not to waste time attempting to express that quantity symbolically here: The answer, my friends, boils down to 6!/3!3! = 20. For example, consider the following 6 scores: 1 2 3 4 5 6 Let us systematically count the specific ways that 3 scores could be assigned to Group 1. (Note: The order in which the 3 scores are listed is not important.) 1. 1 2 3 2. 1 2 4 3. 1 2 5 4. 1 2 6 5. 1 3 4 6. 1 3 5 7. 1 3 6 8. 1 4 5 9. 1 4 6 10. 1 5 611. 2 3 412. 2 3 5 13. 2 3 614. 2 4 515. 2 4 616. 2 5 6 17. 3 4 518. 3 4 6 19. 3 5 620. 4 5 6

7. Example 1: Classroom Means and All Possible Assignments of Them to the Two Conditions (N 1 = N 2 = 3) Condition 1 Condition 2 M Difference (C1-C2) 1.42.6, 40.1, 39.6 (122.3, 40.8) 37.6, 36.7, 36.3 (110.6, 36.9) 3.9* p 1 = 1/20 =.05 2. 42.6, 40.1, 37.6 (120.3, 40.1) 39.6, 36.7, 36.3 (112.6, 37.5) 2.6 p 2 = 2/20 =.10 3. 42.6, 40.1, 36.7 (119.4, 39.8) 39.6, 37.6, 36.3 (113.5, 37.8) 2.0 4. 42.6, 40.1, 36.3 (119.0, 39.7) 39.6, 37.6, 36.7 (113.9, 38.0) 1.7 5. 42.6, 39.6, 37.6 (119.8, 39.9) 40.1, 36.7, 36.3 (113.1, 37.7) 2.2 6. 42.6, 39.6, 36.7 (118.9, 39.6) 40.1, 37.6, 36.3 (114.0, 38.0) 1.6 7. 42.6, 39.6, 36.3 (118.5, 39.5) 40.1, 37.6, 36.7 (114.4, 38.1) 1.4 8. 42.6, 37.6, 36.7 (116.9, 39.0) 40.1, 39.6, 36.3 (116.0, 38.7) 0.3 9. 42.6, 37.6, 36.3 (116.5, 38.83) 40.1, 39.6, 36.7 (116.4, 38.80) 0.03 10. 42.6, 36.7, 36.3 (115.6, 38.5) 40.1, 39.6, 37.6 (117.3, 39.1) -0.6 11. 40.1, 39.6, 37.6 (117.3, 39.1) 42.6, 36.7, 36.3 (115.6, 38.5) 0.6 12. 40.1, 39.6, 36.7 (116.4, 38.80) 42.6, 37.6, 36.3 (116.5, 38.83) -0.03 13. 40.1, 39.6, 36.3 (116.0, 38.7) 42.6, 37.6, 36.7 (116.9, 39.0) -0.3 14. 40.1, 37.6, 36.7 (114.4, 38.1) 42.6, 39.6, 36.3 (118.5, 39.5) -1.4 15. 40.1, 37.6, 36.3 (114.0, 38.0) 42.6, 39.6, 36.7 (118.9, 39.6) -1.6 16. 40.1, 36.7, 36.3 (113.1, 37.7) 42.6, 39.6, 37.6 (119.8, 39.9) -2.2 17. 39.6, 37.6, 36.7 (113.9, 38.0) 42.6, 40.1, 36.3 (119.0, 39.7) -1.7 18. 39.6, 37.6, 36.3 (113.5, 37.8) 42.6, 40.1, 36.7 (119.4, 39.8) -2.0 19. 39.6, 36.7, 36.3 (112.6, 37.5) 42.6, 40.1, 37.6 (120.3, 40.1) -2.6 20. 37.6, 36.7, 36.3 (110.6, 36.9) 42.6, 40.1, 39.6 (122.3, 40.8) -3.9

8. What Summary Measure(s) Can/Should Be Analyzed by The To-Be-Presented Techniques? Means (Levels) of the phases Medians Truncated/Censored data based on a priori rules Even randomly selected observations? Slopes (Trends) of the phases Variances of the phases Any “Predicted Pattern” within or between phases Special considerations when groups are the units of intervention administration

9. Randomized Adaptations of Traditional Single-Case Designs Phase/Intervention Randomization (Within Cases) –the order of the A and B phases/interventions is randomly determined for each case (e.g., participant, pair, group, classroom) Intervention Randomization (Between Cases) ‒ cases are randomly assigned to interventions Intervention Start-Point Randomization –the transition point from one phase to the next is randomly determined for each case Case Randomization –cases are randomly assigned to positions within the design

10. Why Randomization Statistical Tests? Conceptually and computationally straightforward –easy to explain to others –parsimonious Logically consistent connection to the design’s randomization components –implications for Type I error control Statistical power characteristics Underlying statistical assumptions Versatility and adaptability relative to other single- case statistical approaches

11. Background Eugene Edgington’s randomization-test contributions Randomized phase/intervention designs –Basic design (AB) and return to baseline design (ABA), including when A and B consist of two different interventions –Reversal (or withdrawal or “operant”) design (ABAB…AB) and alternating treatment design –Multiple-baseline design Randomized intervention start-point designs –Basic design (AB) and return-to-baseline design (ABA), including when A and B consist of two different interventions –Reversal (or withdrawal or “operant”) design (ABAB…AB) and alternating treatment design –Multiple-baseline design Replications (i.e., multiple units) and extensions of these

12. Three Sample Statistical Simulation Studies ABAB…AB and Alternating Treatment Design (Levin, Ferron, & Kratochwill, 2012) Variety of AB Designs (Levin, Ferron, & Gafurov, 2014) Multiple-Baseline Design (Levin, Ferron, & Gafurov, 2015)

13. Top 10 List of Distinctions Between Actual and Simulated Single-Case Intervention Studies (With thanks to David Letterman, wherever you are!) 10.Participants 9.Planning and Design 8.Methods and Procedures 7.Institutional Review Boards 6.Investigator Personality Characteristics

14. Top 10 Distinctions (Continued) 5.Time and Hassle 4.Data Analysis 3.Rationalizations and Rationale for Future Research 2.Follow-Up 1.Publication Outlets and Impact Issues Bonus Question: Why did I ever bother?

15. Systematic vs. Randomized ABAB…AB and Alternating-Treatment Designs A few selected results from a Monte Carlo simulation study by Levin, Ferron, and Kratochwill (2012)

16. Five 24-Observation Designs (Individual Observations) ABABABABABABABABABABABAB24 Phase AABBAABBAABBAABBAABBAABB 12 Phase AAAABBBBAAAABBBBAAAABBBB 6 Phase AAAAAABBBBBBAAAAAABBBBBB 4 Phase AAAAAAAAAAAABBBBBBBBBBBB 2 Phase

17. Type I Error Probability as a Function of Autocorrelation in the 24-Observation Design (Individual Observations)

18. Five 24-Observation ABAB…AB Designs (Phase Means) ABABABABABABABABABABABABBlock=1 (24 phases) BAABABABBAABBAABABBABABARandom Pair (13-24 phases) AABBAABBAABBAABBAABBAABB Block=2 (12 phases) AAABBBAAABBBAAABBBAAABBB Block=3 (8 phases) AAAABBBBAAAABBBBAAAABBBB Block=4 (6 phases)

19. Type I Error Probability as a Function of Autocorrelation in the 24-Observation Design (Phase Means)

20. Effect-Size Alert for Single-Case Research Outcomes, or Don’t “Dis” Large Effect Sizes Here! Marquis et al. (2000) noted in their meta-analysis of positive behavior support that “[t]he smallest [conventional effect-size measure] for outcomes was 1.5, which would be considered quite large in a group study context” (p. 165); and that their effect-size estimates “ranged from 1.5 standardized units to 3.1 units” (p. 167). Rogers and Graham (2008, p. 885) indicated that “[W]hen we have used [the conventional method of effect-size calculation in meta-analyses of] single subject design studies in writing, the effect sizes are typically 3.0 and higher.” In a single-case enuresis-treatment study conducted by Miller (1973), the conventional effect sizes calculated for the two participants were 5.98 and 6.41 (Busk & Serlin, 1992, p. 201-202).

21. References Busk, P. L., & Serlin, R. C. (1992). Meta-analysis for single-case research. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case research design and analysis (pp. 187-212). Hillsdale, NJ: Erlbaum. Marquis, J. G., Horner, R. H., Carr, E. G., Turnbull, A. P., Thompson, M., Behrens, G. A., et al. (2000). A meta-analysis of positive behavior support. In R. Gersten, E. P. Schiller, S. Vaughn, (Eds.), Contemporary special education research: Syntheses of knowledge base on critical instructional issues (pp. 137-178). Mahwah, NJ: Erlbaum. Miller, P. M. (1973). An experimental analysis of retention control training in the treatment of nocturnal enuresis in two institutional adolescents. Behavior Therapy, 4, 288-294. Rogers, L. A., & Graham, S. (2008). A meta-analysis of single subject design writing intervention research. Journal of Educational Psychology, 100, 879-906.

22. Power for d = 1.00 as a Function of Autocorrelation in the 24-Observation Design (Phase Means)

23. Major Conclusions 1. Claims that systematic ABAB…AB designs produce inflated Type I error probabilities in series with positively autocorrelated observations are grossly “inflated.” 2. A systematic design consisting of alternating A and B individual observations yields very respectable power for detecting larger effect sizes. From a methodological perspective, an even more appealing option is the randomized pairs ─ or, in certain situations, the randomized paired doubles ─ design, perhaps with one or more mandatory initial A' observations in situations where a true baseline is desired (Kratochwill & Levin, 2010). The latter is not an issue if A and B represent two alternative interventions.

24. Adapted from Levin, J. R., & Wampold, B. E. (1999). Generalized single ‑ case randomization tests: Flexible analyses for a variety of situations. School Psychology Quarterly, 14, 59 ‑ 93.

26. Replicated AB Design With Three Cases (“Units”), Two Within-Series Intervention Conditions, 20 Time Periods, and 13 Potential Intervention Points for Each Case Marascuilo, L. A., & Busk, P. L. (1988). Combining statistics for multiple-baseline AB and replicated ABAB designs across subjects. Behavioral Assessment, 10, 1- 28.

27. Levin & Wampold’s (1999) Simultaneous Start-Point Model Time Period 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Pair 1X A A A A A A A A A A A │ B B B B B B B B B Pair 1Y A A A A A A A A A A A │ B B B B B B B B B Note: Potential intervention start points are between Time Periods 5 and 17 inclusive. *Randomly selected intervention start point for the pair of units Levin, J. R., & Wampold, B. E. (1999). Generalized single-case randomization tests: Flexible analyses for a variety of situations. School Psychology Quarterly, 14, 59–93.

28. A Two-Intervention (Between Cases) Example

29. From Levin, J. R., & Wampold, B. E. (1997, July) Single-case randomization tests for a variety of situations. Paper presented at the 10th European Meeting of the Psychometric Society, Santiago de Compostela, Spain.

30. Means and Mean Differences Associated With Each of the 9 Potential Intervention Startpoints, By Reinforcer Type Start Point SocialToken T-S A B B-A A B B-A Diff Rank 3 13.0 13.9 0.9 12.5 16.3 3.8 2.9 9 4 13.3 13.9 0.6 12.7 16.7 4.0 3.4 8 5 13.5 13.9 0.0 12.5 17.3 4.8 4.8 7 6 13.8 13.7 -0.1 12.6 17.9 5.3 5.4 3 *7 13.7 13.8 0.0 13.2 18.2 5.0 5.0 4* 8 14.0 13.4 -0.6 14.0 18.0 4.0 4.6 5.5 9 14.1 13.0 -1.1 14.5 18.0 3.5 4.6 5.5 10 14.4 11.7 -2.7 14.7 18.7 4.0 6.7 2 11 14.5 10.0 -4.5 15.1 18.5 3.4 7.9 1

31. Levin & Wampold’s (1999) Replicated Simultaneous Start-Point Model Time Period 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Pair 1X A A A A A A A A A A A* B B B B B B B B B Pair 1Y A A A A A A A A A A A* B B B B B B B B B Pair 2X A A A A A A A A A* B B B B B B B B B B B Pair 2Y A A A A A A A A A* B B B B B B B B B B B Note: Potential intervention start points are between Time Periods 5 and 17 inclusive. *Randomly selected intervention start point for each pair of units Levin, J. R., & Wampold, B. E. (1999). Generalized single-case randomization tests: Flexible analyses for a variety of situations. School Psychology Quarterly, 14, 59–93

32. Three Sample Statistical Simulation Studies ABAB…AB and Alternating Treatment Design (Levin, Ferron, & Kratochwill, 2012) Variety of AB Designs (Levin, Ferron, & Gafurov, 2014)

33. Proposed Single-Case Intervention Design/Analysis Schemes* 1. AB Randomized Phase-Order Design With Intervention Start-Point Randomization *Levin, J. R., Ferron, J. M., & Gafurov, B. S. (2014). Improved randomization tests for a class of single-case intervention designs. Journal of Modern Applied Statistical Methods, 13(2), 2-52; retrievable from http://digitalcommons.wayne.edu/jmasm/vol13/iss2/2. Onghena, P., Vlaeyen, J. W. S., & de Jong, J. (2007). Randomized replicated single- case experiments: Treatment of pain-related fear by graded exposure in vivo. In S. Sawilowsky (Ed.), Real data analysis (pp. 387-396). Greenwich, CT: Information Age.

34. Suppose that in a 16-observation design, A and B are either a baseline phase and an intervention phase or two different interventions that a single case is to receive. The case is assigned randomly to one of the two phase orders (AB or BA). [With a true baseline-intervention design this can be accomplished by including one or more mandatory baseline/adaptation observations (A') for both phase orders.] The random assignment of phase orders is required for the subsequent AB randomization test (modified Edgington test) to be valid. In addition, the case receives a randomly selected intervention start point, with an a priori specification of 10 potential start points, from Observations 4 through 13 inclusive.

35. AB Randomized Phase-Order Design (With Mandatory Initial A' Baseline Phase) With the original Edgington (1975) model, the study can be diagrammed as: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 A' A' A' A A A B B B* B B B B B B B B B B However, with the revised Edgington model, the opposite “pretend” ordering of As and Bs was also possible and therefore can be included in the randomization distribution: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 A' A' A' B B B A A A* A A A A A A A A A A

36. AB Randomized Phase-Order Design (With Mandatory Initial A' Baseline Phase) For Edgington’s model, with 10 potential intervention points, even if the observed outcome were the most extreme the lowest one-tailed significance probability would be p = 1/10 =.10. In contrast, for the revised model with both orders taken into account, even with only 10 potential intervention points it would be possible to attain a one-tailed significance probability of p = 1/20 =.05. Moreover, our simulation results indicate that the revised procedure: (1) maintains satisfactory Type I error control; and (2) exhibits power that is generally about.20-.40 points higher (and in some situations, even more) than that of the original Edgington procedure.

37. Comparison (α =.05, one-tailed) of randomization tests for a one- case (N = 1) AB randomized intervention start-point design (Single) and the randomized intervention start-point plus randomized intervention-order design (Dual), where the start point was randomly selected between the 6 th through the 25 th observations inclusive in a 30-observation study.

38. Comparison (α =.05, one-tailed) of randomization tests for the Single and Dual basic AB randomized designs replicated across N cases. The rejection rate of the null hypothesis is shown as a function of effect size and N, for a 15 observations design with 5 potential intervention start points designated from between the 6 th and 10 th observations inclusive and an autocorrelation of.3.

39. Selected Single- Versus Dual-Randomization Power Comparisons of Longer and Shorter Series Simulations (SL = Series Length, NSP = Number of Potential Intervention Start Points) NdrSize (SL/NSP)SingleDualDifference 22.0.30Longer (15/5).44.85.41 Shorter (9/5).42.80.38 31.5.30Longer (15/5).49.90.41 Shorter (7/3).28.73.45 51.0.30Longer (15/5).45.89.44 Shorter (8/2).15.71.56

40. Multiple-Baseline Design Revusky’s (1967) statistical procedure Revusky, S. H. (1967). Some statistical treatments compatible with individual organism methodology. Journal of the Experimental Analysis of Behavior, 10, 319-330. Wampold and Worsham’s (1986) improved permutation procedure Wampold, B. E., & Worsham, N. L. (1986). Randomization tests for multiple-baseline designs. Behavioral Assessment, 8, 135-143. Type I error and power assessments have been made by Ferron and Sentovich (2002) of the Wampold-Worsham (1986), Marascuilo-Busk (1988), and Koehler-Levin (1998) procedures Ferron, J., & Sentovich, C. (2002). Statistical power of randomization tests used with multiple-baseline designs. Journal of Experimental Education, 70, 165-178.

41. Three Sample Statistical Simulation Studies ABAB…AB and Alternating Treatment Design (Levin, Ferron, & Kratochwill, 2012) Variety of AB Designs (Levin, Ferron, & Gafurov, 2014) Multiple-Baseline Design (Levin, Ferron, & Gafurov, 2015) Levin, J. R., Ferron, J. F., & Gafurov, B. S. (2015). Comparison of randomization-test procedures for single-case multiple-baseline designs. Unpublished manuscript. University of Arizona, Tucson.

44. Proposed Single-Case Intervention Design/Analysis Schemes 1.AB Randomized Phase Design With Intervention Start-Point Randomization 2.AB Crossover Design With Intervention Start-Point Randomization 3.“Uber Supernova” Multiple-Baseline Comparative- Treatment Design With Intervention Start-Point Randomization

45. Multiple-Baseline Comparative-Treatment Design (Modified Levin-Wampold & Koehler-Levin Designs) Time Period 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Pair 3XA A A A B* B B B B B B B B B B B Pair 3YA A A A B* B B B B B B B B B B B Pair 1YA A A A A A B* B B B B B B B B B Pair 1XA A A A A A B* B B B B B B B B B Pair 4YA A A A A A A A A A B* B B B B B Pair 4XA A A A A A A A A A B* B B B B B Pair 2XA A A A A A A A A A A A A A B* B Pair 2YA A A A A A A A A A A A A A B* B Note: Pairs are randomly assigned to stagger positions. Interventions X and Y are randomly assigned within each pair. The bolded letters represent the potential start points for each pair, with the asterisked B indicating the actual start point randomly selected for each pair.

46. References Levin, J. R., Ferron, J., M., & Kratochwill, T. R. (2012). Nonparametric statistical tests for single-case systematic and randomized ABAB…AB and alternating treatment designs: New developments, new directions. Journal of School Psychology, 50, 599-624. Levin, J. R., Ferron, J. M., & Gafurov, B. S. (2014). Improved randomization tests for a class of single-case intervention designs. Journal of Modern Applied Statistical Methods, 13(2), 2- 52; retrievable from http://digitalcommons.wayne.edu/jmasm/vol13/iss2/2. Levin, J. R., Ferron, J. M., & Gafurov, B. S. (2015). Comparison of randomization-test procedures for single-case multiple-baseline designs. Unpublished manuscript, University of Arizona, Tucson.

47. Some Randomization-Test Software Randomized Phase Designs Edgington & Onghena’s (2007) SCRT (Single-Case Randomization Tests) program, in their book Levin, Ferron, & Kratochwill’s (2012) SAS software for various ABAB…AB and alternating treatment designs Randomized Intervention Start-Point Designs Edgington & Onghena’s (2007) SCRT program, in their book (also Bulté & Onghena, 2008) Koehler’s (2012) program for the Koehler-Levin (1998) multiple- baseline procedure (http://mkoehler.educ.msu.edu/regrand/) Gafurov & Levin’s ExPRT (Excel Package of Randomization Tests) Version 2.0, August 2015; downloadable at http://code.google.com/p/exprt Other Borckhardt et al.’s (2008) Simulation Modeling Analysis (SMA) program

48. Current and Projected Features of Gafurov & Levin’s ExPRT Statistical Software Features of the programs include: exact nonparametric statistical analyses based on some type of randomization options for either random or fixed intervention start points and intervention orders designs based on either one or multiple cases (replications) an unlimited number of total observations for up to 15 cases either individual or paired cases analyses conducted with either raw or standardized data

49. Current and Projected Features of the ExPRT Statistical Software Features of the programs include: user-defined α levels (one- or two-tailed tests) statistical decisions (reject, do not reject) and significance probabilites (p-values) statistical tests based on either mean (level) or slope (trend) output distribution of all possible outcomes graph of the outcomes for each case case-by-case and across-case summary measures and effect-size estimates a randomizing routine for planned studies

50. AB Design Basic time-series design Baseline/Control (A) vs. Intervention (B) Intervention A vs. Intervention B Intervention start-point randomization procedure (Edgington model; Marascuilo-Busk model; Levin- Wampold simultaneous start-point model for two different matched interventions: comparative- effectiveness and general-effectiveness tests) Levin et al.’s (2014) randomized intervention-order option; single-case crossover design: intervention- effect and time-effect.

58. Additional ExPRT Randomization Tests ABA Design Intervention start-point randomization (Onghena model) Overall test and separate two-phase tests Reversal (ABAB) Design Intervention start-point randomization (Onghena model) Overall test and separate two-phase tests Multiple-Baseline Design Within-case comparisons (Wampold-Worsham model) Within-case comparisons; intervention start-point randomization (Koehler-Levin and restricted Marascuilo- Busk models) Between-case comparisons (Revusky model)

59. Additional References Bulté, I., & Onghena, P. (2008). An R package for single-case randomization tests. Behavior Research Methods, 40, 467–478. Edgington, E. S., & Onghena, P. (2007). Randomization tests (4th ed.) Boca Raton, FL: Chapman & Hall/CRC. Koehler, M. J., & Levin, J. R. (1998). Regulated randomization: A potentially sharper analytical tool for the multiple ‑ baseline design. Psychological Methods, 3, 206 ‑ 217. Levin, J. R., & Ferron, J. M. (2014). Review of Dugard, File, and Todman’s Single-case and small-n designs: A practical guide to randomization tests (2 nd ed.). American Statistician, 68, 217-218. Levin, J. R., Lall, V. F., & Kratochwill, T. R. (2011). Extensions of a versatile randomization test for assessing single-case intervention effects. Journal of School Psychology, 49, 55-79. Levin, J. R., Marascuilo, L. A., & Hubert, L. J. (1978). N = nonparametric randomization tests. In T. R. Kratochwill (Ed.), Single subject research: Strategies for evaluating change (pp. 167–196). New York, NY: Academic Press.