2. Actual analyses Visual analysis Increasing trends Immediacy of effects
Nonoverlap indices
PND: Outlier distorting for Craig and Blake
PEM
Tau-U: overlap + B-trend

3. Comments on data analysis
Assessment of the original
Visual analysis: OK, but can focus on more elements
PEM: is the median meaningful for Craig and Blake?
PND: the authors know it is not OK
Tau-U: could have controlled for trend for Craig
Alternatives
NAP
Piecewise regression (Blake B-phase?) + Multilevel extension
Between-cases standardized mean difference (change in level?)

6. Actual analyses Statistical analysis Simulation modelling analysis:
Compute a point-biserial correlation between scores and a dummy (0,1) variable.
Simulate normal data with the same autocorrelation (or extract bootstrap samples)
Compute the same correlation for each «sample»
Compute the proportion of correlations as large as or larger than the actual one

7. Comments on data analysis
Assessment of the original
Simulation modelling analysis: OK for Sara, not OK for Alex (improving trend)
Alternatives
Tau-U controlling for baseline trend or Baseline corrected Tau: especially relevant for Alex, OK for Sara.
Piecewise regression or SLC: OK for Alex, too much variability for Sara?

10. Actual analyses
This is only an example!!!!

11. Actual analyses Visual analysis
Change in slope, in mean, in level, latency

12. Comments on data analysis
Assessment of the original
Visual analysis - example: what is the difference between change in level and in mean? Immediacy of effect mixed with change in slope? Sounds subjective, except for the means.
Alternatives
4 multilevel analyses: 1 per target behavior (most within-phase data are not very variable + trends).
12 Tau-U (Baseline corrected Tau, overcorrects?)

15. Actual analyses
Graphical display
Verbal comments

16. Comments on data analysis
Assessment of the original
Graphical display: unusual and not helpful (staggered introduction not clear, connecting dots from different conditions)
Verbal comments: insufficient
Alternatives
7 NAPs
1 Between-cases standardized mean difference for A-B
1 Between-cases standardized mean difference for B-FU

19. Actual analyses Statistical analysis Visual analysis NAP
Informal, focusing on certain data aspects, such as trends and outliers

20. Comments on data analysis
Assessment of the original
NAP: not OK for number of events forgotten – there is an improving trend
Visual comments: OK for understanding better the effects of the treatment
Alternatives
Percentage change index
Percentage zero data
Tau-U or Baseline corrected Tau instead of NAP

23. Actual analyses Visual analysis Statistical analysis
Stability and presence of trends assessed for choosing a statistical procedure
Statistical analysis
SLC
Piecewise regression

24. Comments on data analysis
Assessment of the original
Visual analysis: OK, although more details could have been provided on all 6 features mentioned in the WWC standards
SLC & Piecewise regression: OK, although it is possible to use onyl one of those.
Alternatives
Percentage change index focussing on the last 3 / phase

27. Actual analyses Statistical analysis
Kruskal-Wallis: non-parametric equivalent of one-way ANOVA, applied to the pain intensity measured on the numerical rating scale (NRS): shaded area

28. Comments on data analysis
Assessment of the original
Kruskal-Wallis: not directly clear how the three moments mentioned in the text are represented by the shaded are; assumption of independence?  why not Friedman’s test for repeated measures?
Alternatives
If the individual measures are available: NAP between the values of different conditions; Percentage change index