Actual analyses Visual analysis Increasing trends Immediacy of effects Nonoverlap indices PND: Outlier distorting for Craig and Blake PEM Tau-U: overlap + B-trend

Comments on data analysis Assessment of the original Visual analysis: OK, but can focus on more elements PEM: is the median meaningful for Craig and Blake? PND: the authors know it is not OK Tau-U: could have controlled for trend for Craig Alternatives NAP Piecewise regression (Blake B-phase?) + Multilevel extension Between-cases standardized mean difference (change in level?)

Actual analyses Statistical analysis Simulation modelling analysis: Compute a point-biserial correlation between scores and a dummy (0,1) variable. Simulate normal data with the same autocorrelation (or extract bootstrap samples) Compute the same correlation for each «sample» Compute the proportion of correlations as large as or larger than the actual one

Comments on data analysis Assessment of the original Simulation modelling analysis: OK for Sara, not OK for Alex (improving trend) Alternatives Tau-U controlling for baseline trend or Baseline corrected Tau: especially relevant for Alex, OK for Sara. Piecewise regression or SLC: OK for Alex, too much variability for Sara?

Actual analyses This is only an example!!!!

Actual analyses Visual analysis Change in slope, in mean, in level, latency

Comments on data analysis Assessment of the original Visual analysis - example: what is the difference between change in level and in mean? Immediacy of effect mixed with change in slope? Sounds subjective, except for the means. Alternatives 4 multilevel analyses: 1 per target behavior (most within-phase data are not very variable + trends). 12 Tau-U (Baseline corrected Tau, overcorrects?)

Actual analyses Graphical display Verbal comments

Comments on data analysis Assessment of the original Graphical display: unusual and not helpful (staggered introduction not clear, connecting dots from different conditions) Verbal comments: insufficient Alternatives 7 NAPs 1 Between-cases standardized mean difference for A-B 1 Between-cases standardized mean difference for B-FU

Actual analyses Statistical analysis Visual analysis NAP Informal, focusing on certain data aspects, such as trends and outliers

Comments on data analysis Assessment of the original NAP: not OK for number of events forgotten – there is an improving trend Visual comments: OK for understanding better the effects of the treatment Alternatives Percentage change index Percentage zero data Tau-U or Baseline corrected Tau instead of NAP

Actual analyses Visual analysis Statistical analysis Stability and presence of trends assessed for choosing a statistical procedure Statistical analysis SLC Piecewise regression

Comments on data analysis Assessment of the original Visual analysis: OK, although more details could have been provided on all 6 features mentioned in the WWC standards SLC & Piecewise regression: OK, although it is possible to use onyl one of those. Alternatives Percentage change index focussing on the last 3 / phase

Actual analyses Statistical analysis Kruskal-Wallis: non-parametric equivalent of one-way ANOVA, applied to the pain intensity measured on the numerical rating scale (NRS): shaded area

Comments on data analysis Assessment of the original Kruskal-Wallis: not directly clear how the three moments mentioned in the text are represented by the shaded are; assumption of independence?  why not Friedman’s test for repeated measures? Alternatives If the individual measures are available: NAP between the values of different conditions; Percentage change index