1. Responsiveness: how much change on SASNOS is evidence of improvement in NBD symptoms?

2. “Is my treatment the cause of this person’s change?” (Wilson, 1991)
Measuring Outcomes
“Is my treatment the cause of this person’s change?” (Wilson, 1991) T1 T2
Repeated measurement can inform range of contexts at individual and group levels

3. Measuring Outcomes ‘Holy Trinity’ of Psychometric Properties Validity
Reliability
Responsiveness
The ability of an instrument to detect change
Essential property to demonstrate for standardised measures used for repeated measurement, needs to work at both group and individual levels
“What is the minimum T1-T2 difference score that reflects change?”

4. XXXX ? ? ? ? ? ?

5. Detecting Change on Psychometric Measures
Despite its obvious functionality and requirement to demonstrate it in repeated measures context, information on responsiveness is often missing
10/27 (37%) instruments concerned in part/whole in measuring NBD had no responsiveness data; most that do use t-tests

6. Measuring Outcomes
Lack of consensus re definitions of Responsiveness, no ‘gold standard’ agreed on how determined
Minimally Detectable Change (MDC): Smallest T1 – T2 difference that falls outside the measurement error of an instrument
Minimally Important Change (MIC): Smallest T1 – T2 difference that corresponds to the smallest change in status that stakeholders (persons, patients, significant others, or clinicians) consider important.

7. Detecting Change on Psychometric Measures
Minimally Important Change determined using an external anchor – difficulties for NBD in ABI
Distribution-based methods:
Paired t-tests
Effect Size (SRM)
Standard Error of Measurement (SEM)
MDC Confidence Intervals (90, 95%)
0.5 Standard Deviation
Reviews have identified some distribution based methods coincide with change on anchors associated with MIC and can be used as proxies
SEM cited in literature as both a method used to discriminate change beyond that expected from error, and as a MIC proxy

8. Detecting Change in NBD using SASNOS
Alderman, Williams, Knight & Wood (submitted).
Measuring change in symptoms of neurobehavioural disability: responsiveness of the St Andrew’s – Swansea Neurobehavioural Outcomes Scale (SASNOS).
Determine whether SASNOS has the statistical properties to effectively measure expected change in symptoms of NBD over time
Investigate what aspects of NBD are most amenable to NbR by determining magnitude of change in aggregate data in each SASNOS domain
Identify cut-off scores for clinicians and researchers to employ across a range of contexts to reliably discriminate genuine improvement from those due to error in the instrument.

9. Responsiveness SASNOS Database
Anonymous database of 542 participants in a range of UK residential NbR services
At least 2x repeat SASNOS assessments for 145
Mean age at time of ABI 42.4 yrs, 71% male
Mean time in NbR at T weeks
T1-T2 < 12 months for 75% participants

10. SASNOS Responsiveness: measures of change
T1-T2 Mean Difference t p
Effect Size SRM
SEM % ‘improved’
(expected to improve)
SEM % ‘improved’ (expected to improve)
Total
5.25
6.05 **
moderate
53.1
63.7
Interpersonal Behaviour
8.00
7.45
59.3
64.3
Cognition
8.32
8.81
large
64.8
68.2
Inhibition
2.61
3.13 *
small
35.9
68.8
Aggression
1.27
1.37 ns
32.4
77.8
Communication
2.67
3.72
43.4
66.7
** p<.001, * p<.01

11. SASNOS Responsiveness: change in mean ratings T1 vs. T2
Total Score**
Interpersonal Behaviour**
Cognition**
Inhibition*
Aggression
Communication**
** p<.001, * p<.01

12. SASNOS Responsiveness: measures of change
T1-T2 Mean Difference t p
Effect Size SRM†
SEM % ‘improved’ ††
Effect Size SRM
(expected to improve)
SEM % ‘improved’ (expected to improve)
Total
5.25
6.05 **
moderate
53.1
63.7
Interpersonal Behaviour
8.00
7.45
59.3
64.3
Cognition
8.32
8.81
large
64.8
68.2
Inhibition
2.61
3.13 *
small
35.9
68.8
Aggression
1.27
1.37 ns
32.4
77.8
Communication
2.67
3.72
43.4
66.7
** p<.001, * p<.01
†<20 ‘trivial’; ≥20 to <50 ‘small’; ≥50 to <80 ‘medium’; ≥80 ‘large’ effect size
†† T1-T2 difference > 1 SEM = meaningful change

13. Is change expected across all recipients of NbR?

14. SASNOS Responsiveness: change in mean ratings T1 vs. T2
Total Score
Interpersonal Behaviour
Cognition
Inhibition
Aggression
Communication
Not expected to change
Expected to change
** p<.001, * p<.01

15. SASNOS Responsiveness: measures of change
T1-T2 Mean Difference t p
Effect Size SRM
SEM % ‘improved’
(expected to improve) †
SEM % ‘improved’ (expected to improve) ††
Total
5.25
6.05 **
moderate
53.1
63.7
Interpersonal Behaviour
8.00
7.45
59.3
64.3
Cognition
8.32
8.81
large
64.8
68.2
Inhibition
2.61
3.13 *
small
35.9
68.8
Aggression
1.27
1.37 ns
32.4
77.8
Communication
2.67
3.72
43.4
66.7
** p<.001, * p<.01
†<20 ‘trivial’; ≥20 to <50 ‘small’; ≥50 to <80 ‘medium’; ≥80 ‘large’ effect size
†† T1-T2 difference > 1 SEM = meaningful change

16. SASNOS Responsiveness: service level performance indicators and benchmarking

17. Median change on SASNOS for PiC BIS NbR participants, 2016
Effect Size SRM
SEM % ‘improved’
Interpersonal Behaviour
moderate
55.9
Cognition
51.9
Inhibition
large
62.5
Aggression
75.0
Communication
Total Score
57.7
‘Improvement’ indicators for SASNOS change scores, PiC BIS NbR participants ‘in residence’ 2016

18. SASNOS Responsiveness: determining individual change
Probabilities of the normal curve applied to SEM values
68% probability T1-T2 difference falls ±1 SEM
96% probability T1-T2 difference falls ±2 SEM
If T1-T2 difference > 1 SEM = meaningful change
If T1-T2 difference > 2 SEM =
meaningful change & beyond error

19. SASNOS
SEM
T1-T2
Outcome
Interpersonal Behaviour
3.77
19.3
> 2 SEM
Cognition
2.67
15.5
Inhibition
4.56
1.6
< 1 SEM
Aggression
3.58
-3.3
Communication
3.79
-6.2
< 2 SEM
Total Score
2.88
8.7

20. In conclusion
Study provides evidence SASNOS reliably measures change over time in NBD symptoms
Extends the psychometric properties reported by Alderman, Wood and Williams (2011), confirming a role for use of the instrument by clinicians and researchers investigating the multidimensional outcomes arising from ABI
Recommendations
When question asked necessitates specifying an overall index of the magnitude of change, SRM is recommended. Is simple to calculate using repeated SASNOS assessments and the ES magnitude determined using Middel and van Sonderen’s (2002) solution
When the main goal is to determine individual change in scores, SEM thresholds recommended, associated with MDC and MIC

21. Tom Sustained a TBI as a result of a road traffic accident aged 30
Neuropsychological assessment revealed specific cognitive deficits in memory and executive functioning
Disinhibition and aggression
Various residential placements and inpatient admissions

22. On Admission Disinhibition Aggression
Limited initiation of functional tasks
Limited attendance and engagement in group sessions
Low mood

23. Tom’s Neurobehavioural Rehabilitation Programme
Timetable of group and individual therapeutic and leisure activities
Behavioural programmes
Psychoeducation
Compensatory strategies
Individual psychological therapy
Family work
Medication

24. Outcomes Reduction in disinhibition Reduction in aggression
Some increase in insight
Engagement
Repeat neuropsychological assessment found memory and executive functioning deficits remained static

25. Pre and Post Neurobehavioural Rehabilitation
General improvement in subdomains and reduction of support and structure
Cognition remains static
Interpersonal behaviour, inhibition and aggression all 2 SEM
TDT scores Pre Neurobehavioural Rehabilitation
Support ratings
TDT scores Post Neurobehavioural Rehabilitation
Support ratings

26. Pre and Post Neurobehavioural Rehabilitation
TDT scores Pre Neurobehavioural Rehabilitation
Support ratings
TDT scores Post Neurobehavioural Rehabilitation
Support ratings

27. SASNOS SASNOS was sensitive to:
Tom’s progress in his rehabilitation programme
areas that had remained static i.e. cognition
areas where support and/or intervention was still being offered to further improve or maintain functioning

28. What’s Next: SASNOS-Revised
No tool is perfect the first time….

29. NBD and Communication Current version has only 4 Communication items
Feedback indicated current set of communication items was insufficient
30 new communication items
The original SASNOS communications items were retained based on their statistical properties but unfortunately that does not always translate into real world application.
The current communication items primarily evaluate verbal communication, and word use: for example “Speech is characterised by repetition and excessive wordiness”
To address the feedback 30 new communication items have been developed by working with a speech and language therapist as well as clinicians to identify more relevant items.

30. As you can see from the example questions-the current items at assess a wider range of communication behaviours- including non verbal communication (standing too close to others), verbal communication (speech pattern and volume) as well as self-awareness of ability to effectively communicate. Items are positively and negatively coded

31. Validity and Reliability
Recruit Larger Samples
Neurologically Healthy: 300 (SASNOS-R) vs 100 (SASNOS)
Stability of Neurobehavioural symptoms and behaviours in a normative sample
50 Participants over a two week period
While assessing the validity of the new communication items, it makes sense to re-assess the reliability and validate of the rest of the SASNOS and to re-develop the standardised scores using a larger sample. Consequently we are using Survey Monkey to run a two part study. The first part will be to collect data from 300 neurologically healthy participants to calculate a new set of standardised scores. At the moment the study is on-going and we have collected 250 responses with 175 fully complete response (70% completion rate).
The second part of the study is to run a short longitudinal study where participants complete the SASNOS-R twice over a two week period. This will help us assess reliability in responses.
Once we have the neurological healthy data we will collect data from those who have suffered from an acquired brain injury.
Benefits of normative data: allows use to understand how dynamic certain behaviours/symptoms are. This can then inform decisions regarding clinical behaviours
Additional need to validate self/other rated version on a larger scale

32. Want To Be Involved? Take part in one of our current studies
Help spread the word about our studies
Follow us on
us:
Website in Development
If the SASNOS seems like a scale who think might be useful for you to use or you already use it and you want to be more involved in it’s development there are lots of ways to get involved. As already mentioned we are running two studies to validate the SASNOS-R and assuming you meet our acceptance criteria you can take the questionnaire online. Maybe you have already taken part in one of our studies, you can always spread the word by posting it on a organisation pages, or facebook group etc.
If you want to get in touch with an idea for a colloraboration or help us with some future developments such as validating the questionnaire in other NBD that arise from progressive neurological conditions you can us or get in touch on twitter. Twitter is also a great place to find out about updates and new research relating to the SASNOS. You can also if you want a copy of the SASNOS to use in your practice or in research.
We are currently developing a website where you will also be able to find out projects and news relating to the SASNOS
Validated specifically for ABI but looking to validate in other NBD areas
On going use/feedback
Introduced the SASNOS
It’s clinical use
New developments: Dependency rating scale

33. Thanks for
Listening