1. Mixing Validated Outcome Measures and Social Validity Tools to Monitor Outcomes in Psychotherapy
Janet Vogt, Kim Ankers, Barry Isaacs
Surrey Place Centre
BACKGROUND AND PURPOSE
RESULTS
DISCUSSION
Children and youth with developmental disabilities (DD) are at risk for developing mental health difficulties1-4, but research on the effectiveness of psychotherapy in this population is lacking1,5,6.
This poster reports on an outcome evaluation of psychotherapy for children and youth at Surrey Place Centre (SPC).
Group Level Analysis:
There were statistically significant improvements in the total behaviour problem score (TBPS) and 3 of 5 sub-scale scores on the DBC (Table 1).
At 6 months, parent ratings of the current size of the problem on the GCQ-P were positively correlated with all DBC scales (Table 2).
Agreement between therapist and parent ratings on the GCQ were strong.
Individual Level Analysis:
Applying the RCI, 5 clients scored a big enough improvement in TBPS for it to be considered reliable (Table 1).
6 clients moved from above to below the TBPS clinical cutoff (58th percentile); 4 of these achieved reliable change.
11 clients showed reliable improvement in one or more DBC scale. One client showed deterioration (Table 3).
Using the RCI to determine statistically significant and reliable improvements for individuals is a better indicator of program success than statistical comparison of mean changes at the group level.
In this study, about half of the clients showed reliable improvement in at least one DBC sub-scale after 6 months of therapy.
Using tools such as the GCQ enriches our understanding of the social significance of changes, as reflected in these comments:
This preliminary analysis compares baseline and 6-month data. It does not represent the end of therapy for most clients.
Future Directions:
Analysis of data for clients who have completed therapy.
Exploration of clinical, as opposed to statistical, significance.
Exploration of the factors (e.g., age, gender, clinical presentation) associated with improvement or deterioration.
References:
(1) Arthur, A. (2003). The emotional lives of people with learning disability. Brit J Learn Disabil, 31,
(2) Einfeld, S.L., & Tonge, B.J. (2002). Manual for the Developmental behaviour Checklist (2nd edition). Melbourne, Australia: Monash University Centre for Developmental Psychiatry.
(3) Emerson, E. (2003). Prevalence of psychiatric disorders in children and adolescents with and without intellectual disability. J Intellect Disabil Res, 47(1),
(4) Whitaker, S, & Read, S. (2006). The prevalence of psychiatric disorders among people with intellectual disabilities: An analysis of the literature. J Appl Res Intellect Disabil, 19,
(5) Butz, M.R., Bowling, J.B., & Bliss, C.A. (2000). Psychotherapy with the mentally retarded: A review of the literature and the implications. Prof Psychol Res Pr,31(1),
(6) Hurley, A.D. (1989). Individual psychotherapy with mentally retarded individuals: A review and call for research. Res Dev Disabil, 10,
(7) Tassé, M.J., Aman, M.G., Hammer, D., & Rojahn, J. (1996). The Nisonger Child Behavior Rating Form: age and gender effects and norms. Res Dev Disabil, 17(1),
(8) Jacobson, N.S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol, 59,
PSYCHOTHERAPY AT SPC
One-hour sessions, usually once per week
Integrative therapy approach
Flexible and responsive to client profiles
Supportive client-clinician relationship
Engages “key players” in the client’s life
Outcome measures at baseline, every 6 months, and end of therapy
Developmental Behaviour Checklist - parent version (DBC)
Nisonger Child Behaviour Rating Form - parent version (NIS)7
General Change Questionnaire – parent (GCQ-P) and therapist (GCQ-T) versions
Table 1.
Assessing change at the group and individual level.
DBC scales
Mean raw score (Percentile)
Reliable change index
Time 1
Time 2
Number that changed reliably
TBPS: Total Behaviour Problem Score
55.1 (72nd)
(56th) *
5 better; 1 worse
SS1: Disruptive/anti-social
(78th)
(65th) **
9 better; 1 worse
SS2: Self-absorbed
(54th)
9.5 (40th) *
SS3: Communication disturbance
7.9 (78th)
6.8 (70th)
2 better
SS4: Anxiety
5.3 (74th)
5.2 (74th)
no reliable changes
SS5: Social relating
5.6 (74th)
4.1 (60th) **
3 better *
Time and Time 2 different at p < 0.05 **
Time and Time 2 different at p < 0.01
Parent: “[She] is able to understand her emotions better and why she is feeling that way [anxious] and utilize the tools to deal with them which gives her more control of the situation.”
Therapist: “[She] is able to tolerate several medical appointments toward the end of our sessions. Increased compliance, decreased anxiety. She is better able to discuss/identify her feelings.”
METHODS
Sample Characteristics:
25 clients (11 ♂; 14 ♀) with complete data at baseline and 6 months
Average age at start of therapy = 14 years (range = years; 75% are 13 or older)
Statistical Analyses:
This study focussed on analysis of the DBC and GCQ measures
Baseline and 6-month data for total and subscale (SS) scores on DBC compared by paired t-test
Spearman correlations between the GCQ tools (GCQ-P; GCQ-T) and the DBC
The Reliable Change Index (RCI), which accounts for test-retest reliability and internal consistency, was used to assess significant change on an individual client level8
Table 2.
Correlations between DBC scores and parent rating of size of problem (GCQ-P) at 6 months.
Scale r
TBPS: Total Behaviour Problem Score
0.73
SS1: Disruptive/anti-social
0.59
SS2: Self-absorbed
0.69
SS3: Communication disturbance
0.75
SS4: Anxiety
0.67
SS5: Social relating
0.64
Table 3.
Number of clients that changed reliably according to one or more DBC scale.
Number of scales
Number of clients 1
6 better 3
3 better; 1 worse 4
1 better 5
** All correlations significant at p < 0.01