Mixing Validated Outcome Measures and Social Validity Tools to Monitor Outcomes in Psychotherapy Janet Vogt, Kim Ankers, Barry Isaacs Surrey Place Centre BACKGROUND AND PURPOSE RESULTS DISCUSSION Children and youth with developmental disabilities (DD) are at risk for developing mental health difficulties1-4, but research on the effectiveness of psychotherapy in this population is lacking1,5,6. This poster reports on an outcome evaluation of psychotherapy for children and youth at Surrey Place Centre (SPC). Group Level Analysis: There were statistically significant improvements in the total behaviour problem score (TBPS) and 3 of 5 sub-scale scores on the DBC (Table 1). At 6 months, parent ratings of the current size of the problem on the GCQ-P were positively correlated with all DBC scales (Table 2). Agreement between therapist and parent ratings on the GCQ were strong. Individual Level Analysis: Applying the RCI, 5 clients scored a big enough improvement in TBPS for it to be considered reliable (Table 1). 6 clients moved from above to below the TBPS clinical cutoff (58th percentile); 4 of these achieved reliable change. 11 clients showed reliable improvement in one or more DBC scale. One client showed deterioration (Table 3). Using the RCI to determine statistically significant and reliable improvements for individuals is a better indicator of program success than statistical comparison of mean changes at the group level. In this study, about half of the clients showed reliable improvement in at least one DBC sub-scale after 6 months of therapy. Using tools such as the GCQ enriches our understanding of the social significance of changes, as reflected in these comments: This preliminary analysis compares baseline and 6-month data. It does not represent the end of therapy for most clients. Future Directions: Analysis of data for clients who have completed therapy. Exploration of clinical, as opposed to statistical, significance. Exploration of the factors (e.g., age, gender, clinical presentation) associated with improvement or deterioration. References: (1) Arthur, A. (2003). The emotional lives of people with learning disability. Brit J Learn Disabil, 31, 25-30. (2) Einfeld, S.L., & Tonge, B.J. (2002). Manual for the Developmental behaviour Checklist (2nd edition). Melbourne, Australia: Monash University Centre for Developmental Psychiatry. (3) Emerson, E. (2003). Prevalence of psychiatric disorders in children and adolescents with and without intellectual disability. J Intellect Disabil Res, 47(1), 51-58. (4) Whitaker, S, & Read, S. (2006). The prevalence of psychiatric disorders among people with intellectual disabilities: An analysis of the literature. J Appl Res Intellect Disabil, 19, 330-345. (5) Butz, M.R., Bowling, J.B., & Bliss, C.A. (2000). Psychotherapy with the mentally retarded: A review of the literature and the implications. Prof Psychol Res Pr,31(1), 42-47. (6) Hurley, A.D. (1989). Individual psychotherapy with mentally retarded individuals: A review and call for research. Res Dev Disabil, 10, 261-275. (7) Tassé, M.J., Aman, M.G., Hammer, D., & Rojahn, J. (1996). The Nisonger Child Behavior Rating Form: age and gender effects and norms. Res Dev Disabil, 17(1), 59-75. (8) Jacobson, N.S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol, 59, 12-19. PSYCHOTHERAPY AT SPC One-hour sessions, usually once per week Integrative therapy approach Flexible and responsive to client profiles Supportive client-clinician relationship Engages “key players” in the client’s life Outcome measures at baseline, every 6 months, and end of therapy Developmental Behaviour Checklist - parent version (DBC) Nisonger Child Behaviour Rating Form - parent version (NIS)7 General Change Questionnaire – parent (GCQ-P) and therapist (GCQ-T) versions Table 1. Assessing change at the group and individual level. DBC scales Mean raw score (Percentile) Reliable change index   Time 1 Time 2 Number that changed reliably TBPS: Total Behaviour Problem Score 55.1 (72nd) 44.3 (56th) * 5 better; 1 worse SS1: Disruptive/anti-social 19.9 (78th) 15.5 (65th) ** 9 better; 1 worse SS2: Self-absorbed 12.4 (54th) 9.5 (40th) * SS3: Communication disturbance 7.9 (78th) 6.8 (70th) 2 better SS4: Anxiety 5.3 (74th) 5.2 (74th) no reliable changes SS5: Social relating 5.6 (74th) 4.1 (60th) ** 3 better * Time and Time 2 different at p < 0.05 ** Time and Time 2 different at p < 0.01 Parent: “[She] is able to understand her emotions better and why she is feeling that way [anxious] and utilize the tools to deal with them which gives her more control of the situation.” Therapist: “[She] is able to tolerate several medical appointments toward the end of our sessions. Increased compliance, decreased anxiety. She is better able to discuss/identify her feelings.” METHODS Sample Characteristics: 25 clients (11 ♂; 14 ♀) with complete data at baseline and 6 months Average age at start of therapy = 14 years (range = 6 - 18 years; 75% are 13 or older) Statistical Analyses: This study focussed on analysis of the DBC and GCQ measures Baseline and 6-month data for total and subscale (SS) scores on DBC compared by paired t-test Spearman correlations between the GCQ tools (GCQ-P; GCQ-T) and the DBC The Reliable Change Index (RCI), which accounts for test-retest reliability and internal consistency, was used to assess significant change on an individual client level8 Table 2. Correlations between DBC scores and parent rating of size of problem (GCQ-P) at 6 months. Scale r TBPS: Total Behaviour Problem Score 0.73 SS1: Disruptive/anti-social 0.59 SS2: Self-absorbed 0.69 SS3: Communication disturbance 0.75 SS4: Anxiety 0.67 SS5: Social relating 0.64 Table 3. Number of clients that changed reliably according to one or more DBC scale. Number of scales Number of clients 1 6 better 3 3 better; 1 worse 4 1 better 5 ** All correlations significant at p < 0.01