Service-related research: Therapy outcomes audit Sarah Howley Trainee Clinical Psychologist UCL

Background (1) Big changes in NHS – current emphasis on NICE guidance & evidence-based practice NICE currently does not consider practice-based evidence as equivalent to RCT evidence CBT is NICE-recommended therapy for depression and anxiety problems based on RCT evidence

Background (2) BUT – does evidence of efficacy from RCTs prove the effectiveness of therapies in real-life clinical settings? Many say no! AND – does absence of efficacy evidence mean therapies are not effective? There has been very little research generally comparing CBT (“gold standard”) to other therapies

Rationale for study in this Service No recent audit of therapy outcomes in PTS Little explicit information about how clients are allocated to CBT or Exploratory waiting lists Little info on comparison of therapies (PCP, Existential and Cognitive Behavioural therapies)

Aims of current study To investigate: Potential factors influencing allocation to different therapy modalities: demographic variables (age, gender, ethnicity), problem characteristics (type, severity, duration) Are therapies equivalent at (a) reducing CORE-OM scores from pre- to post-therapy and (b) producing reliable and clinically significant change across therapy?

Methods CORE-OM forms used routinely in service across all therapeutic modalities Data taken from CORE database from 2002 to 2010 CBT sample = 386, PCP = 15, ExT = 45, “Exploratory therapy” = 5 Therefore: PCP, ExT + Exploratory amalgamated (“Exploratory Therapy - EPT” N = 65) and random sample taken from CBT dataset (N = 65; representative of full CBT dataset as determined by statistical analyses)

Methods Variables: Demographics: Age, Sex, Ethnicity Referral source (GP, PCMHT etc) Problem characteristics: type, severity & duration CORE-OM mean scores at Assessment (IAS), Pre-therapy (1st treatment session) and Post-therapy (last treatment session)

Sample characteristics GROUP N (M, F) Age (SD) Age Range Primary referral source Primary ethnic group (%) CBT 65 (21, 44) 41.66 (10.87) 18 - 64 GP (49%) White British (96%) EPT 65 (18, 47) 41.70 (10.90) GP (52%) White British (94%)

Outcomes analyses CORE-OM scores within therapy groups (Effectiveness): Assessment vs Pre-therapy (change on W/L) Pre-therapy vs Post-therapy (therapy-related change) Reliable Change Index – calculation to determine whether clients achieved reliable and clinically significant change (Jacobson and Truax, 1991)

Outcomes analyses CORE-OM scores between groups (comparison of therapies) Assessment scores – do clients vary in severity at assessment? Pre-therapy scores (baseline) Post-therapy scores – are therapies equivalent in terms of outcome i.e. reduction in CORE scores? Reliable Change Index – are therapies equivalent in terms of achieving reliable and clinically significant change?

Results Allocation to therapeutic modality: Only difference between therapy groups was in primary problem type Anxiety disorders significantly more likely to be found in CBT group as primary presenting problem Chi Square test: χ2 = 6.65, p < 0.01 GROUP Primary problem Secondary problem CBT Depression (68.8%) Anxiety (46%) Anxiety (25%) Depression (17%) EPT Depression (75.8%) Anxiety (6%) Anxiety (33%) Inter/p problems (14%)

Results: Within groups CBT & EPT: significant reductions in CORE domain scores from IAS to pre-therapy Effectiveness? CBT - sig reduction in all CORE domain scores pre-post EPT – sig reduction in all CORE domain scores pre-post except Risk (p = .059) Sig differences in CORE-OM domain scores Group Ax – Pre-Therapy Pre-Post CBT Functioning Risk ALL EPT Subjective Wellbeing ALL except Risk

Results: Between groups Comparing CBT and EPT on CORE-OM: IAS: No statistically significant differences between groups Pre-therapy: No statistically significant differences between groups HOWEVER: In CBT group Problems/Symptoms domain score was below clinical cut-off Post-therapy: CBT group had significantly lower Problems/Symptoms and All Items scores compared to EPT

Comparing CORE-OM scores STAGE Assessment Pre-therapy Post-therapy Subjective Wellbeing -0.39 p < .05 -0.32 n.s. -0.34 Problems/ Symptoms -0.12 -0.24 Life Functioning -0.17 -0.20 -0.22 Risk -0.14 ALL Items -0.27

Reliable Change GROUP 45.3% 54.7% 0% 50.8% 44.1% 5.1% Improvement Jacobson & Truax (1991) Method used to identify whether individual clients achieve reliable and clinically significant change (improvement OR deterioration) This indicator of clinical significance is distinct from statistical significance There was no statistically significant difference between the groups in terms of no. of clients achieving reliable improvement GROUP Improvement No change Deterioration CBT (n = 53) 45.3% 54.7% 0% EPT (n = 59) 50.8% 44.1% 5.1%

Recap of results No differences observed in age, sex, ethnicity, referral source, problem severity or chronicity between therapy groups Anxiety disorders more likely to be main problem in CBT group Both therapies showed statistically significant improvement in CORE scores across domains (apart from EPT group in Risk but score below clinical cut off) CBT group showed significantly more improvement on Problems/Symptoms domain (but were below clinical cut off at pre-therapy) No statistically significant difference in number of clients achieving reliable clinical improvement between groups

Interpretation of results Clients presenting with anxiety as main difficulty tend to be referred more often to CBT – Service is in line with NICE guidance on treating anxiety Other factors in allocation to W/L – client preference, length of waiting lists, assessor’s preferred model? No significant deterioration in Risk scores in CBT or EPT groups – effects of waiting list times on client risk? CBT targets specific “symptoms” – therefore unsurprising that CBT clients show more reduction in this domain? (assuming result is valid – small effect size…) Both CBT and EPT are effective in reducing CORE-OM scores from pre- to post-therapy in this Service and clients in both achieve comparable levels of reliable improvement

Limitations Relatively small sample sizes (n = 65 and smaller for most analyses due to missing data) Use of data from up to 10 years ago – does this reflect current service? Amalgamation of PCP, ExT & “exploratory” – difficult to draw conclusions (but unavoidable!) Use of non-parametric stats (increased risk of not finding significant differences i.e. type 1 error)

Recommendations Encourage all clinicians to record CORE-OM scores at each stage of therapy – vital to practice-based evidence for therapies offered here Carry out similar audits at regular intervals in order to establish a bedrock of practice-based evidence…