Reducing bias in randomised controlled trials involving therapists: To randomise or not to randomise - what are the questions? Alex Wright-Hughes Monday, May 8th, 2017

Objectives Reducing bias in randomised controlled trials involving therapists: To randomise or not to randomise - what are the questions? Background Rationale Trial examples SHIFT OK-Diabetes TIGA-CUB Challenges Conclusions

Partially Nested Design Therapist variation Clinical trials of behavioural and psychological interventions often involve therapists or other clinical specialists. Adds further complexity to trial design: Clustering effects ’multilevel’ randomised trials Partially nested design Therapist effects Therapist allocation: Randomisation? Partially Nested Design Therapy 1 Therapy 2 T1 T2 T3 T4

Internal validity Allocation of therapists to patients Involvement of more than one therapist per site can introduce selection bias - therapists can choose which patients they treat. Therapist selection based on participant characteristics, e.g. the ‘best’ treat the ‘worst’ - confounds the effect of therapists and patient characteristics. Limits the inferences a trial is able to make, particularly where the therapist effect is of interest. Reduces the ability to attribute treatment effects to treatment alone, thus impacting a trial’s internal validity. Walwyn R, Roberts C (2010) Therapist variation within randomised trials of psychotherapy: Implications for precision, internal and external validity. Statistical Methods in Medical Research

Random allocation of therapists to patients To avoid this source of selection bias, allocation of therapists to patients can be concealed and random. However, doing so can be problematic under constraints of therapist’s capacity, location, and a trials relative priorities. Therapist Intervention Patient Random Non random

Example I: SHIFT HTA-funded phase III definitive RCT set in 41 Child and Adolescent Mental Health Services (sites) Teams of three Family Therapists each, 24 therapists overall, worked together to deliver therapy across services. Allocation of lead therapist to patients: Random: where teams worked across services Aligned: where therapists more aligned within a particular service. Random allocation accounting for full and part-time working Manchester All 15 Sites Yorkshire Leeds 3 Sites Bradford 4 Sites W & E Yorkshire 7 Sites London SLAM OXLEAS N London City & Hackney 2 Sites Randomly allocated lead therapists Service aligned lead therapists

% participants seen by allocated lead therapist Example I: SHIFT Randomised therapists 76% compliance Reasons where not possible due to: case load (40%) therapist leave/organisational issues (29%) clinical reasons (11%) therapist changeover (9%) Aligned therapists 65% compliance Reasons more likely due to case load (61%) or organisational issues (27%) Impact of different recruitment rates across sites % participants seen by allocated lead therapist

Example II: TIGA-CUB NIHR RfPB-funded feasibility RCT set in 6 sites involving 14 child psychotherapists Variable numbers of therapists per service Separate therapist allocation to child & primary carer participants Trial therapy provided ‘within’ therapists usual clinical role Greater constraints on therapist capacity (one case at a time) Random allocation of therapists: Possible for 9/16 (56%) patients 7/16 (44%) allocated based on single therapist availability. Trust 1 Dudley Site 1 6 therapists Site 2 3 therapists Trust 2 Bradford Site 3 & 4 Trust 3 Sheffield Site 5 & 6 2 therapists

Example III: OK-Diabetes NIHR HTA-funded feasibility RCT recruited community-based participants with diabetes and a learning disability. Diabetes nurse specialists were allocated to participants based on prior contact to ensure consistency in contact Nurse randomisation occurred where there had been no prior contact. Considerations Vulnerability of patients limited randomisation Timing of random allocation to nurse Screening visit: Nurse X SSM TAU Nurse X Screening visit: No Nurse SSM TAU Nurse 1 Nurse 2

Challenges Challenges have included the practicality of randomly allocating therapists within a restricted service, therapist capacity following variable recruitment rates, and deviations from the allocated therapist. Success of random allocation to therapists influenced by: Therapist capacity and context Number of therapists per service Location: Randomisation may be possible in some but not all sites Pre-existing therapist-patient relationships: Timing of allocation to therapists Difficulties when limited capacity, especially if few therapists

Conclusions Random allocation of therapists to patients improves internal validity: avoids selection bias and allows causal interpretation of therapist effects The context in which therapists work, their capacity and the needs of the trial population all influence the feasibility and appropriateness of randomly allocating therapists. Implications for analysis: under the intention-to-treat principle participants are analysed as allocated - how to analyse according to therapist?

The SHIFT, TIGA-CUB & OK-Diabetes teams Acknowledgements University of Leeds: Rebecca Walwyn Amanda Farrin Liz Graham Sadie Reed The SHIFT, TIGA-CUB & OK-Diabetes teams

Questions? Alex Wright-Hughes – Statistician a.wright-hughes@leeds.ac.uk Dr Rebecca Walwyn – Statistician r.e.a.walwyn@leeds.ac.uk Professor Amanda Farrin – Statistician a.j.farrin@leeds.ac.uk Dr Sadie Reed – Trial Manager s.n.Reed@leeds.ac.uk

References Walwyn R & Roberts C. Therapist variation within randomised trials of psychotherapy: Implications for precision, internal and external validity. Statistical Methods in Medical Research. 2010; 19(3). Wright-Hughes A, et al. Self-Harm Intervention: Family Therapy (SHIFT), a study protocol for a randomised controlled trial of family therapy versus treatment as usual for young people seen after a second or subsequent episode of self-harm. Trials. 2015; 16 (1). Edginton E, et al. TIGA-CUB: Trial on improving Inter-Generational Attachment for Children Undergoing Behaviour problems: study protocol for a randomised controlled feasibility trial of manualised psychoanalytic child psychotherapy versus treatment as usual for children aged 5-11 with treatment resistant conduct disorders and their primary carers. Trials (submitted). Walwyn RE, et al. Supported self-management for adults with type 2 diabetes and a learning disability (OK-Diabetes): study protocol for a randomised controlled feasibility trial. Trials. 2015;16(1).