TCP Treatment Change Project Overcoming problems associated with the utilisation of RCTs in forensic settings Jane Clarbour, Cynthia McDougall and Amanda Perry for the Treatment Change Design Team (TCDT). RCT in the Social Sciences: Challenges and Prospects

Treatment Change Design Team Acknowledgements TCP Treatment Change Project Treatment Change Project: Evaluating cognitive behavioural programs in prisons Treatment Change Design Team Cynthia McDougall Roger Bowles Jane Clarbour Amanda Perry David Richardson Jeremy Miles Catherine Hewitt Padraic Monaghan Ben Cross Funded by Home Office, in collaboration with RDS and OBPU. Funded for 2 years RCT in the Social Sciences: Challenges and Prospects

RCTs in the UK Criminal Justice System TCP Treatment Change Project Review of RCT literature in UK CJS (Farrington & Welsh, 2003) RCTs at height of popularity in 1960s in the UK CJS Clarke and Cornish (1970’s) disillusionment with RCTs in the CJS following their study of the Kingswood training school RCT Therapeutic community Traditional regime  2 year follow up = no difference in reconviction rates RCT in the Social Sciences: Challenges and Prospects

RCTs in the UK Criminal Justice System TCP Treatment Change Project Martinson (1974) doctrine that Nothing Works: Led to policy change in CJS and reduction of RCTs to evaluate CBT programmes What Works literature of 1980’s Led to increasing use of accredited CBT programmes Increasing number of evaluations using pre-to-post test design Department of Health/Home Office 2000 White Paper Part II Reforming the Mental Health Act suggested: “There is a need to undertake long term randomised trials with long term follow up” (section 6.53) RCT in the Social Sciences: Challenges and Prospects

Problems encountered TCP Feasibility studies Treatment Change Project Feasibility studies (Farrington & Jolliffe, 2002; Farrington et al., 2001). Young offenders had education disrupted Institutional staff Insufficient case flow Few people assessed RCT in the Social Sciences: Challenges and Prospects

Background to current RCT TCP Treatment Change Project Research on Cognitive Behavioural Programmes in 1990s Positive results on reconviction rates Research in 2000-2004 Mixed results unsure whether ETS and R&R courses are effective for all. RCT in the Social Sciences: Challenges and Prospects

Aims of the current project TCP Treatment Change Project To evaluate ETS courses. Which offenders benefit from treatment and under what conditions? What effect do programmes have on prison behaviour of those who complete & dropout from participation? RCT in the Social Sciences: Challenges and Prospects

The study TCP Treatment Change Project Evaluation of ETS treatment programme undertaken within H.M. Prison Service Cognitive-behavioural programme 10 prisons Target group Young & Adult male offenders Normal IQ On remand/sentenced Eligible to complete Enhanced Thinking Skills (ETS) Project evaluates short-term interim effects of the Enhanced Thinking Skills intervention programme. Aim of the study is to identify factors most associated with change in young male and adult offenders who took part in the intervention programme between April 2005 and October 2006. RCT in the Social Sciences: Challenges and Prospects

Waiting-list control design Group 1 Pre-test 4-week intervention Post-test 1 Post-test 2 Time 1 Time 2 Time 3 Group 2 (Waiting-list control) (Time) RCT in the Social Sciences: Challenges and Prospects

Waiting-list control design Group 1 Pre-test 4-week intervention Post-test 1 Post-test 2 Time 1 Time 2 Time 3 Group 2 (Waiting-list control) *Pre-PreTest (Test – Retest Reliability) * Group 2 will be administered a ‘pre-pre-test’ battery of tests at the same time as the Pre-test battery is administered to Group 1. This will enable test-retest reliability to be established prior to the administration of the intervention and those in Group 2 will not receive the intervention until after a minimum of a 3-month interval has taken place to avoid practice effects. TEST- RE-TEST PERIOD VARIES CONSIDERABLY DEPENDING UPON HOW THE COURSES ARE RUN AT THE PRISONS – E.G FOR COURSES THAT ARE RUN CONCURRENTLY THERE IS LITTLE OR NO CHANCE OF A PRE-PRE ASSESSMENT. (Time) RCT in the Social Sciences: Challenges and Prospects

Ethical considerations TCP Treatment Change Project Unique identification number Waiting list control avoids withholding treatment BUT random allocation considered unethical if release date is before start of treatment hence cohort design 1.6.2 Ethical considerations Ethical approval was gained from the Home Office and the Psychology Department at the University of York. Informed consent was obtained from each participant in the research regardless of whether they were randomised or not to the ETS course. Each participant was given a unique identification (ID) number for the purposes of random allocation. The use of an ID number in the random allocation allows individuals to be randomly allocated only once. This reduces the potential for contamination of the randomisation procedures by Treatment Managers and allows individuals to be traced when transferred to other establishments. Names and ID numbers remain with Treatment Managers in the prison establishments and data taken from the prison retains the ID number only. OBPU receives these ID numbers along with named participants with post course data returns. The ID number is used by both OBPU and the University of York to track participants and match participants with data entered into a database. All data is stored in a locked filing cabinet and all security issues regarding the data are taken into account when managing the data. Individuals not wanting to take part in the research project attend the ETS courses as usual and data from the test battery collected by OBPU will not form part of the data collection. RCT in the Social Sciences: Challenges and Prospects

Random Allocation Model TM ASSESS WITH RAs the suitability of individuals for ETS, this process is known as the semi-structured interview (SSI) – WASTED RESOURCES – especially at Pentonville prison Random allocation completed on a ratio of 2:1 – for a 10 place course we require a minimum of 15 individuals, 10 start the course immediately, 5 go on a waiting list and are guaranteed to start the next course. Problems with the random allocation so far are due to an insufficient pool of readily assessed individuals from which the randomisation can occur. For example, Treatment Managers only identify 12 participants for a 10 place course; for the randomisation to be successful we require a total of 15 participants. RCT in the Social Sciences: Challenges and Prospects

Design TCP Randomised control trial Experimental group Treatment Change Project Randomised control trial Experimental group Individuals randomly allocated to start course immediately Waiting list control group Individuals randomly allocated to start the next course PLUS: Additional cohort group Individuals prioritised by treatment managers as requiring treatment immediately A randomised controlled design was utilised to evaluate the effectiveness of the ETS programme since simple pre-test/post test analyses, although being able to address questions such as whether or not offenders have made improvements on various measures having completed treatment, cannot clarify whether or not some improvement would have been made for reasons other than the programme. For example, improvement could be simply due to measurement error or simply be an artefact of time. RCT in the Social Sciences: Challenges and Prospects

Example: Joe Bloggs TCP 25 years old Treatment Change Project 25 years old Convicted of Burglary and Theft (prolific offender) Referral to psychology department from sentence planning Highly motivated to attend ETS course Release Date (4 weeks after completion of course) NOT RANDOMLY ALLOCATED PROLIFIC OFFENDER – GOVERNMENT POLICY TO GIVE THESE INDIVIDUALS PRIORITY FOR TREATMENT THEREFORE MUST BE ON NEXT COURSE RELEASE DATE IS 4 WEEKS AFTER COMPLETION – WHICH WOULD BE TOO SHORT IF RANDOMLY ALLOCTED TO THE WAITING LIST RCT in the Social Sciences: Challenges and Prospects

Problems in random allocation TCP Treatment Change Project Formation of the group Group Dynamics (e.g., motivational factors) Operational decisions (e.g., security issues) Offender characteristics (e.g., release dates) Static factors (e.g., prison population) 1.8.1. Methodological concerns Treatment Managers at the majority of establishments report this as a resource problem. At one establishment this is due to the role of the establishment (e.g. high security tends to have a static population). Two strategies have been employed to tackle this issue. Research Assistants are being trained to conduct the SSIs and, secondly, the University of York has requested that three additional prisons are recruited to join the project. RCT in the Social Sciences: Challenges and Prospects

Reasons for non-completion TCP Treatment Change Project Change in prisoner status (e.g., mains to VP). Removed from course due to behaviour Transferred to another prison Released Deselected from course RCT in the Social Sciences: Challenges and Prospects

Summary TCP Despite advantages of RCT, few have been conducted. Treatment Change Project Despite advantages of RCT, few have been conducted. Acknowledgement of the practical and ethical implications in a forensic setting BUT  333 individuals have been randomly allocated so far…. one of the largest trials to be conducted 333 individuals randomly allocated target of 400 required RCT in the Social Sciences: Challenges and Prospects

RCT in the Social Sciences: Challenges and Prospects Centre for Criminal Justice Economics & Psychology Wentworth College Heslington York, YO1O 5DD E-mail:criminaljustice@cj.york.ac.uk Tel: +44 (0) 1904 434880 RCT in the Social Sciences: Challenges and Prospects