1. TCP
Treatment Change Project
Overcoming problems associated with the utilisation of RCTs in forensic settings
Jane Clarbour, Cynthia McDougall and Amanda Perry for the Treatment Change Design Team (TCDT).
RCT in the Social Sciences: Challenges and Prospects

2. Treatment Change Design Team
Acknowledgements
TCP
Treatment Change Project
Treatment Change Project: Evaluating cognitive behavioural programs in prisons
Treatment Change Design Team
Cynthia McDougall
Roger Bowles
Jane Clarbour
Amanda Perry
David Richardson
Jeremy Miles
Catherine Hewitt
Padraic Monaghan
Ben Cross
Funded by Home Office, in collaboration with RDS and OBPU.
Funded for 2 years
RCT in the Social Sciences: Challenges and Prospects

3. RCTs in the UK Criminal Justice System
TCP
Treatment Change Project
Review of RCT literature in UK CJS
(Farrington & Welsh, 2003)
RCTs at height of popularity in 1960s in the UK CJS
Clarke and Cornish (1970’s) disillusionment with RCTs in the CJS following their study of the Kingswood training school RCT
Therapeutic community
Traditional regime
 2 year follow up = no difference in reconviction rates
RCT in the Social Sciences: Challenges and Prospects

4. RCTs in the UK Criminal Justice System
TCP
Treatment Change Project
Martinson (1974) doctrine that Nothing Works:
Led to policy change in CJS and reduction of RCTs to evaluate CBT programmes
What Works literature of 1980’s
Led to increasing use of accredited CBT programmes
Increasing number of evaluations using pre-to-post test design
Department of Health/Home Office 2000 White Paper Part II Reforming the Mental Health Act suggested:
“There is a need to undertake long term randomised trials with long term follow up” (section 6.53)
RCT in the Social Sciences: Challenges and Prospects

5. Problems encountered TCP Feasibility studies
Treatment Change Project
Feasibility studies
(Farrington & Jolliffe, 2002; Farrington et al., 2001).
Young offenders had education disrupted
Institutional staff
Insufficient case flow
Few people assessed
RCT in the Social Sciences: Challenges and Prospects

6. Background to current RCT
TCP
Treatment Change Project
Research on Cognitive Behavioural Programmes in 1990s
Positive results on reconviction rates
Research in
Mixed results unsure whether ETS and R&R courses are effective for all.
RCT in the Social Sciences: Challenges and Prospects

7. Aims of the current project
TCP
Treatment Change Project
To evaluate ETS courses.
Which offenders benefit from treatment and under what conditions?
What effect do programmes have on prison behaviour of those who complete & dropout from participation?
RCT in the Social Sciences: Challenges and Prospects

8. The study
TCP
Treatment Change Project
Evaluation of ETS treatment programme undertaken within H.M. Prison Service
Cognitive-behavioural programme
10 prisons
Target group
Young & Adult male offenders
Normal IQ
On remand/sentenced
Eligible to complete Enhanced Thinking Skills (ETS)
Project evaluates short-term interim effects of the Enhanced Thinking Skills intervention programme. Aim of the study is to identify factors most associated with change in young male and adult offenders who took part in the intervention programme between April 2005 and October 2006.
RCT in the Social Sciences: Challenges and Prospects

9. Waiting-list control design
Group 1
Pre-test
4-week intervention
Post-test 1
Post-test 2
Time 1
Time 2
Time 3
Group 2
(Waiting-list control)
(Time)
RCT in the Social Sciences: Challenges and Prospects

10. Waiting-list control design
Group 1
Pre-test
4-week intervention
Post-test 1
Post-test 2
Time 1
Time 2
Time 3
Group 2
(Waiting-list control)
*Pre-PreTest
(Test – Retest Reliability)
* Group 2 will be administered a ‘pre-pre-test’ battery of tests at the same time as the Pre-test battery is administered to Group 1.
This will enable test-retest reliability to be established prior to the administration of the intervention and those in Group 2 will not receive the intervention until after a minimum of a 3-month interval has taken place to avoid practice effects.
TEST- RE-TEST PERIOD VARIES CONSIDERABLY DEPENDING UPON HOW THE COURSES ARE RUN AT THE PRISONS – E.G FOR COURSES THAT ARE RUN CONCURRENTLY THERE IS LITTLE OR NO CHANCE OF A PRE-PRE ASSESSMENT.
(Time)
RCT in the Social Sciences: Challenges and Prospects

11. Ethical considerations
TCP
Treatment Change Project
Unique identification number
Waiting list control avoids withholding treatment
BUT random allocation considered unethical if release date is before start of treatment
hence cohort design
1.6.2 Ethical considerations
Ethical approval was gained from the Home Office and the Psychology Department at the University of York. Informed consent was obtained from each participant in the research regardless of whether they were randomised or not to the ETS course. Each participant was given a unique identification (ID) number for the purposes of random allocation. The use of an ID number in the random allocation allows individuals to be randomly allocated only once. This reduces the potential for contamination of the randomisation procedures by Treatment Managers and allows individuals to be traced when transferred to other establishments. Names and ID numbers remain with Treatment Managers in the prison establishments and data taken from the prison retains the ID number only. OBPU receives these ID numbers along with named participants with post course data returns. The ID number is used by both OBPU and the University of York to track participants and match participants with data entered into a database. All data is stored in a locked filing cabinet and all security issues regarding the data are taken into account when managing the data. Individuals not wanting to take part in the research project attend the ETS courses as usual and data from the test battery collected by OBPU will not form part of the data collection.
RCT in the Social Sciences: Challenges and Prospects

12. Random Allocation Model
TM ASSESS WITH RAs the suitability of individuals for ETS, this process is known as the semi-structured interview (SSI) – WASTED RESOURCES – especially at Pentonville prison
Random allocation completed on a ratio of 2:1 – for a 10 place course we require a minimum of 15 individuals, 10 start the course immediately, 5 go on a waiting list and are guaranteed to start the next course.
Problems with the random allocation so far are due to an insufficient pool of readily assessed individuals from which the randomisation can occur. For example, Treatment Managers only identify 12 participants for a 10 place course; for the randomisation to be successful we require a total of 15 participants.
RCT in the Social Sciences: Challenges and Prospects

13. Design TCP Randomised control trial Experimental group
Treatment Change Project
Randomised control trial
Experimental group
Individuals randomly allocated to start course immediately
Waiting list control group
Individuals randomly allocated to start the next course
PLUS:
Additional cohort group
Individuals prioritised by treatment managers as requiring treatment immediately
A randomised controlled design was utilised to evaluate the effectiveness of the ETS programme since simple pre-test/post test analyses, although being able to address questions such as whether or not offenders have made improvements on various measures having completed treatment, cannot clarify whether or not some improvement would have been made for reasons other than the programme. For example, improvement could be simply due to measurement error or simply be an artefact of time.
RCT in the Social Sciences: Challenges and Prospects

14. Example: Joe Bloggs TCP 25 years old
Treatment Change Project
25 years old
Convicted of Burglary and Theft (prolific offender)
Referral to psychology department from sentence planning
Highly motivated to attend ETS course
Release Date (4 weeks after completion of course)
NOT RANDOMLY ALLOCATED
PROLIFIC OFFENDER – GOVERNMENT POLICY TO GIVE THESE INDIVIDUALS PRIORITY FOR TREATMENT THEREFORE MUST BE ON NEXT COURSE
RELEASE DATE IS 4 WEEKS AFTER COMPLETION – WHICH WOULD BE TOO SHORT IF RANDOMLY ALLOCTED TO THE WAITING LIST
RCT in the Social Sciences: Challenges and Prospects

15. Problems in random allocation
TCP
Treatment Change Project
Formation of the group
Group Dynamics (e.g., motivational factors)
Operational decisions (e.g., security issues)
Offender characteristics (e.g., release dates)
Static factors (e.g., prison population)
Methodological concerns
Treatment Managers at the majority of establishments report this as a resource problem. At one establishment this is due to the role of the establishment (e.g. high security tends to have a static population). Two strategies have been employed to tackle this issue. Research Assistants are being trained to conduct the SSIs and, secondly, the University of York has requested that three additional prisons are recruited to join the project.
RCT in the Social Sciences: Challenges and Prospects

16. Reasons for non-completion
TCP
Treatment Change Project
Change in prisoner status (e.g., mains to VP).
Removed from course due to behaviour
Transferred to another prison
Released
Deselected from course
RCT in the Social Sciences: Challenges and Prospects

17. Summary TCP Despite advantages of RCT, few have been conducted.
Treatment Change Project
Despite advantages of RCT, few have been conducted.
Acknowledgement of the practical and ethical implications in a forensic setting
BUT  333 individuals have been randomly allocated so far…. one of the largest trials to be conducted
333 individuals randomly allocated target of 400 required
RCT in the Social Sciences: Challenges and Prospects

18. RCT in the Social Sciences: Challenges and Prospects
Centre for Criminal Justice Economics & Psychology
Wentworth College
Heslington
York, YO1O 5DD
Tel: +44 (0)
RCT in the Social Sciences: Challenges and Prospects