Enhancing Practice in Work with Offenders: the Role of Evaluation Jean Hine, De Montfort University

Workstream 1: Aim To learn: – Better understanding of what evaluation is undertaken, who does it, how, why, with what result – What facilitates/inhibits – What might help To develop – Evaluation guidance/toolkit – Library of examples of evaluation practice

Four Workstreams Good and Effective Practice 3. Literature Review 1. Evaluation 4, EPR 2. SEED

Interviews across the EU Mapping of evaluation Working with 2 pilot evaluations Co-ordination with workstreams 2, 3 and 4 Development of evaluation guidance Workstream 1: Components

Mapping More scoping than mapping. Not comprehensive but does include good range countries, organisations, individuals Complexity – language and meaning – range organisations involved – structuring of work with offenders in different countries

Evaluation takes many forms Specific independent research studies MoJ national studies/reviews Monitoring/management information Target assessment Compliance Project outcomes Individual casework Serious case reviews

Who evaluates Some countries have strong commitment to evaluation and active programme, some do not see evaluation as important Most countries have some governmental level unit undertaking some data work, some basic statistical, some special centres/units Most countries have some university involvement but approach varies by discipline Some use of independent organisations

What is evaluated? Specific programmes and practices – Electronic monitoring, thinking skills, domestic violence, sex offender programmes, etc Much focus on recidivism/outcomes – But different abilities to assess Competition with prison evaluation Generally short term impact

What’s not? Limited: Systematic evaluation of general supervision Service user evaluation Detailed analysis of routinely collected data

Key learning from Mapping Wide variation Most evaluation is centrally led – Central commitment important to enable – But prioritises central concerns – Tends to be large scale and quantitative There are interesting pieces of work which need to be more widely known General commitment to a desire for more evaluation, sometimes externally driven

Facilitators for evaluation Central commitment External pressures – EU – Funders – Government threat Individual enthusiasm

Inhibitors for evaluation Cultural/practitioner resistance Limited Resources – Time as well as money Lack of skills/knowledge Lack of interested audience Overestimation of the task Limited awareness of possibilities Lack of clarity about objectives of practice

Pilot work Shared process Working together In some ways artificial but beneficial – for pilots – for guidance Useful differences and similarities Two partners

Italy NGO, large scale drug rehabilitation community Offending not a direct focus but desistance an anticipated benefit Formative evaluation How to retain more alternative sentencees at end of sentence Mixed approach – Focus groups – Data analysis Highlighted importance of shared language

England Social enterprise offering peer support for employability Offending not direct focus but an anticipated benefit Formative evaluation How to improve success of specific project with probationers

Common issues Identifying evaluation question Data system limitations Designing manageable project Resources Time frame Insider role tension Support Changing context

Pilots’ Focus Group Right time to evaluate Political context Availability of respondents Rigour and confidence Ethical issues Written guidance too ‘academic’ Cop or coach? Critical friend/benevolent challenging Evaluation is time and labour intensive (but worth it!)

How does this inform guidance? Clarity Flexibility Range of uses/users Different starting points Making the most of existing data Making evaluation manageable

International comparability difficult Differing terms and meanings Differing criminal justice processes Different criteria for sanctions Different legal contexts

Example of Recidivism Usually means reconviction Availability of data: – Routine standard collection or not – Data protection laws Definition of conviction – Legal frameworks Time frames – Judicial procedures

The Hope Quality evaluation Sound evidence Informed practice Good outcomes