Tired of hanging around Evaluating projects with young people

Content of the slide pack This slide pack aims to give a basic overview of issues to be considered in carrying out evaluation: –What is evaluation: why should you do it –The steps to planning your evaluation –Basic principles of evaluation –Deciding what outcomes to measure –Deciding how to measure your outcomes –Collecting and analysing your data The appendix includes further details and explanations of the terms used and examples of outcome measures.

What is evaluation Evaluation uses evidence to make a judgement about whether, and how well, a project has delivered its aims and objectives Evaluation looks at the change the project has brought about in young people: outcomes (sometimes called ‘impacts’) You will probably need some expert advice and input in designing and carrying out an evaluation – ask funders, data analysts or researchers in your council or CDRP Evaluation is an integral part of your project: it must be planned as part of project design – not as an afterthought

Why carry out evaluation? It gives evidence of a project’s impact Evidence can demonstrate accountability to funders and other stakeholders. It can be used in future funding bids Evidence can demonstrate whether the project is meetings the needs of young people. Evidence can demonstrate the value of the project to local communities. By looking at project costs and impacts you can demonstrate value for money Evaluation should involve project users and show that their views are important Regular evaluation (monitoring) helps inform project development – identifying any problems so that they can be resolved Regular evaluation helps to show project progress over time

Planning your evaluation Plan the evaluation when setting up the project to ensure you collect the right data Allocate resources to the evaluation – including staff time to collect evidence Think through project aims and objectives – you should measure progress against each objective Decide what outcomes you are going to measure and how to measure them Collect and analyse data Feed results into project development and future review process

Planning - basic principles of evaluation Collect data on the project when it starts (baseline), at regular intervals, and when it finishes. Collect evidence about young people when they join, at regular intervals, and when the leave. If possible, follow-up young people sometime after they have left. You must allow time for change to take place – so plan follow- up assessments for short-term projects. Focus your data collection on the things that your project influences. Young people’s behaviour can be influenced by many different factors which all contribute to reducing anti- social behaviour. Try to assess the contribution made by your project. Some outcomes are only achievable over the medium to long term – but progress can be measured by assessing the milestones young people go through to reach them (see appendix for details)

Deciding what outcomes to measure Outcomes demonstrate the impact of the project on young people and their communities. For example – residents are less worried about anti-social behaviour, young people re-engage with mainstream education To decide which outcomes to measure, ask yourself: –What is the reason for the projects existence? –What does the project want to achieve with the young people who attend? –What difference will attending the project make to the lives of young people or local residents? –How will people notice?

Deciding how to measure your outcomes Once you’ve decided on your outcomes, you need to consider the following questions when deciding how to measure them: –If the outcome has been achieved, what changes would you see in local residents and young people? –What evidence already exists about these changes – who collects this and how could we access it? –How could you go about measuring these changes yourself? To get a reliable assessment that allows you to compare progress in different ways, also think about: –Using several different outcome measures – some hard, some soft (see appendix for details) –Including outcomes that measure different aspects of progress e.g. behaviour, skills, attitudes etc (see appendix for details)

How to collect your data Make sure that collection and sharing of information complies with your organisation’s rules – and the law. Use data or records that already exist where possible Build in opportunities to measure outcomes – for example use exercises during a session to collect data or have a feedback discussion at the end of a session Use interactive feedback methods to engage young people: –Focus groups –Suggestion boxes –Graffiti walls, story boards –Young people interviewing each other as peer researchers –Votes –Videos, shared diaries –Web-based feedback methods Remember that data is not just numbers.

Analysing the data Collate data regularly rather than once a year – a simple spreadsheet will often help Look out for similarities and differences in outcomes for different groups of young people e.g. boys and girls Try to record findings from interviews or focus groups Get a balance between numbers and stories: –Proportions of young people with positive stories, key features in those stories, about the project –Report how many people were spoken to, and then summarise the main strengths and areas for improvement discussed –Using quotes to give examples of key themes With support, young people may able to help collate findings Ask for support from data analysts or researchers in your council or CDRP

Other sources of guidance Charities Evaluation Services provide a range of downloadable resources on monitoring, evaluation and measuring outcomes: The Home Office Crime Reduction Centre ‘Passport to Evaluation’ outlines what evaluation is, and the process for carrying out an evaluation: evaluation.htm The Crime Reduction site gives guidance on tracking progress and achievements, and a sample evaluation framework: IDeA guidance on tools to help services focus on outcomes: