Evaluation at The Prince’s Trust Fire Service Prince's Trust Association meeting 18 th February 2010 Subtitle

Context to Trust evaluation In 09/10 the Trust plans to help over 27,000 new young people ➔ & nearly 14,000 ongoing participants ➔ on six core programmes, local & pilot programmes ➔ all young people should be from our core target groups ➔ we aim to support young people into ‘Positive Outcomes’ – education, training, jobs, volunteering ➔ our evaluation system overarches all programmes

The Trust evaluation system Ongoing monitoring & evaluation ➔ Data collection ➔ Performance reporting ➔ Analysis Evaluation projects ➔ Independent evaluation ➔ Internal evaluation

Ongoing evaluation & monitoring

Data collection Young people on all programmes: Participant Profile Form at start Outcome form 3 months after leaving ➔ Under review - piloting new methods but plans not finalised to extend to 6 months improve consistency of collection enhance soft outcomes Additional programme data– e.g. Team End of Team a Team Complete Sheet Achievement of qualifications – difficulties matching C&G data Data entered on Trust On Track – bespoke Client relationship Management system Quality of reports relies on accurate, legible, timely provision of forms

Analysis & reporting Do our programmes reach disadvantaged young people? Are we reaching target groups? How may young people are we helping? Profile of young people we support KPIs include ➔ no / % against plan ➔ % in target groups ➔ % in sub-target groups Detailed statistics: ➔ age, gender, ethnic background

Analysis & reporting Are programmes being implemented effectively? KPIs % who would recommend Team to others Programme KPIs Retention rate Average Team size

Analysis & reporting Do programmes make a difference to young people? To what extent do young people progress into employment, education & training? KPIs % ‘Positive Outcome’ - in employment, education, training, volunteering at 3 months % skills improvement % personal development Detailed analysis ➔ % improved in individual skills (e.g. confidence, team working, motivation etc) ➔ % life satisfaction

Use of evaluation data within Trust Monitor performance – for the Trust overall, programmes, regions ➔ Including key performance indicators against targets Informs programme management ➔ discuss variations in performance & share good practice Senior management can oversee and compare performance between different programmes Performance Reports to Council Reporting by different levels of geography, programmes, specific young people, funders.

Use of evaluation data – partners & stakeholders Delivery Partner management meetings Evidence to support fundraising activities Reporting to existing funders on success Supporting partnership working Overall communications on the Trust’s work and its impact - Powerful messages for media, government policy makers & key stakeholders e.g. More than three in four (78%) young people we supported last year moved into work, education or training Examples - Good statistics on Team to show targeting high volumes of NEET & getting 2/3 into jobs/education & training have helped us ➔ gain our previous national agreement for funding from LSC ➔ national agreement from JobCentrePlus to allow Team members to keep benefits on Team ➔ significant corporate support – e.g. Marks & Spencer's

Evaluation projects

We commission independent evaluations by external evaluators We undertake small–scale internal evaluation projects independent evidence on programme outcomes & effectiveness Inform programme development Valuable for external audiences e.g. potential funders, partners explore a specific issue or area of interest – e.g. Football Initiative on Team National Trust partnership Social work students on Team opportunity to: ➔ use different methods (e.g. qualitative data) ➔ hear young people’s perspectives ➔ explore & understand other evaluation questions not answered in ongoing system e.g. longer-term outcomes Implementation effectiveness

E.g. Evaluation of Team Programme (08) Methodology: –Interviews with over 100 stakeholders, consultation with other 675 current & past Team members, desk based analysis – document review, data analysis Found that the Team Programme remains ‘successful, effective and fit for purpose’ has a positive impact on young people’s lives: a key success of the programme is the significant personal development journey that young people experience - ‘the Team journey’ most young people find that Team builds their team working skills (97%), confidence (95%) and motivation (90%) employment is the most common progression route and also the one most sought after 89% find the qualification they gained on Team helpful in getting a job or joining a course. Delivery effectiveness is enhanced by various factors including: the mix of the team, the presence of a support worker or assistant team leader and an employed team member and the expertise, energy, experiences and commitment of the Delivery Partner staff