NAVCA Annual Conference 2006 The National Outcomes Dissemination Programme: Evaluation Mike Aiken and Rob Paton Public Interest and Non-profit Management Research Unit Open University

National Outcomes Dissemination Programme 1. What was the Programme? 2. Why outcomes? 3. Some Programme achievements 4. How was it implemented by infrastructure organisations? 5. What have we learnt from the programme? 6. What next?

1. National Outcomes Dissemination Programme (NODP) Aimed over 3 years to: increase the pool of individuals and organisations across the sector with the skills and confidence to train and mentor VCOs in outcomes management. improve the skills understanding and confidence of a diverse range of 500 VCOs throughout England. increase the understanding of Government and sector policy makers of the support and information needs of the sector in outcomes management.

Programme conception and delivery Concept: cascade training and development programme Funded by Big Lottery Devised & piloted by CES Major contribution of infrastructure organisations in early concepts, recruitment and delivery (especially NAVCA, ACRE, CEMVO).

2. Why outcomes? Some recent policy & practice drivers The increasing importance of stakeholder transparency, quality processes and an outcomes focus in the voluntary sector, government departments, funding and regulatory bodies The need to demonstrate voluntary organisations are effectively run and delivering on promises. Outcomes have become mainstream - for infrastructure organisations too! See (Charities Evaluation Services 2005; NCVO 2004; Home Office 2005; Big Lottery 2005; Charity Commission 2005; (Audit Commission 2005)

3. Some Programme achievements A 200 plus page manual containing curriculum and course organisation 8 four day cascade training courses were developed and delivered 76 Outcomes Champions trained in 9 English regions and 3 sub sectors Outcomes Champions cascaded training to over 631 Voluntary and Community Organisations

…longer term effects? The majority of Outcomes Champions have started to implement outcomes approaches in their own organisations Most Outcomes Champions intend to continue to deliver outcomes training or built it into their role The majority of VCOs trained are seeking to shift towards, develop or improve an outcomes focus in their organisation Early signs that VCOs trained over a year ago are maintaining and integrating the ideas into their ways of working,

4. How was it implemented by infrastructure organisations? ‘Classic’ and ‘jazz’ styles by the Outcomes Champions in their training and development role of VCOs OCs Striving to implement in own infrastructure organisation important Post- programme, OCs varied roles: –Activists: ‘deliver more courses and keep active’ –Integrationists: ‘building the work into their role’ –Programmers: ‘happy but programme has ended.’

5. Some programme learning Cascade worked Local adoption important for OCs & VCOs Importance for policy makers of outcomes - & their need to meet their own needs Funder education important

6. What next? Report & summary of evaluation: coming soon New programme: in development phase by CES Outcomes: likely to remain and become part of any new structures and crucial for infrastructure organisations Measurement: a search and find experience? The education of funders?

NAVCA Annual Conference 2006 The National Outcomes Dissemination Programme Evaluation: summary Mike Aiken and Rob Paton Public Interest and Non-profit Management Research Unit Open University

Additional notes Copies of the report and summary will be available in October from: Charities Evaluation Service 4 Coldbath Square London EC1R 5HL Further notes are included in the following pages

NODP - Programme in outline A phased cascade training programme devised and developed by CES Delivered over a three year period (2003-6) Active in all nine English regions Also active in three ‘sub-sectors’ (BME; rural community sector and volunteer development).

NODP - Programme detail Lead Trainers set up by CES provided training, information and mentoring support to 76 intermediary trainers known as ‘Outcomes Champions’) Outcomes champions recruited in partnership with infrastructure networks in each region & 3 sub-sectors (BME, Rural, volunteer development The aim was to equip Outcomes Champions’ to pass on their learning to VCOs in their networks. Outcomes Champions agreed to offer training and support to 10 further VCOs within 6 months

Programme delivery Outcomes Champions (OCs were recruited in partnership with infrastructure networks, such as CVSs Recruitment in each of the English regions (54 OC intermediaries in all) …and the three sectors of BME, rural and volunteer development organisations (22 OC intermediaries in all).

Outcomes Champions adoption strategies from cascade The ‘classical’ players - stayed close to the programme as written The ‘jazz players’ - improvised more freely from the programme Between classic and jazz - largely stayed but also improvised

Approaches to implementation by VCOs (A) No problem - some VCOs saw it as unproblematic (B) Step-by-step - saw it requiring significant shifts. (C1) Enthusiastic integrators - some had taken it on enthusiastically and rapidly. (C2) Culture change integrators - some VCOs found that implementation would take them longer and require a culture change. (D) Not right now, thanks! A few VCOs were sceptics & felt they had sufficient processes.

Programme achievements (1) A comprehensive 200 plus page manual containing curriculum and course organisation devised, piloted and delivered to Outcome Champions 8 four day cascade training courses were developed and delivered by CES’s Lead Trainers to Outcomes Champions (target 7) 76 Outcomes Champions were trained (target: 60) … in 9 English regions and the 3 sub sectors

Programme achievements (2) Outcomes Champions trained included: 5 working specifically with Black and Minority Ethnic voluntary organisations; 8 in Volunteer Development support organisations 9 in rural development infrastructure organisations Outcomes Champions cascaded training to over 631 Voluntary and Community Organisations (target 500) The VCO came predominately from local or community organisations Reports from OCs and VCO participants: course satisfaction was very high.

Programme impacts The impact of the programme is likely to be considerable: The majority of Outcomes Champions have started to implement outcomes approaches in their own organisations Most Outcomes Champions intend to continue to deliver outcomes training or have built this into their role The majority of VCOs trained by the Champions are seeking to shift towards, develop or improve an outcome focus of some kind in their organisation There are early signs that some VCOs trained over a year ago have maintained or integrated some of the outcomes thinking into their ways of working.