What Gets Measured Gets Done Presented by Frances Head George Elliott

Many organisations fail to optimise the outcomes that they achieve. This is in part due to the lack of a strategic approach to evaluation and planning.

Not About Policing Organisations that measure their outcomes are much more likely to improve performance over time.

Whitelion and Social Responsibility Network have worked to together to build a multi-layered evaluation framework. This presentation discusses the methodology for the development of the framework and the implementation process.

The evaluation framework that was developed for Whitelion is based on Results-Based Accountability™ (RBA).

RBA has been used as an effective planning and evaluation tool for community and organisational development for nearly twenty years. It had its beginnings in the USA and is now used internationally in over 15 countries.

RBA asks three important questions: How much did we do How well did we do it Is anyone better off

The standard RBA model is two dimensional

Firstly, it looks at information for a given population group; determines some aspirational goals; and establishes a set of measures that reflect progress toward these goals. Secondly, it looks at how the specific client group is faring in relation to the population and how to improve outcomes.

In order to do this a set of Performance Measures needs to be established. This is done through a process that encourages prioritisation and collaborative partnerships.

One of the most important aspects of the model is that it is intended to encourage continuous improvement. This is partially facilitated by the fact that performance measures are monitored and charted over time.

The evaluation framework that was developed for Whitelion is intended to be multi-dimensional and to demonstrate the outcome relationships at a number of levels within the organisation.

A strategy for gathering information at the individual client level was needed in order to complete the framework.

Based on research done by Jenny Crosbie and Goetz Ottman: What are the most important issues in the lives of people with intellectual disabilities? A comparative Study

Photovoice Photovoice can be defined as a method of seeing the world through the viewpoint of people who are leading different lives (Wang and Burris, 1994)

Cameras are provided to members of underrepresented groups, and a “voice’ is created through the photographs that are taken.

The viewpoints of these underrepresented groups can be eye-opening since they often contrast with the majority of viewpoints found in society.

There are three main goals 1.Enabling people to record and reflect upon their strengths, goals and concerns 2.Promote critical dialogue and knowledge about important issues 3.Reaches important decision makers

By combining methodologies we were able to achieve a more holistic approach to performance evaluation, planning and continuous improvement.

Framework Implementation  National Staff Workshops  Assess current available data and fill holes  Build an evaluation module and relevant reports  Design and implementation of a Well Being Index & Client Surveys  Volunteer and stakeholder feedback Implementation of the framework through existing structures e.g. staff meetings, K.P.I’s and work plans.

 Assess available data and fill gaps

Case Study - Employment Program Data Collection # of young people being support in employment # of young people reaching 13, 26, 39 & 52 week outcomes Average length of employment for young people Reasons young people disengage Employers with the best outcomes Data input by staff. Reports are run and provided to all staff. Performance Measure % of young people engaged in the program who will be placed into full time employment. Average length of employment Reports are provided to program managers. New targets are set and ways to improve process are discussed in team meetings. This inturn increases professionalism and encourages continuous improvement. Client outcomes WBI show increases in the domains; standard of living, achievements in life and community connection. Client surveys will further explore areas like confidence, probable solving ability, financial stability and future security. Informs program practise. Provide for reporting and funding purposes. Staff engagement and morale.

 Build an evaluation module and develop relevant reports

Case Study- Employment Program Data Collection # of young people being support in employment # of young people reaching 13, 26, 39 & 52 week outcomes Average length of employment for young people Reasons young people disengage Employers with the best outcomes Data input by staff. Reports are run and provided to all staff. Performance Measure % of young people engaged in the program who will be placed into full time employment Average length of employment Reports are provided to program managers. New targets are set and ways to improve processes are discussed in team meetings. This in turn increases professionalism and encourages continuous improvement. Client outcomes WBI show increases in the domains; standard of living, achievements in life and community connection. Client surveys will further explore areas like confidence, probable solving ability, financial stability and future security. Informs program practise. Provide for reporting and funding purposes. Staff engagement and morale.

All clients will complete a Well Being Index at entry point to the program  Well Being Index & Client Survey; Design and implementation

Once per month, SYARD (client management system) will select 20 random young people from the system to re-survey. They will be re-complete the Well Being Index and respond to questions regarding service delivery. As a direct result of services received: Strongly agree AgreeNeutralDisagreeStrongly Disagree Not Applicable I am more confident I am more independent I am better at maintaining relationships  Well Being Index & Client Survey; Design and implementation

Case Study- Employment Program Data Collection # of young people being support in employment # of young people reaching 13, 26, 39 & 52 week outcomes Average length of employment for young people Reasons young people disengage Employers with the best outcomes Data input by staff. Reports are run and provided to all staff. Performance Measure % of young people engaged in the program who will be placed into full time employment Average length of employment Reports are provided to program managers. New targets are set and ways to improve process are discussed in team meetings. This inturn increases professionalism and encourages continuous improvement. Client outcomes WBI should show increases in key domains, including standard of living, achievements in life and community connection. Client surveys will further explore areas like confidence, probable solving ability, financial stability and future security. Informs program practice. Provides data for reporting and funding purposes. Increases staff engagement and morale.

Volunteers and other external stakeholders will be provided with opportunities to complete surveys on their experience. This will further enhance our services and strengthen partnerships across the community. Due to my experiences volunteering with WL OFA: Strongly agree AgreeNeutralDisagreeStrongly Disagree Not Applicable I am more aware of issues facing at risk young people I believe I am making a difference I would recommend volunteering for WL/OFA  Volunteer and stakeholder feedback

Challenges  Internal changes  Database development  Engagement of staff  Correct data entry  Staff capacity  Securing funding

Anticipated Benefits  An organisational commitment to continuous improvement  Increased professionalism  The framework can be applied to all areas of the organisation  Outcomes and data can be provided to potential and current funders, partners and stakeholders  Better informed planning and program development  An open dialogue with clients about service improvement Better outcomes for young people!

Questions? Frances Head George Elliott