How good is our third sector organisation? Elizabeth Morrison March 2014

The third sector makes a direct impact on the growth of Scotland's economy, the wellbeing of its citizens and the improvement of its public services.

This resource is designed to help third sector organisations self-evaluate and plan for improvement more effectively, there is no expectation that this needs to be used for this or any other purpose. It is important that organisations use a toolkit of different approaches matched most appropriately to specific circumstances and differing contexts. Many third sector organisations already have very good systems and processes in place for this which they should continue to use.

The culture for self-evaluation Looking inwards Looking outwards Looking forwards

Looking inwards Basic process of self-evaluation  How are we doing?  How do we know?  What are we going to do now?

A ‘standard’ is a Benchmark You can measure whether or not you have reached a standard - achieved/not achieved

STANDARDS yes/no achieved or not achieved seen or not seen

‘quality’, on the other hand, is associated with the idea of value worth degree of extent of merit significance

When we evaluate quality we ask questions like How good? To what extent? How well?

NOT ‘IS IT THERE?’ or HOW MANY?

Evaluating quality Quality is evaluated through the use of ‘quality indicators’ or ‘QIs’. Why do we use quality indicators? Evaluations against QIs are professional judgments. By using QIs, we ensure consistency across different contexts

Using indicators to evaluate quality  The use of QIs requires the evaluator to balance the strengths and areas for improvement and come to an overall view.  The evaluation is based on the relative importance of each of the strengths and areas for improvement, and their impact  not on the numbers of strengths/weaknesses.  It is generally easier to measure achievement of a standard than to evaluate quality.  Evaluation of quality is therefore a professional skill.

Quality indicators balance strengths and areas for improvement importance NOT numbers

Public assurance requires us to measure whether organisations reach minimum standards. We need to evaluate quality if we want to help organisations to improve.

Structure  Based on outcomes, processes and leadership  Structure in line with other public service self-evaluation frameworks  QIs grouped under 6 important questions

Six key questions  What key outcomes have we achieved?  How well do we meet the needs of our stakeholders  How good is our delivery of key processes?  How good is our operational management?  How good is our strategic leadership?  What is our capacity for improvement?

Our self-evaluation frameworks

outcomes processes inputs Evaluating quality

Links between quality indicators within a quality framework QIs are related to each other. The relationships are causal. When you make evaluations in relation to one QI you also need to check the evaluations given for other QIs to make sure they make sense.

How do you evaluate the overall quality of a third sector organisation?

How do you evaluate the overall quality of anything?

How do you evaluate the overall quality of a…….. football club?

Is it a successful football club? 1. The team’s performance How well is the club doing? Number of games won – getting better or worse? How many goals scored? Performance compared with that of other clubs? Outcomes and achievements

2. Impact on people What do supporters think of the club? How well is the club doing? Does it live up to people’s expectations? What do the fans think? How many people come to the games? Do people want to become regular supporters? Do people look sad or happy during the game? Outcomes and achievements

How well is the club doing? How well does it live up to people’s expectations? 3. Impact on team members Are the players highly motivated? Are there a lot of transfer requests Does it have a high absence rate ? Does it have a high turnover of players? What do players think of the quality of coaching? Is it a good club to play for? Outcomes and achievements

How well is the club doing? How well does it live up to people’s expectations? Does it have a good reputation? In the local area? In Scotland? In Europe? Worldwide? 4. Impact on the community Outcomes and achievements

How well is the club doing? How well does it meet people’s expectations? How well does the team play? 5.Teamwork 6.Skills 7.Fitness Does the team play well? Does the team play with flair? Does it have a good blend of strengths in defence, mid-field and attack? Outcomes and achievements Quality of play Is the team fit and prepared for the rigours of a full game?

How well is the club doing? How well does it live up to people’s expectations ? How well does the team play? How well is the club managed? 8. Observation and monitoring of matches Identifying strengths in the team Identifying areas for improvement Is the club well run? Providing feedback to players Quality of play Outcomes and achievementsManagement and leadership

How well is the club doing? How well does it live up to people’s expectations ? How well does the team play? How well is the club managed? 9. Management of staff Is there a good training programme? Does it make sure players play to their strengths? Is the club well run? Does it deal well in the transfer market? Quality of play Outcomes and achievementsManagement and leadership

How well is the club doing? How well does it live up to people’s expectations ? How well does the team play? How well is the club managed? 8. Observation and monitoring of matches Identifying strengths in the team Identifying areas for improvement Is the club well run? Providing feedback to players Quality of play Outcomes and achievementsManagement and leadership

How well is the club doing? How well does it live up to people’s expectations ? How well does the team play? How well is the club managed? 9. Management of staff Is there a good training programme? Does it make sure players play to their strengths? Is the club well run? Does it deal well in the transfer market? Quality of play Outcomes and achievementsManagement and leadership

How well is the club doing? How well does it live up to people’s expectations? How well does the team play? How well is the club managed? Is the club well run? 10. Management of Resources Does it manage its money well? Is it good at getting sponsorship? Does it manage its grounds and facilities well? Management and leadership Quality of play Outcomes and achievements

How well is the club doing? How well does it live up to people’s expectations? How well does the team play? How well is the club managed? How well is the club led? 11. Leadership of the manager 12. Leadership of the Chief Executive & Board of Directors Does the club have a clear vision of what it is aiming for? Do the chief executive, manager and captain provide strong direction? Is the club well led? Are they prepared to make hard decisions to improve performance? Do they have the support of fans and players? Does the club know its strengths and weaknesses and is it making improvements? Does the club have a plan for improving results? Outcomes and achievements Quality of play Management and leadership

How do you evaluate the overall quality of a football club?

How well is the club managed? How well does the team play? How well does it live up to people’s expectations? 1. The team’s performance 2. Impact on people 3. Impact on team members 4. Impact on the community 5. Teamwork 6.Skills 7. Fitness 8. Observation and monitoring of matches 9. Management and support of team 10. Management of resources 11. Leadership of the manager 12. Leadership of the Chief Executive & Board of Directors How well is the club doing? How well is the club led? Outcomes and achievements Quality of play Leadership and management Cause and effect Framework for evaluating the quality of a football club

How good is our delivery of key processes? What key outcomes have we achieved? 1. Key performance outcomes 1.1 Improvement in performance 1.2 Adherence to statutory principles and fulfilment of statutory duties 5. Delivery of key processes 5.1 Delivering services 5.2 Developing, managing and improving relationships with service users and other stakeholders 5.3 Inclusion, equality and fairness 9. Strategic leadership 9.1 Vision, values and aims 9.2 Leadership and direction 9.3 Leading people and developing partnerships 9.4 Leadership of innovation, change and improvement 9.5 Securing improvement of quality and impact of services 2. Impact on service users 2.1 Impact on service users 3. Impact on staff 3.1 Impact on staff 4. Impact on the community 4.1 Impact on the local community 4.2 Impact on the wider community 6. Operational management 6.1 Policy review and development 6.2 Participation of service users and other stakeholders 6.3 Planning of key processes 7. Staffing 7.1 Management and deployment of staff 7.2 Career-long professional learning 8. Partnerships and resources 8.1 Partnership working 8.2 Financial management 8.3 Resource management 8.4 Knowledge and information management How good is our strategic leadership? How well do we meet the needs of our stakeholders? How good is our operational management? What is our capacity for improvement? 10. Capacity for improvement Global judgement based on evidence of all key areas, in particular, outcomes, impact and leadership