1. An evaluation framework… Strategic Planners Network 2 November 2018 Jacqui Henry

2. Overview What is evaluation? Why an evaluation framework?
Framework development
Introduce framework
Evaluation example
Summary 2

3. What is evaluation?
The systematic determination of the quality, value or importance of something.
Its purpose is to determine how well something is doing, its value, and if its worth doing more of.

4. Why an evaluation framework?
Support greater consistency and efficiency in the way evaluation is utilised
Strengths
Data rich
Strategic, analytical, technical skills
Compliance driven
Data collection for monitoring purposes

5. Development of an evaluation framework
Start with those interested willing to engage in evaluation process - Strategy team
Develop and test at a strategic level
Make time to engage in process so that evaluation is meaningful to those involved

7. Method Overall Purpose 1 2 3 Setting up evaluation for success
What is evaluation?
What’s the evaluation purpose?
Who will use the evaluation? 2
Develop key evaluation questions
What do we want to know about quality, value and importance?
What are the learnings going forward? 3
Agree on performance criteria and importance
What’s valued? What matters?
What are the needs we are trying to address?

8. Training programme evaluation
Method
Overall Purpose 1
Setting up evaluation for success
Continuous improvement
Whose involved - Social and Economic Science section and others 2
Develop key evaluation questions
How well has the programme been delivered?
How effective the programme is at realising key results?
What are the learnings going forward? 3
Agree on performance criteria and importance
Aspects of programme delivery and effectiveness of the programme

9. Method Overall Purpose 4 5 6 Develop a framework to assess performance
What does quality and performance look like in this context? Good and poor? 5
Agree on credible evidence
What data sources will be used?
Do we have multiple sources for each criteria? 6
Design and implement data collection
What methods of data collection are appropriate?

10. Training programme evaluation
Method
Overall Purpose 4
Develop a framework to assess performance
Rating scale from excellent to poor in terms of quality of programme delivery, and overall effectiveness. 5
Agree on credible evidence
Delivery – feedback forms, planning documents, attendance records of numbers
Effectiveness – survey/interviews post training 6
Design and implement data collection
As above plus planning

11. Method Overall Purpose 7 8 9
Analyse, compile, assemble and order the evidence
What does each of the data sources show?
How robust is the data? 8
Synthesis and sense making
Who should be part of the evaluative reasoning process?
What do we think about performance and why? So what? 9
Reporting, using and socialising results
Now what?
What’s the best way to frame the evaluation findings and conclusions?
How do we share the results in a useful and meaningful way?

12. Training programme evaluation
Method
Overall Purpose 7
Analyse, compile, assemble and order the evidence
What does each of the data sources show? What are the common themes? 8
Synthesis and sense making
So what? SES come together and go through sensemaking / reasoning process?
What do we think about performance on a scale of excellent to poor and why? 9
Reporting, using and socialising results
Now what? Action to be taken from results. Share the results in a useful and meaningful way with staff and others.

13. Guidance document

14. In summary Think about evaluation planning early on
Involve multiple sources of data
Framework is an inclusive process
Evaluation is about the quality, value and importance of something