2. Evaluation: what next? Dr Fiona Fylan Director of Brainbox Research & Leader in Psychology, Leeds Metropolitan University Beth Fylan Social Researcher & Director of Brainbox Research

3. + Evaluation and beyond Fiona Fylan and Beth Fylan

4. + What we’re going to do

5. + Evaluation refresher

6. + Brainbox queries We need to do an evaluation Our funders want to know what we’ve achieved How can we tell what effect we’re having? My director wants to know whether we’ve made a difference My staff budget is under threat. Help me defend what my team does We have to bid for our budget next year and we need to prove what we do is working How can we do our own evaluation? I need to write a questionnaire

7. + Why evaluate? intervention

8. + Evaluate to improve

9. + Evaluation types

10. + Audit / monitoring A form of process evaluation

11. + What evaluation questions should you ask?

12. + Defining your objectives

13. + What are the evaluation research questions?

14. + What different designs are there?

15. + Evaluation pitfalls  First things first: no clear intervention objectives.  No outcome measures  Process versus outcome – happy sheets  Asking the wrong questions – Did it meet your expectations?  How to interpret the results. Does the drop in perceived skill levels result from increased knowledge of what is required?

16. + Your evaluation questions

17. + Some in reserve How many people should I include in my evaluation? Who should I include? What questions should I ask? How should I analyse my data? What about E-valu-it?

18. + What next? Outputs and stakeholders

19. + Report types WrittenPost-fieldwork summaryFull written report SpokenInformal feedback / verbal debrief (qualitative) Full presentation of the research ImmediateConsidered

20. + Basic logic models

21. + Recommendations

22. + Revisit your intervention

23. + Create some headlines

24. + Mapping and engaging your stakeholders

25. + INFORM (low) CONSULT (mid) INVOLVE (mid) PARTNER (high) Joint working group, problem solving team. Seminars, workshops, project meetings. Consultation documents, feedback forms. Presentations, press releases, newsletters, twitter.

26. Power High LowHigh Interest

27. StakeholderCategoryWhy are they interested/ What do we want? LevelStatusActions 1. Fleet managers ReferrerAccess to drivers PartnerManage closely Short report and action group 2 3 4 5 6 7

28. + Any questions? Any elephants?