Evaluation: what next? Dr Fiona Fylan Director of Brainbox Research & Leader in Psychology, Leeds Metropolitan University Beth Fylan Social Researcher & Director of Brainbox Research
+ Evaluation and beyond Fiona Fylan and Beth Fylan
+ What we’re going to do
+ Evaluation refresher
+ Brainbox queries We need to do an evaluation Our funders want to know what we’ve achieved How can we tell what effect we’re having? My director wants to know whether we’ve made a difference My staff budget is under threat. Help me defend what my team does We have to bid for our budget next year and we need to prove what we do is working How can we do our own evaluation? I need to write a questionnaire
+ Why evaluate? intervention
+ Evaluate to improve
+ Evaluation types
+ Audit / monitoring A form of process evaluation
+ What evaluation questions should you ask?
+ Defining your objectives
+ What are the evaluation research questions?
+ What different designs are there?
+ Evaluation pitfalls First things first: no clear intervention objectives. No outcome measures Process versus outcome – happy sheets Asking the wrong questions – Did it meet your expectations? How to interpret the results. Does the drop in perceived skill levels result from increased knowledge of what is required?
+ Your evaluation questions
+ Some in reserve How many people should I include in my evaluation? Who should I include? What questions should I ask? How should I analyse my data? What about E-valu-it?
+ What next? Outputs and stakeholders
+ Report types WrittenPost-fieldwork summaryFull written report SpokenInformal feedback / verbal debrief (qualitative) Full presentation of the research ImmediateConsidered
+ Basic logic models
+ Recommendations
+ Revisit your intervention
+ Create some headlines
+ Mapping and engaging your stakeholders
+ INFORM (low) CONSULT (mid) INVOLVE (mid) PARTNER (high) Joint working group, problem solving team. Seminars, workshops, project meetings. Consultation documents, feedback forms. Presentations, press releases, newsletters, twitter.
Power High LowHigh Interest
StakeholderCategoryWhy are they interested/ What do we want? LevelStatusActions 1. Fleet managers ReferrerAccess to drivers PartnerManage closely Short report and action group
+ Any questions? Any elephants?