Controlling Your Evaluation Destiny Marla Steinberg Director of Evaluation MSFHR & The CAPTURE Project

Can you spot the PEAR cake?

What does evaluation mean to you? “ Evaluation is the systematic acquisition and assessment of information to provide useful feedback about some object” Evaluation can be defined as the systematic collection and analysis of information on the performance of a policy, program, or initiative to make judgments about relevance, progress or success and cost-effectiveness, and/or to inform future programming decisions about design and implementation. Evaluation means asking good, critical questions about programs to improve them and help them be accountable for the wise use of resources. Don’t be put off by the language of evaluation

Planning Tips Program ParticipantPractitioner or Manager Funder Set up evaluation advisory committee Jointly determine purpose of evaluation and generate evaluation questions Jointly develop program logic model so multiple perspectives are represented Jointly determine data collection strategy Choose methods that meet required level of rigour and are acceptable to participants Jointly identify other stakeholders and develop engagement and communication strategy Jointly develop reporting and dissemination strategy Think about different products for different audiences

Data Collection Tips Chosen data collection methods that meet required level of rigour and are acceptable to participants Include qualitative and quantitative data, get people to tell their stories Use multiple methods If possible try to use engaging methods Use lay or participant data collectors

Analysis and Interpretation Tips Hold interpretation sessions Find creative ways to display data that can speak to different audiences

Reporting and Dissemination Tips Don’t just create the one long evaluation report Video Art Brief reports

Using Results Tips Quality improvement Management Decision-making Jointly develop action plan

Joint Action Planning Action Planning Worksheet Evaluation Recommendation: Action Step(s)Person(s) Responsible TimelinePlans for monitoring results

Final Tip: Plan Evaluation Up Front How many evaluators does it take to change a light bulb? – one to do a needs assessment – one to do a feasibility study – one to do a qualitative study to find out what bulb to change – one to empower the bulb to change – one to tender a contract for further study – one to write performance indicators for success – one to do a cost benefit analysis to determine the best to buy – one to do a meta-evaluation showing that all previous studies have left everyone in the dark SO how many evaluators does it take?