E is for EVALUATION Practical steps to building evaluation into your projects By Zoe Hawke and Lorna Dyall

WHAT IS EVALUATION “Evaluation is about collecting and providing information on the ‘value’ of something” Helen Moewaka-Barnes (2009)

WHY DO WE NEED TO EVALUATE OUR PUBLIC HEALTH PROJECTS? Brainstorm Activity

“Health improvements are what we as public health professionals strive to achieve. To reach this goal we must continuously evaluate the effects of the public health work that we undertake” Centre for Disease Control

“ Evaluation not only justifies funding, it also determines effectiveness, compares standards of practice and assists in legitimising public health projects” Mason Durie (2005)

Evaluation Can: Assist with project objectives, & identify strategies to increase likelihood of success Help to ensure that a project actually does what it intends Answer the question “is this the best way to do this” Identify strengths and weaknesses of your project Help improve your project and its delivery Provide information to show key people/iwi/whanau/organisations how project is meeting its objective and whether it is effective Help you know if your project is reaching people you want it to. Provide information about how resources are being used Enable others to learn from your experiences (Adapted from Helen MoeWaka Barnes – The Evaluation Hikoi 2009)

HOW DO WE EVALUATE THE WORK WE UNDERTAKE IN PUBLIC HEALTH?

Step 1: DEVELOP EVALUATION QUESTIONS

HEADINGS TO CATEGORISE YOUR QUESTIONS UNDER: Participation: Number of people reached, iwi working alongside, population group, frequency of contact and intensity of participation/contact. Activities: Events, awareness raising methods (presentations), media work, promotional activities, Actions – recommendations adopted, practices implemented, actions taken, policies enacted Resources – staff and volunteer time; equipment, travel Learning – Knowledge gained by participants, opinions, views shared Impact - social economic, environmental conditions intended as end results, impacts or benefits to community of programs, Reactions – degree of interest, feelings toward the project, positive or negative interest in project, acceptance of workers in that community, community leaders responses and commitment to kaupapa. *

BUILD THE QUESTIONS INTO YOUR PROJECT PLAN LOGIC MODEL Inputs Outputs Outcomes -- Impact ActivitiesParticipationShort MediumMedium Long Questions for Evaluation: PARTICIPATION How many people actually participated, what iwi attended? ACTIVITIES: How many presentations were given, was there any media coverage? ACTIONS: Did anyone implement a gambling policy in their organization? RESOURCES How many staff members were needed on the day? Did we have enough staff? Did we have enough equipment? What equipment actually got used? LEARNING Was their any verbal comments from community people who attended the hui? What questions were asked by the audience? IMPACT: Are there going to be less pokies in the community? Did the council acknowledge that the pokies are in lower-social economic areas? Did a bar decide to close their doors for Gamble Free Day? REACTIONS: Did kaumatua make verbal submission for a Sinking Lid policy? What made them decide to talk to their council on the gambling issue? How many letters to the editor on the issue? What did the councils report state? What did the media say?

COLLECT FEEDBACK EVIDENCE Step 2:

STAKEHOLDER FEEDBACK EVIDENCE Engage your stakeholders in the evaluation process using feedback/evaluation forms/video feedback/evaluation to assist with the fine tuning of your projects and to supplement your evaluation questions for your reports : Stakeholders to include: Those in project operations/internal – other staff, coalition partners, sponsors, iwi, hapu who have supported and assisted in the organising of the project, volunteers, your agencies kaumatua. Those affected by the project/external: Hapu, Iwi, the community at large, whanau, community leaders, kaumatua, rangatahi etc... *NB remember to find a way to let stakeholders know the results of any survey evaluation forms etc..(this could be done visually, in writing or verbally. Don’t ask questions, get there answers and then disappear!

Step 3: ADDITIONAL EVIDENCE

ADDITIONAL EVIDENCE COMES IN MANY FORMS Photo Videos Letters of support Media articles Testimonials s Break down of evaluations results Copies of evaluation comments Include references to reports to backup you project goal

Step 4: ANALYSE AND INTERPRET YOUR EVALUATION FINDINGS

PROCESS THE EVALUATION DATA COLLECTED Collate all the findings/answers together from the first 3 steps: Step 1: Answers from you Evaluation Questions Step 2: Collect Feedback from Stakeholders Step 3: Additional Evidence Using all the collated evaluation material ask your team: What did we learn? What were the projects strengths? What were the projects weaknesses What would we do differently next time?

Step 5: USE YOUR EVALAUTION FINDINGS

USE YOUR EVALUATION RESULTS AS TEACHING TOOLS Share findings with other PG providers (using the new Te Kakano Website! Do a write up on evaluation results in your six monthly report, attach the evidence collected in appendices to show MOH and industry how successful public health activities are! Use results in decision making internally (future project focus, organisational policies), externally – gambling policy reviews, community events, development of other organisational policies Celebrate your success – Do a radio event on your project using evidence to highlight project success, write a letter to the editor, do a press release using the photos and quotes from feedback, present your results to the iwi, hapu community that were involved in the project.

EVALUATION ACTIVITY Put the steps into order and share with the group

Whāia te iti kahurangi Ki te tūohu koe, me he maunga teitei Pursue excellence for our people – should you stumble, let it be to a lofty mountain