1. Introduction to Evaluation Tammy Horne, Ph.D. WellQuest Consulting Ltd. (780) 451-6145 thorne@wellquestconsulting.com www.wellquestconsulting.com

2. What is Evaluation? “Program evaluation is the systematic collection of information about the activities, characteristics, and outcomes of programs to make judgments about the program, improve program effectiveness, and/or inform decisions about future program development… 10/14/20152

3. What is Evaluation? …Program evaluation does not occur in a vacuum and is influenced by real-world constraints. Evaluation should be practical and feasible and must be conducted within the confines of resources, time, and political context… 10/14/20153

4. What is Evaluation? …Moreover, evaluation should serve a useful purpose, be conducted in an ethical manner, and produce accurate findings. Evaluation findings should be used to make decisions about program implementation and to improve program effectiveness.” (Centers for Disease Control, 2001, p. 6) www.cdc.gov/tobacco/evaluation_manual/Evaluation.pdf 10/14/20154

5. What Is Evaluation? Formative evaluation – improve existing program, ‘stabilize’ it for summative evaluation – focus is usually processes and sometimes early outcomes Summative evaluation – judgment of merit or worth of program; make decisions about to keep, change, expand, cut – often emphasizes longer-term outcomes 10/14/20155

6. Why Do We Evaluate? To learn what is happening in our programs (Processes - what we do and how people respond) To learn what is changing, what differences we are making (Outcomes - desired or expected results; the changes we hope to see.) To learn how processes and outcomes are related (Theory of change/program theory) 10/14/20156

7. Why Do We Evaluate? To learn about how our program is influenced by other factors in our environments (Context) The above kinds of learning help us to:  Be accountable to funders  Continue to improve our programs  Share our knowledge with others doing similar work 10/14/20157

8. Some Example Process Questions What activities do we offer? How often? For whom? What types of people do we reach, and how? Who expresses interest in our program? 10/14/20158

9. Some Example Process Questions How do we respect diversity – e.g., be inclusive of/responsive to people from particular cultural backgrounds? How do people participate? (e.g., recipients of a service; partners in program planning and/or delivery, community builders or leaders, advocates for healthy policies/practices) 10/14/20159

10. Some Example Process Questions What makes it easy for people to participate? What makes it hard for people to participate? (barriers – if any) Who stays? Who drops out and why? What is running smoothly? What ‘kinks’ still need to be worked out? 10/14/201510

11. Some Example Process Questions What do participants like best about the program? What changes would they like to see? What would they tell others about your program? 10/14/201511

12. Some Example Outcome Questions: Individual Level What new knowledge do participants learn? What new skills do they develop or strengthen? What changes do they make in their behaviour? (e.g., changes in personal lifestyle, relationships, community engagement) 10/14/201512

13. Some Example Outcome Questions: Organizational Level Do organizations you are working with get involved in new collaborations or partnerships,? Do organizations develop more culturally competent practices that are responsive to the cultural groups with whom they work? 10/14/201513

14. Some Example Outcome Questions: Community Level How has community capacity been enhanced, such as:  People involved in community leadership?  New community structures/resources? (e.g., community gardens, walking trails)  People participating, with others, in community-building activities?  People sharing knowledge or skills? 10/14/201514

15. Some Example Outcome Questions: Community Level How is the quality of community life improving? (e.g., positive relationships between youth & seniors, neighbours getting to know each other, better access to local food, people feel safe on streets or in parks) 10/14/201515

16. Some Example Outcome Questions: Policy/System Level What policy changes are happening in your school district in areas like: availability of healthy food, opportunities for physical activity, prevention of bullying and harassment? How well are public non-smoking policies being enforced? 10/14/201516

17. Some Example Outcome Questions: Policy/System Level How receptive are municipal councils to community members’ requests for recreation /parks opportunities, land for community gardens? 10/14/201517