1. Measuring the Impact of Your Volunteer Program Barbra J. Portzline, Ph.D. Liz Benton, MBA

2. Introductions Name Organization Role/Position Experience (if any) with evaluation © 2013 Barbra Portzline, used with permission

3. Evaluation… What comes to mind when you hear the word evaluation… © 2013 Barbra Portzline, used with permission

4. Definition of Evaluation Evaluation is an intentional, purposeful, systematic and useful inquiry designed to collect information about a program, process, system, structure, organization, individual, and/or group in order to inform decisions or actions © 2013 Barbra Portzline, used with permission

5. Benefits of Evaluation Evaluation ensures quality programs Helps prioritize resources Contributes to organization members learning Evaluation encourages accountability Helps build a case to convince others of the effectiveness of certain programs and initiatives Shows volunteers that their time is worthwhile © 2013 Barbra Portzline, used with permission

6. Some Things to Evaluate Extent to which program met its goals Impact of program on clients Awareness of program Impact of volunteers on the program Impact of volunteers on the clients Costs and benefits of a program Implementation of program Marketing efforts Extent that staffing is sufficient Recruitment of volunteers Satisfaction Transfer of skills © 2013 Barbra Portzline, used with permission

7. Evaluation Planning Process Rationale Purpose Stakeholders Key Questions Methods Budget Timeline Communication Strategy © 2013 Barbra Portzline, used with permission

8. Focusing the Evaluation Helps to ensure that the evaluation concentrates on those issues that will provide the most useful information, yield the greatest insights, and/or provide the most solid basis for action It is never possible to look at everything; focusing involves a choice “I would really like to know ___ about the work we do” © 2013 Barbra Portzline, used with permission

9. Reasons to Link Evaluation Efforts to Programs Data Driven Decisions Efficient use of resources Systematic and Purposeful Credible results © 2013 Barbra Portzline, used with permission

10. Key Questions The Key Questions should be broad, overarching, questions that guide the evaluation In what ways are we improving families’ lives? To what extent do our volunteers feel valued? How can our relationships with key stakeholders be improved? © 2013 Barbra Portzline, used with permission

11. Paired Interviews In pairs, interview your partner using the questions provided Take notes as needed Share the key elements of a “exceptional volunteer program” with the larger group © 2013 Barbra Portzline, used with permission

12. Developing Key Questions Based on the strengths discussed, develop 4-6 key questions, that if answered will help you reach your vision Examples: To what extent are our volunteers satisfied with the level of communication provided? How can communication be improved? To what extent is our program meeting volunteers needs? How can it be improved? What impact are volunteers having on our clients well-being? © 2013 Barbra Portzline, used with permission

13. Data Collection Methods Interviews (individual and focus group) Observation Surveys and questionnaires Document analysis Testimonials © 2013 Barbra Portzline, used with permission

14. Methods Depend On… Key Questions Money Time Accessibility of respondents Requirements © 2013 Barbra Portzline, used with permission

15. Data Collection Matrix Matrix is used to present the key questions, data collection methods, and data sources Across the top the key questions are written Methods are listed in the column to the left Inside each cell are the data sources © 2013 Barbra Portzline, used with permission

16. 16 © 2013 Barbra Portzline, used with permission

17. Evaluation Plan The Matrix is the “down and dirty” plan Often you need a rationale or justification for evaluation work. © 2013 Barbra Portzline, used with permission

18. Reasons to Communicate and Report To convey information about the program To describe how a program is working and to what effect To demonstrate results and accountability To aid decision-making about continued funding, prospective funding, and replication at other sites/organizations To learn, grow, and improve the program To inform decision making by program staff and management about changes that will improve the program To build support for evaluation © 2013 Barbra Portzline, used with permission

19. Reporting to Stakeholders Primary Stakeholders –Usually request the evaluation –Typically are major decision makers –Can consist of program staff, supervisors, senior managers, and funders © 2013 Barbra Portzline, used with permission

20. Reporting to Stakeholders Secondary Stakeholders –Usually are involved, but with little or no daily contact –Can consist of program participants, their supervisors or managers, and other affected by the evaluation © 2013 Barbra Portzline, used with permission

21. Reporting to Stakeholders Tertiary Stakeholders –Usually are more distant but are possibly interested in findings –Can consist of future program participants, the general public, and members of the same profession © 2013 Barbra Portzline, used with permission

22. Reporting During the Evaluation To include stakeholders in decision making about evaluation design/activities To inform different stakeholders about upcoming evaluation activities To keep stakeholders informed about the progress of the evaluation To convey information about the program and its evaluation To demonstrate results and accountability To learn, grow, and improve the program © 2013 Barbra Portzline, used with permission

23. Some Formats Comprehensive written reports Working sessions Executive summaries Personal discussions Newsletters, bulletins, briefs, brochures News media communications Video presentations Memos and postcards Verbal presentation Posters Email © 2013 Barbra Portzline, used with permission

24. Keep in Mind Think about what purposes your communicating and reporting can (and should) serve Review and identify the evaluation stakeholders and think about the purposes for which they should be considered a communicating and reporting audience Choose the best format – based on purposes (that is, content), audiences, and timing © 2013 Barbra Portzline, used with permission

25. Thank you! Liz Benton 505.254.3700 ext 124 Barbra Portzline 505.254.3700 ext 110 © 2013 Portzline Enterprises 25