University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Unit 1: Getting ready

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation What comes to mind when you hear the word evaluation?

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation What does evaluation mean to you? On a piece of paper, write down your definition of evaluation.

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Definition of evaluation Definitions of evaluation may vary by approach and field of practice. We use: Program evaluation is… “the systematic collection of information about the activities, characteristics, and results of programs to make judgments about the program, improve or further develop program effectiveness, inform decisions about future program development, and/or increase understanding.” – Patton, 2008 Source: Patton, M. Q. (2008). Utilization-focused evaluation (4th ed.). Thousand Oaks, CA: Sage Publications, Inc.

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation In simplest terms, evaluation answers 3 questions: What? So what? Now what?

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation An outcome is… the change that occurs as a result of our activities and investments.

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation A new way of thinking… FROM: What are you doing? What activities do you deliver? Who participates? How many? ACTIVITIES TO: So what? What difference are you making? What results were achieved? Who benefited? How? OUTCOMES

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation A metaphor for outcomes: Not how many worms the bird feeds its young, but how well the fledgling flies –United Way, 1999 Source: United Way of America. (n.d.). Outcome measurement: What and Why?: An overview. Retrieved July 31, 2008, from

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Create your own “outcome metaphor” Create your own metaphor or drawing that will help you remember the meaning of outcome. Metaphor: a word or phrase denoting one kind of object or idea in place of another

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation What does the term program mean? Program refers to the object of evaluation: direct service interventions, community mobilization efforts, research initiatives, surveillance systems, policy development activities, outbreak investigations, laboratory diagnostics, communication campaigns, infrastructure-building projects, training and educational services, administrative systems. –CDC, 1999 Source: Centers for Disease Control and Prevention. (1999). Framework for program evaluation in public health [Electronic version]. Morbidity and Mortality Weekly Report, Vol. 48, No. RR-11, (September 17). Retrieved July 31, 2008, from

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Why am I evaluating outcomes? “Don’t accept your dog’s admiration as conclusive proof that you are wonderful.” –Ann Landers

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation What’s in this for you? Benefits of evaluating outcomes: Program/organizational improvement Improved accountability Better ability to describe what a program does – program description for stakeholders Data for fund raising Data for public relations Input for policy decision making Better planning Satisfaction in knowing how you are doing

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Assessing organizational readiness for evaluating outcomes Does your organization have a clearly defined and commonly understood/supported Mission – what your organization does Vision – what your organization should look like in the future Values/beliefs/principles – what you value/ believe – manifested in how you behave

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Assessing organizational readiness for evaluating outcomes Are your identified priorities reflected in your goals? Are the leaders committed to evaluation? Is there a general commitment to evaluation throughout the organization?

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Assessing your organization’s readiness… Does your organization have the resources needed to do outcome evaluation? Human −Leadership – committed and involved −Board and staff – understand the purpose −Time – of leadership, staff, and participants Fiscal −Often entails some financial commitment

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Assessing your organization’s readiness… Is there a plan and a timeline? Is there a common framework/language for evaluation? Do funders/stakeholders expect reporting of outcomes?

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Assessing your organization’s readiness… Is your organization going to be reporting to a funder? If so, the funder may have a specific format or expectation for reporting outcomes. Design outcome evaluations to meet the format or expectation. Remember, mission/vision/values are core to what you do – don’t compromise by “chasing” dollars that don’t “fit” your organization.

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Assessing your organization’s readiness… Are staff members interested in outcome evaluation? Do they have skills to engage in outcome evaluation? Does the organization support professional development for staff?

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Assessing your organization’s readiness… Is evaluation rewarded in your organization? Are evaluation processes, data, and findings valued? Are evaluation data and findings used? Do staff know/see how their evaluations are used?

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Assessing your organization’s readiness… Is evaluation data used by the organization to improve programs, guide resource allocations, and support planning? Is evaluation data used outside the organization to enhance its image, increase funding, and share promising practices?

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Barriers to evaluating outcomes Lack of leadership commitment Limited time and resources, including computer technologies Limited knowledge and skills in evaluation Attitude that programs cannot be measured Fear that evaluation data will hurt programs Lack of use or misuse of evaluation data

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Assets in evaluating outcomes Committed leadership Resources allocated for outcome measurement Evaluation expertise exists or can be secured Evaluations are used Incentives exist for doing quality evaluation