Goals, Outcomes and Program Evaluation Community Memorial Foundation March 5, 2014

Agenda Introductions Logic Models Goals, Outcomes, Baselines and Results Implementation and Outcomes Evaluation Q and A

Session Objectives To define terms used in CMF responsive grants application regarding program outcomes and evaluation To introduce tools and frameworks to facilitate the learning of program evaluation

Presenters Tom Fuechtmann Program Officer, Community Memorial Foundation Jon Korfmacher, PhD Associate Professor, Erikson Institute

Where Program Evaluation is helpful Understand, verify or increase the impact of services on clients Improve delivery mechanisms to be more efficient and less costly Verify that you're doing what you think you're doing Facilitate management really thinking about what their program is all about, including goals, how it meets goals and knowing if it has met its goals or not.

Where Program Evaluation is helpful Produce data or verify results that can be used for public relations and promoting services in the community. Produce valid comparisons between programs to decide which should be retained, e.g., in the face of pending budget cuts. Fully examine and describe effective programs for duplication elsewhere

Logic Models A way to summarize how your program works: a “picture” of your program. Note: There is no one “correct” way to develop a logic model * Logic models must be submitted for requests over $25,000

How is a logic model used? Logic Models have multiple functions: –Program planning –Program management –Evaluation –Communication Within your agency With people outside your agency

Logic Model Components Outcomes: Short-term Assumptions: What is necessary for this to happen? External Factors: What else may impact outcomes? Outcomes: Medium Outcomes: Long-term Outputs Activities (What we do) Participants (Who we serve) Inputs (What we invest) (Expect)(Want)(Hope)

Resources dedicated to or consumed by the project –Staff or volunteer resources –Funding –Equipment –Materials or other resources Inputs (What we invest) Logic Model Components

Outputs Activities (What we do) Participants (Who we serve) Inputs (What we invest) Sometimes described as activities accepted by participants Ex: 10-week parent training session for teen parents Ex: Teen parents attend at least 7/10 weeks of parent training

Logic Model Components Outcomes: Short-term Outcomes: Medium Outcomes: Long-term Outputs Activities (What we do) Participants (Who we serve) Inputs (What we invest) (Expect)(Want)(Hope) What changes do we expect to see right away? What changes would we want to see after that? What changes would we hope to see after that?

Ex: College HIV prevention Short Term College students have knowledge of how HIV/AIDS is spread Intermediate College students practice safe sex Long Term Lowered incidences of HIV cases on college campuses Closer in Time Easier to Measure More attributable to Project More distant in time Harder to measure Less attributable to project

Congratulations! You now have a logic model! (Now what?)

Remember… 1.Logic models are working documents Responsive programs need evolving models They can and should be open to change 2.The process is as important as the product 3.Your logic model is only the first step of an evaluation

CMF Application

Application continued

Goal: What you hope to achieve with this program (ex. To Reduce Homelessness) Outcomes: Changes that will occur as a result of your work (short term, intermediate, long term). Ex: 20% of those receiving services will be in permanent housing 6 months after graduating from program. Baseline: What is the current level of care? Ex. Only 2% are in permanent housing after a period of 6 months.

Evaluating Outcomes: What changes have occurred? Outcomes: the changes you expect to see as a result of your work Indicators: the specific way you measure an outcome. In other words, “How will I know it?”

Outcome Examples Short Term What do we Expect to Occur? Intermediate What do we Want to occur? Long Term What do we Hope will occur? Seniors increase knowledge of food contamination risks Seniors practice safe food handling Seniors experience lowered incidence of food borne illness Participants learn job- seeking skills Teen mothers report negative attitudes towards smoking while pregnant

Outcomes: Common types of Indicators New knowledge Increased skills Changed attitudes, opinions or values Changed motivation or aspirations Changed decisions Modified behavior Changed policies Changed conditions

Ex: HIV/AIDS Prevention Outcome Intermediate: Students practice safe sex Long Term: Number HIV/AIDS cases decrease on college campus Indicator Poor: Number of college students in program over time. Better: Percent students who talk about illness with friends Strong: Percent students reporting regular condom use

Evaluating Outcomes: Indicator Statements Elements of a Strong Statement: –How much? –Who? –What ? –When?

Evaluating Outcomes: Indicator Statements Example: “ 75% of program participants find part- time employment within six months of enrollment.” But… Have a rationale for the indicator (e.g., why 75%?)

Collecting Evaluation Information Questions to Ask Yourself –What resources do you have? –Quantitative or Qualitative? –Standardized or non-standardized? What makes the most sense for your program and the outcome in question?

CMF and Evaluation Outcome Objectives and Program Review –What is the impact and how will this move CMF’s mission forward? –What are the chances those results will be achieved? –Given the opportunities before us, is this the best use of the Foundation’s assets?

CMF and Evaluation (continued) Grantees as Partners Trust Open Dialogue A Learning Foundation

Questions and Answers

Additional Resources Innovation Network Free Management Library University of Wisconsin- Extension (Logic Model)