1. LOGIC MODELS AND OUTCOME MEASUREMENT United Way Community Investment Process Training

2. Workshop Overview I.Welcome and Introductions II.Basics of Outcome Measurement III.Developing an Effective Logic Model IV.Evaluating your Logic Model V.Wrap-Up

3. What are your other expectations?

4. Logic Models What difference are you making? How do you know it? What is the value of your program? Theory of Change Program Matrix Logical Framework

5. 5 Logic Model Conceptual Chain Goal Theory Target Population Inputs Activities Outputs Short-term Outcomes Mid-term Outcomes Long-term Outcomes

9. What Exactly is An Outcome? Mid-Term Change in Behavior Short-Term Increased Knowledge Long-Term Change in Condition

10. 10 What is Outcome Measurement? Outcome measurement is the regular, systematic measuring of progress toward intended outcomes. Internal External

11. 11 Measuring Program Outcomes: Eight Steps to Success 8. Use Your Outcome Findings 7. Improve Your System 6. Analyze and Report Your findings 5. Try Out your Outcome Measurement System 4. Prepare to Collect Data on Your Indicators 3. Specify Indicators for Your Outcomes 2. Choose the Outcomes You Want to Measure 1. Get Ready

12. 12 What happens with data that we are measuring?

13. Logic Model Process Questions What outcome do you want to achieve? What activities will it take to get you there? What resources (inputs) will you need to complete the activities? How will you know when you have achieved your outcome? (How will you measure?)

14. 14 Logic Model Foundation GOAL Describe the Goal of your program: The goal should be client- focused - broad statement of intended change which identifies the target population.  Broad statement of desired condition  Client focused  Describe intended change of target population Example: Working poor individuals and families living in the Roanoke Valley will attain gainful employment and become self-sufficient.

15. 15 Logic Model Foundation TARGET POPULATION Describe the population of clients that will be served.  Age group of program targets  Description of population (i.e. gender, race, disability, socio economic status, etc)  Unique characteristics of the target population (i.e. needs, condition, barriers, etc)  Geographic location or community of focus of program targets  May include a profile of a ‘typical’ client

16. Inputs Human Financial Organizational Community If I have access to these resources, I can conduct…. Activities Processes Events Programs If we conduct these activities, we should see…. Outputs Amount of Service delivered Number of Product If the target population receives these services and products, they should demonstrate… Outcomes Indicates a measurable change to the target Increased behavior Increased knowledge Increased skills If these benefits are realized, then we have IMPACT!

17. 17 Logic Model Components INPUTS Inputs are resources needed for operation of the program, such as:  Staff positions & staff time  Volunteers & volunteers time  Facilities  Equipment and supplies  Funding and funding sources  Collaborative partners

18. 18 Logic Model Components ACTIVITIES Activities, processes or events undertaken with the inputs  Services a program provides for its participants to fulfill its purpose  No quantities (percentages, numbers of participants, numbers served, etc.) Examples: Feed and shelter homeless families Provide job training Educate the public about signs of child abuse Counsel pregnant women Create mentoring relationships for youth

19. 19 Logic Model Components OUTPUTS The direct results of program strategies, activities, processes, and/or events.  Usually described in terms of size and scope of the services or products delivered or produced by the program.  Outputs indicate whether or not a program was delivered to the intended audiences at the intended “dose.” Example: the number of classes taught, meetings held, materials distributed, counseling hours, program participation rates, or total service delivery hours.

20. 20 Logic Model Components OUTCOMES Outcomes are specific statements of the desired changes in the lives of a particular group (the target population).  All of the outcomes relate to the clients listed in the target population  All of the outcomes relate to the program’s components listed in the strategies  All of the outcomes state significant changes in or desirable benefits to be achieved by the target population  All of the outcomes are stated positively and are appropriate for the target population  All of the outcomes follow a logical progression (if….then)

21. 21 Logic Model Components INDICATORS Indicators track a program’s success on outcomes.  All of the indicators are clearly linked to and measure progress toward the associated outcome.  Each indicator is stated with a target number and target percentage of the total population that is expected to achieve the outcome.  All the indicators are based on the target population.  All the indicators are observable will measure some aspect of the outcomes they are attached to.

22. 22 Logic Model Components OUTCOMES vs. INDICATORS Outcome: Participating students succeed in school. Indicator: The # and % of participating students who pass all their core subjects. Outcome: Job training graduates become gainfully employed. Indicator: The # and % of graduates who secure full-time employment. Outcome: Parents read to their preschoolers more often Indicator: The # and % of parents who read to their preschoolers everyday for a minimum of 30 minutes per day.

23. TEST YOUR UNDERSTANDING! Build the logic model for your program

24. 24 What are Outcomes? Outcomes are specific statements of the desired changes in the lives of a particular group (the target population).  All of the outcomes relate to the clients  All of the outcomes flow logically from the strategies  All of the outcomes state significant changes in or desirable benefits to be achieved by the target population  All of the outcomes are stated positively and are appropriate for the target population  All of the outcomes follow a logical progression (if….then)

25. 25 Client-focused Outcomes vs. Process-focused Outcomes OUTCOMES: Should be client-focused.

26. 26 Outputs vs. Outcomes Program ExampleOutputs (amount of work) might be … Intended outcomes (results of the work) might be … Comprehensive child care Number of children registered Number of days of care Children exhibit age- appropriate social skills Children are ready for kindergarten Treatment for youth abusing drugs and alcohol Number of counseling sessions conducted Number of youth completing treatment Participating youth know the risks of substance abuse Participants remain clean and sober for 6 months Congregate meals for seniors Number of seniors enrolled Number of meals served Seniors eat at least one nutritious meal each day Seniors interact socially with their peers

27. 27 Writing Effective Outcome Statements Some tips: Specifically states what will be true of the clients/participants as the result of your program’s influence on their lives Outcomes focus on the participants/beneficiaries, not on the program Statement contains only one intended change or benefit Statement is very intentional (specific) in the desired change or benefit, and not simply that something is now improved, increased, decreased, enhanced, or expanded

28. 28 Developing Your Outcomes Questions to ask yourself What do we want to be true of participants because of their involvement with our program? … What do we want to be able to say about them? If we succeed with a participant (or don’t), what has changed (or hasn’t)? If we carry out these strategies, then what do participants believe, know, have, or do as a result? … And what benefit or change flows from that?

29. 29 What are Indicators? INDICATORS Indicators track a program’s success on outcomes.  All of the indicators are clearly linked to and measure progress toward the associated outcome.  Each indicator is stated with a target number and target percentage of the total population that is expected to achieve the outcome.  All the indicators are based on the target population.  All the indicators are observable will measure some aspect of the outcomes they are attached to.  Indicators are SMART: Specific, Measurable, Attainable, Relevant, and Time Bound

30. 30 Outcomes & Associated Indicators Outcome: Benefits for participants due to their involvement with a program. Indicator: The specific information collected to track a program’s success on outcomes. e.g., The number and percent of …

31. 31 Outcomes & Associated Indicators Outcome: Participating students succeed in school. Indicator 1: 70 of the participating students (78%) will pass all their core subjects. Indicator 2: 60 of the participating students (67%) will advance to the next grade level. Outcome: Job training graduates become gainfully employed. Indicator 1: 52 of the graduates (69%) will secure full-time employment. Indicator2: 40 of the employed graduates (77%) will remain employed for at least six months. Outcome: Parents read to their preschoolers more often Indicator: 280 parents (80%) will read to their preschoolers everyday for a minimum of 30 minutes per day during the Fall 2007 semester.

32. 32 Common Misconstructions when Writing Indicators  Vague Indicators e.g. Job Promotions 50 clients (64%) will receive job promotions within 3-months of completing the course.  Not Written Measurably e.g. Able to converse better 42 ESL graduates (91%) will demonstrate improved English language conversation skills by graduation.

33. 33 Common Misconstructions when Writing Indicators  Listing Measurement Documents e.g. Attendance Forms 33 students (67%) will have perfect attendance during the Fall 2007 semester.  Listing Measurement Plans e.g. Review of Test Scores 75 students (85%) will pass all of their core subject final exams.

34. 34 Common Misconstructions when Writing Indicators  Listing Outputs or Strategies (process-focused) e.g. Clients attend all 10 workshops 45 clients (74%) who attended all 10 work shops will demonstrate improved credit score ratings. e.g. Six trainings are offered during the year 123 training participants (90%) will show increased knowledge of the subject based on pre/post testing.

35. 35 Common Misconstructions when Writing Indicators  Listing Outcomes e.g. Clients change behaviors and make healthier choices 58 clients (89%) show improved nutritional intake at the 3-month re-assessment  Not Associated to Outcome e.g. Homework assignment completed (measuring self confidence) 12 students (72%) who complete the leadership training course will self-report improved self confidence.

36. 36 Selecting Outcomes You Want to Measure Pick the fewest number of outcomes that, as a group, will yield information for three key purposes … 1.Assure that you are achieving meaningful benefits for the intended beneficiaries – Which outcomes are the most important for program participants to achieve? 2.Communicate the value of your program to key audiences – Which outcomes do key audiences, including funders, care most about? Which best tell your story? 3.Guide program managers and staff in increasing the effectiveness of your program – Which outcomes will be the most helpful in identifying where the program is, or is not, being successful?

37. Evaluating Your Logic Model for Improvements

38. Ask the following general questions: Are the outcomes really outcomes? Do they describe changes or benefits for program clients/participants? Is the logic logical? Does it make sense that the strategies will lead to the initial outcomes, which will lead to the next outcomes, etc.? Is the long-term outcome meaningful for program participants? Have clients/participants experienced a change or benefit that makes a real difference in their lives? Is the long-term outcome reasonable? Is reasonable to think that the program actually can influence this outcome to a reasonable degree? Is the long-term outcome plausible? Will clients actually achieve this outcome? Evaluating your Logic Model