1. ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation NIH Science Education Projects Annual Conference May 15, 2012

2. Why Outcomes Matter Constructive Failure Constructive Failure Unconstructive Failure Unconstructive Failure http://www.philanthropyroundtable.org/topic/excellence_in_philanthropy/f ailure_in_philanthropy Results-based Evaluation

3. Three buckets of data…. Evaluations Process PeopleResults

4. The Evaluation Process Should Answer: How has my program made a difference? How are the lives of the program participants improved as a result of my program? Results-based Evaluation

6. What are Outcomes? A target audiences changed or improved skills, attitudes, knowledge, behaviors, status, or life condition brought about by experiencing a program. Results-based Evaluation

7. What is a Program? Activities and services leading towards intended outcomes Has a definite beginning and end Designed to change attitudes, behaviors, or knowledge, or increase skills and abilities related to assumed need Results-based Evaluation

8. Program Purpose Program purpose is driven by assumptions about need. It relates to the organizations mission statement. It defines audience, services, and outcomes. In summary... We do what, for whom, for what outcome or benefit? Results-based Evaluation

9. What: We provide assessment, goal development and one-to one tutoring To whom: Adults living in Reynolds County and surrounding area who express an interest and demonstrate a need for our services For what outcomes: Adults will improve their reading, writing and communication skills and achieve their personal literacy goals Program Example: Literacy Results-based Evaluation

10. What: We will establish a community-wide partnership with health professionals, teachers, parents, children, health researchers and the media to provide raise awareness about obesity among children, promote healthy diets and lifestyles, create family health plans and support parents efforts to help their children reach their diet and fitness goals. To whom: Parents, children Pre-K and K, and community members in Springfield Results-based Evaluation

11. For what outcomes: 1.Children understand the dangers of bad eating habits and a sedentary lifestyle 2.Children improve their diet 3.Children adopt an active lifestyle 4.Families support one another in maintaining a healthy lifestyle 5.Families create, use and adopt a regular healthy meal and exercise plan 6.The community demonstrate a greater understanding of the cost of poor dietary and exercise habits of its citizens 7.The community make a commitment to sustaining programs to help families become and stay healthy and active Results-based Evaluation

12. Logic Models The Evaluation Plan: An outcomes logic model (evaluation plan) is a clear, graphic representation of the links between program activities, the results these activities produce, and how the results will be measured.

13. InputsActivitiesOutputsOutcomesImpact Resources; people, materials, supplies, money Program Process Products Counts and Amounts Knowledge, Skills, Behaviors, Attitude changes of targeted population Community- level change EXAMPLE 1 Logic Models

14. ImpactOutcomeIndicatorsInputsActivities Community Results Intended individual change Observable and measurable behaviors or conditions Resources; people, materials, supplies, money Program Process EXAMPLE 2 Logic Models

15. OutcomeIndicatorsApplied toData SourceData IntervalTarget Intended individual change Observable and measurable behaviors or conditions The population(s) being measured Information sources When information is collected The #/% who will achieve the outcome The Practical Model Logic Models InputsServicesOutputs Resources; people, materials, supplies, money Program Process Products Counts and Amounts

16. Start the Plan with Outcomes First A target audiences changed or improved skills, attitudes, knowledge, behaviors, status, or life condition brought about by experiencing a program. Outcomes IndicatorsData SourceApplied toData IntervalTarget

17. Outcomes are Logical and Sequential Immediate Intermediate Long-term IMPACT Outcomes IndicatorsData SourceApplied toData IntervalTarget

18. Outcomes IndicatorsData SourceApplied toData IntervalTarget 1.Children understand the dangers of bad eating habits and a sedentary lifestyle 2.Children improve their diet 3.Children adopt an active lifestyle 4.Families support one another in maintaining a healthy lifestyle 5.Families create, use and adopt a regular healthy meal and exercise plan 6.The community demonstrate a greater understanding of the cost of poor dietary and exercise habits of its citizens 7.The community makes a commitment to sustaining programs to help families become and stay healthy and active

19. Indicators Measure the Outcomes State the measurable conditions or behaviors chosen to show an outcome was achieved. Show what you hope to see or know Are observable evidence of accomplishments, changes, or gains. Indicators OutcomesIndicatorsData SourceApplied toData IntervalTarget

20. Outcome # 1Indicator: Children understand the dangers of bad eating habits and a sedentary lifestyle The #/% of children who can identify at least 2 poor eating habits that harm ones health and The #/% of children who can identify at least 3 consequences of an inactive lifestyle Indicators Outcome # 2Indicator: Children improve their diet The #/% of children who eat 80% of the six food groups at least three times a week Outcome # 3Indicator: Children adopt an active lifestyle The #/% of children who exercise or engage in active play for 30 minutes at least 6 times a week

21. Outcome # 4Indicator: Families support one another in maintaining a healthy lifestyle The #/% of families who have more than one family member supporting the child in the Fit from the Start program Indicators Outcome # 5Indicator: Families create, use and adopt a regular healthy meal and exercise plan The #/% of families who take an active role in meal planning and choosing foods which are healthy The #/% of families who plan and have outings and activities at least 4 times a week month

22. Outcome # 6Indicator: The community demonstrates an understanding of the cost of poor dietary and exercise habits of its citizens The #/% of community members who accurately complete the online survey on fitness and diet Indicators Outcome # 7Indicator: The community makes a commitment to sustaining programs to help families become and stay healthy and active The #/% of project partners who complete their obligations to this project and The #/% of project partner who continue their efforts beyond the grant period

23. Data Sources Data captured at critical points in time Look at indicators to see what data sources are required. Follow up – Continued Benefit End – Results - Outcomes Middle - Progress Beginning - Baseline OutcomesIndicatorsData SourceApplied toData IntervalTarget

24. Data Sources Used to assess: Behaviors Life Condition-Life Status Attitudes Skills Knowledge OutcomesIndicatorsData SourceApplied toData IntervalTarget

25. Where Will the Information Come From? Data sources are tools, documents, and locations for information that will show what happened to your target audience. Pre-post test scores Program records Assessments Records from other organizations Observations Portfolio review Data Sources OutcomesIndicatorsData SourceApplied toData IntervalTarget

26. What kind of data do you need? Often data sources are categorized into types: Primary – Information you have collected yourself Secondary – Existing information collected from someone else Qualitative – Types of information that is describing quality rather than quantity – answers questions concerning how Quantitative – Information concerning counts, amounts, percentages; answers questions concerning how much Data Sources OutcomesIndicatorsData SourceApplied toData IntervalTarget

27. Applied to: Target Audience Who Gets Measured? The target audience to whom the indicator applies Decide if you will measure all participants, completers of the program, or another subgroup Special characteristics of the target audience can further clarify the group to be measured OutcomesIndicatorsData SourceApplied toData IntervalTarget

28. Data Intervals When to Measure? Data intervals are the points in time when data are collected: Data can also be collected at the end of an activity or phase and at follow-up Data are usually collected at program start and end for comparison when increases in skill, behavior, or knowledge are expected OutcomesIndicatorsData SourceApplied toData IntervalTarget

29. Targets Targets of Performance Targets, also known as goals, are the stated expectations for the performance of outcomes. Stated in terms of a number and/or percent Meets influencers expectations May be estimated by the programs past performance OutcomesIndicatorsData SourceApplied toData IntervalTarget

30. Logic Models Inputs: The resources you will dedicate to your program. Staff Facilities Equipment Consultants Materials Curricula

31. Logic Models Inputs: Staff – researchers, health professionals, sports and dance teachers, public relations and web development team Hands-on activities and curricula Healthy Lifestyle curricula Informal science education PSAs Print media Website

32. Logic Models Services: Services are activities, tasks and events that engage the target audience. Services are designed to have a direct effect on program outcomes. Workshops Programs Mentoring Classes Training Job Try-outs

33. Logic Models Services: Case assessment Diet and exercise plan and monitoring Age appropriate games and activities Field trips Hands-on experiences Dance and sports orientation Website development Campaign to promote healthy diets and lifestyles

34. Logic Models Outputs: A direct program product, typically measured in numbers: # Participants# Education sessions # Plans developed# Online surveys taken # Plans completed# Activities completed # Field trips# Non-completers

35. Characteristics of People Served Age Race Health Status Income Urban vs. Rural Primary Language Transportation Education Level Logic Models

36. Reporting In summary: We wanted to do what? We did what? So what?

37. Three buckets of data…. Evaluations Process PeopleResults

38. Success…. Claudia B. Horn Performance Results, Inc. 20203 Goshen Road Suite 314 Gaithersburg, Maryland 20879 www.performance-results.net 240-207-3071 Claudia.horn@performance-results.net www.resultsonline2.net