Nancy L. Weaver, PhD, MPH Department of Community Health School of Public Health Saint Louis University 16 July 2010 LOGIC MODEL FUNDAMENTALS

What is a Logic Model? Diagram depicting relationships between Program goal - long term public health outcome Program outcomes - short, intermediate and long term impact Program inputs, activities and outputs Also called Analytic framework Causal frameworks Program plan

Cooking class Completed Activities Examples: # of people participating in each event # of newsletters distributed # of classes conducted Increased knowledge, skills and abilities Increase in physical activity and healthy eating (behavior) Decrease in Obesity Reduce type-2 diabetes Short-Term Intermediate Long-Term Goal InputsActivitiesOutputsOutcomes Newsletters Exercise class Activities Seminars Walking groups Financial Resources Human Resources Informational Resources

Step 1: State program goal(s) Describe the public health problem Provide justification, e.g.:  Who is most affected?  What are the geographic differences?  Are there trends over time?  What are the consequences of the problem?  What causes it?  What are the costs of treating it?

Step 2: State long-term outcomes Health outcomes (e.g. reduced obesity rates, fewer complications from diabetes) To reduce the percent of individuals, ages 35 to 55 in County A, who are overweight (BMI>25) from 80% in 2010 to 65% in 2015

Step 3: State intermediate outcomes Behaviors (e.g. physical activity, talking to your doctor, getting screened) By 2012, women over 50 in St. Louis City will have a 50% increase in use of mammography screening services.

Step 4: State short-term outcomes Knowledge, attitudes, beliefs, skills (e.g. understanding nutrition labels, the importance of routine check-ups, confidence in using condoms) Increase the percentage of high school students reporting they understand the ABCs of melanoma from a baseline of 30% to a follow-up of 60%.

Notice the Clarity SMART Specific Measurable Achievable Relevant Time-bound Who will benefit from the program? What benefit should they receive? How much of that benefit should be achieved? By when should it be achieved?

Step 5: Develop an action plan What activities will you complete in order to achieve the short-term goals? What are the outputs of these activities? What resources are required? Evidence and Theory guide these decisions

Project activities, e.g. If you want to… Change knowledge: Change attitudes: Increase support: Change policies Increase safety behaviors: Increase confidence: Decrease risk behaviors: Then you might… Distribute materials Provide counseling Foster support groups Partner for org. change Provide prevention tools Use role modeling Shift peer influence

Using the logic model for program evaluation

What is evaluation? Root word “value” Dictionary To judge the worth of something To examine carefully Program evaluation Use of various procedures to determine if a program has been developed and implemented as planned and the degree to which program goals and objectives are met.

PROGRAMEFFECT ?

Answers the questions: What have you done? How well did you do it? How much have you done? How effective have you been? In changing attitudes, beliefs, behaviors? In changing health outcomes?

Evaluation allows you to: Determine if goals/objectives were met Assess strengths and weaknesses of program elements Determine the relationship between the intervention and outcomes Change the program to make it better

CDC framework for program evaluation

Three types of evaluation Process Summative: Impact Summative: Outcome

Process Evaluation Describes activities and outputs Shorter-term feedback on program implementation, content, methods, participant response, practitioner response What is working, what is not working Uses quantitative or qualitative data Data usually involves counts, not rates or ratios

Was the program implemented as planned? Did the program reach its intended target group? What services did the people in the program receive? Were people satisfied with the program’s services? What is the average cost for the program per person? How many people were trained? How many videos were produced? Answers the questions:

Summative Evaluation Assesses program effectiveness Impact – assess the short to mid range impact of the intervention on cognitive and behavioral indicators Outcome – assesses the longer term impact of the intervention on behavioral, health, and quality of life indicators

Decrease in Obesity Cooking class Completed Activities Examples: # of people participating in each event # of newsletters distributed # of classes conducted Increased knowledge, skills and abilities Increase in physical activity and healthy eating (behavior) Reduce type-2 diabetes Short-Term Intermediate Long-Term Goal InputsActivitiesOutputsOutcomes Newsletters Exercise class Activities Seminars Walking groups Financial Resources Human Resources Informational Resources

Decrease in Obesity Cooking class Completed Activities Examples: # of people participating in each event # of newsletters distributed # of classes conducted Increased knowledge, skills and abilities Increase in physical activity and healthy eating (behavior) Reduce type-2 diabetes Short-Term Intermediate Long-Term Goal InputsActivitiesOutputsOutcomes Newsletters Exercise class Activities Seminars Walking groups Financial Resources Human Resources Informational Resources Process Outcome Impact