1. Step 2. Describe Program Logic Models & Outcome Measurement
Luann D’Ambrosio, MEd

2. Exercise
Improving on a hobby…
Expectations by you and your agency

3. CDC Evaluation Framework

5. Overview Measuring and articulating what we do and what we accomplish
Why?
What does this process look like?
What does it mean?
How might I do this?
What next?

6. Exercise
When did you have to articulate what you do at work and why? To whom?
How did it go?

7. Goals
Understand the basics of performance/outcome measurement for our services
Appreciate the importance of measurement systems for public health activity
Goal 1- Build capacity in the local public health workforce
for understanding the basics of performance measurement for public health services for children and families.
Goal 2- Build consensus and appreciation within the local public health workforce
on the importance of measurement systems for public health activity.
…to improve quality and improve health!
No EXPERTS expected out of this training
No test in the end
Possible “prizes” however

8. Objectives Identify components and uses of a logic model
Describe outcome measures and why they are important
Learning Objectives
Describe the components and uses of a logic model
Define outcome measures and why they are important
Differentiate between indicators and outcome measures
Describe some potential uses of outcome measures
Identify measurable outcomes
Determine levels (e.g. community, system, agency, program) of outcomes
Define outcomes vs. goals & objectives

9. Outcome measurement is a systematic way to assess the extent to which a program has achieved its intended results.
Outcome measurement is a systematic way to assess the extent to which a program has achieved its intended results.
We have often (Yesterday). . .been more often focused on what we did.
More and more (Today). . . We are focusing on what happens when we do it!
To get there…. Outcome Measurement starts w/ the simple but IMPORTANT question:
What is changed in the lives of our families, agency, community, system as a result of what we’ve been doing??
Our outcomes are thereby, less like:
How many visits did I make
How many people came to my clinic
And more like:
No. of cmty members that voted for a violence-free school zone
No of pg women getting AP care in the 1st trimester
Proportion of healthy foods snacks offered & purchased out of school vending machines
Yesterday: More often focused on what we did. Today: Focusing on what happens when we do it!

10. Ways to Think About Outcome Measurement
Performance measurement
Evaluation
Logic models
We can think about Outcome measurement in terms of:
Performance measurement
Evaluation
Logic models
Other
None of these are not mutually exclusive!
interrelated
The terms we will use are really moving targets
Can mean something different to different persons
They all advance the cause of measuring outcomes
Performance/Outcome measurement can be simply understood to be
regular collection & reporting of data to track
work produced
results achieved.
Outcome measurement is not an end itself, but rather a means to an end.
Outcome measurement is of little value
unless it leads to further decisions and actions
that seek to improve the performance of a program (or agency or individual)
What outcomes are
Important to achieve (are directly related to our core business)
Meaningful (make a real difference for participants)
Most useful (help in decision-making)
Most reasonable (non-trivial, logical)
Most realistic (achievable, given our resources)
These are not
mutually exclusive!

11. Why Do We Need to Do This? To provide accountability
To improve program quality
To make decisions about resource allocation
To help programs market themselves
To understand our contribution to our community’s health
Why do Outcome Measurement?
To provide accountability
To improve program quality
To support decision-making about resource allocation
To help programs market themselves
…to improve health!
Background:
Increasing emphasis on articulating what we do in PH and what our contributions are to the public’s health. R/t:
Funding pressures
The outcomes of our programs
(accountability, programs marketing)
The contribution of our resources/staff/PHNs to the health of communities
Supporting recruitment/retention
Staffing resources
(accountability, resource allocation)
Setting standards for the public to expect from us and request from their policy makers
(quality, accountability)
A recognition that we are not all prepared yet to do this
HRSA funding for training to the PH practice community
WA Family Child Health program directors are doing work around this already—creating logic models
To evaluate and improve programs
To communicate our successes
Telling our story of how we did it and what happened as a result
The “so what” of our efforts

12. “Everyday” Example: Marathon Runner
Goal: To do better than last year’s time
Resource: New running shoes
Activities: Running workouts
Outputs: Train 5x per week; run increasing #s of miles
Outcome: Training times/mile, overall time/distance, energy level
simple example
that we can relate to at one level or another (running or watching others run)
an example that allows us to readily identify some of the important
concepts we associate with the term performance management
In this example-- easy to determine the who,
equally apparent should be what this person is seeking to accomplish.
Even if not runners, it should be clear that
training at various distances
keeping track of time for these distances
are useful activities that
can assist a serious runner in determining whether some change in their race strategy might make sense in order to maximize their outcome and improve on last year’s time in this event.

13. How Do We Do This? Set a goal Define activities
Identify inputs or resources
Define outcomes
Develop outcome measures
Build consensus around these measures
Continue the runner example here….
Set a goal.
(Run a faster marathon.)
Define activities.
(Train 5x a week and increase # of miles.)
Identify inputs or resources.
(Purchase new shoes.)
Define outcomes.
(Develop physical conditioning and increase running skills.)
Develop outcome measures or indicators.
(Improve running times and endurance.)
Build consensus around these measures.
(Important—if I’m doing this w/ my friend.)
Resources
Activities
Outputs
Outcomes
Goals

14. Logic Models: “So That” Chains
Resources
Activities
Outputs
Outcomes
Goals
so that
so that
so that
so that
Logic Model
So why do we need those new running shoes….
So that….we can go running
So why do we go running
So that….increase our mileage, improve stamina
So why do we want to increase our miles
So that….we can improve our time from last year
Or decrease our blood pressure
In the past we often stopped asking the “so that” questions after our INPUTS
This kept us from answering the real questions related to what we intended to ACHIEVE
These elements are what make up a LOGIC MODEL

15. Logic Models: Public Health Example
RESOURCES
ACTIVITIES
OUTPUTS
OUTCOMES
GOALS
Staff
Immunization clinic
Shots
Less VPD
Healthier communities
OFS, page 29
Resources—
program inputs—the elements that constitute our program.
e.g. staff, funds, participants
What must this program have in order to function well?
Activities—
methods for providing our programs to participants
specific processes and/or events which you undertake to carry out the program
E.g. training, advocacy, case mgmt
What must we do to achieve our intended results?
Outputs—
Services or product units
E.g. the # of clients we served
How much do you do for how many people?
Outcomes—
The changes in people’s lives (or our community or our agency) that you believe will happen as a result of your efforts
So what? What difference does our program make?
Should be
Important to achieve
(directly related to our core business)
Meaningful
(make a real difference for participants)
Useful
(help in decision-making)
Reasonable
(non-trivial, logical)
Realistic
(achievable, given our resources)
Goals—
ultimate impact our program is expected to make
Beyond what you or your program do alone
the core mission/value does my program addresses?
HAVE A LUNCH BREAK HERE?????
so that
so that
so that
so that

16. Exercise
Create a So That chain for your project or program.

17. Outcome Evaluation Logic Model
Resources
What must this program have in order to function well?
Goals
What core community value does my program address?
Activities
What must we do to achieve our intended results?
If we’re planning we might go backwards around this diagram
If we’re evaluating we will go forwards
Define a common goal
Define common outcomes
Define our outputs
units of service
Define common activities
Interventions to improve individual and family life course
Identify the resources we have or need to achieve this
Outcomes
So what? What difference does our program make?
Outputs
How much do you do for how many people?

18. Developing Logic Models
Process
Outcomes
RESOURCES
ACTIVITIES
OUTPUTS
OUTCOMES
GOALS
Program inputs. Elements or ingredients that constitute the program.
Methods for providing the program. Specific processes or events undertaken.
Units of service or product units. How many, over what duration?
Short, intermediate, or longer-term changes anticipated in participants’ lives or in organizational or community conditions.
Ultimate impact(s) expected to occur, usually beyond what one program alone can achieve.
Define a common goal
Define common outcomes
Define our outputs
Define common activities
Service-based e.g.
WIC vouchers School-based clinics
Immunizations CD case investigation
Dental sealants Family planning
Education-based
Breastfdg ed & support Car seat education
Nutrition education Tob. prevention
Child Care Consultants
Mobilization-based
Train cmty mbrs in a body of info
Assm’t & data sharing
Coalition bldg
Identify/build relationships w/ diverse stakeholders
Educate comty & policy-makers re scope of situation
Identify the resources we have or need to achieve this
DO PH RAP SONG HERE…..???

19. Why a Logic Model? A logic model Clarifies each program element
Shows the relationship of resources and activities to expected outcomes
Summarizes how program parts relate to the whole
Makes explicit the theory of how the program works
Identifies categories to measure in the program evaluation

20. Exercise
Start your own (work related) logic model (individually)

21. Logic Models: “So That” Chains
Resources
Activities
Outputs
Outcomes
Goals
so that
so that
so that
so that
Logic Model
So why do we need those new running shoes….
So that….we can go running
So why do we go running
So that….increase our mileage, improve stamina
So why do we want to increase our miles
So that….we can improve our time from last year
Or decrease our blood pressure
In the past we often stopped asking the “so that” questions after our INPUTS
This kept us from answering the real questions related to what we intended to ACHIEVE
These elements are what make up a LOGIC MODEL

22. Exercise
Logic model scramble (everyone)

23. Outcome Levels Individual performance outcomes Program outcomes
Agency outcomes
System outcomes
Community outcomes
OFS, Page 23
1) Individual performance outcomes
2) Program outcomes
Changes in individs, families, organizations, or the cmty
that can be influenced by the program alone
3) Agency outcomes
Community level change based on the agencies as a GROUP of programs
e.g. increased birth spacing (our FP clinic, PHN HVs, DSHS liaison, etc)
4) System outcomes
A group of agencies working together to increase safety for vicitms of DV
5) Community outcomes
A joint effort of multiple agencies, funders, community groups
Established through community goal setting
E.g. decreasing teen pg

24. Developing Logic Models
Process
Outcomes
RESOURCES
ACTIVITIES
OUTPUTS
OUTCOMES
GOALS
Program inputs. Elements or ingredients that constitute the program.
Methods for providing the program. Specific processes or events undertaken.
Units of service or product units. How many, over what duration?
Short, intermediate, or longer-term changes anticipated in participants’ lives or in organizational or community conditions.
Ultimate impact(s) expected to occur, usually beyond what one program alone can achieve.
Define a common goal
Define common outcomes
Define our outputs
Define common activities
Service-based e.g.
WIC vouchers School-based clinics
Immunizations CD case investigation
Dental sealants Family planning
Education-based
Breastfdg ed & support Car seat education
Nutrition education Tob. prevention
Child Care Consultants
Mobilization-based
Train cmty mbrs in a body of info
Assm’t & data sharing
Coalition bldg
Identify/build relationships w/ diverse stakeholders
Educate comty & policy-makers re scope of situation
Identify the resources we have or need to achieve this
DO PH RAP SONG HERE…..???

25. Exercise
Further develop your own (work related) logic model (individually)

26. Reading Logic Models 1 2 3 4 5 Your planned work Your intended results
Certain
resources
are needed to
operate your
Program.
If you have
access to them,
then you can
use them to
accomplish
your planned
Activities.
If you
accomplish
your planned
activities, then
you will
hopefully
deliver the
amount of product or service that you intended.
If you
accomplish
your planned
activities to
the extent you intended, then your participants
will benefit in certain ways.
If these benefits
to participants
are achieved,
then certain
changes in
organizations,
communities,
or systems might be expected to occur.
Resources/Inputs
Activities
Outputs
Outcomes
Impact 1 2 3 4 5
Your planned work
Your intended results

27. What Next With Outcome Measures?
Remember…
outcome measurement is not an end itself, but rather a means to an end.
Outcome measurement is of little value
unless it leads to further decisions and actions that seek to improved program (or agency or individual) performance.
Performance improvement doesn’t miraculously happen.
There are a number of specific steps and techniques
These steps are used in and outside of the health field to improve quality and performance.