1. Developing Indicators of Success
Tammy Horne, Ph.D.
WellQuest Consulting Ltd.
(780)

2. Indicators of Success/Progress
Indicators are specific, observable, and measurable characteristics or changes that show the progress a program is making toward achieving a specified outcome. (CDC, 2001, page 52)
Indicators need to be specific. A common error in writing indicators is to repeat the outcome without actually defining it:
8/23/2019

3. Indicators of Success/Progress
Outcome: Stay-at-home parents will increase their social connections with each other over the next 3 months
Indicator: % (or #) of parents at home full- time who report making new friends with other parents at home full-time who have children of similar ages
NOT: % (or #) of stay-at-home parents who become socially connected
8/23/2019

4. Indicator Examples Outcome:
Decreased exposure of adult nonsmokers to environmental tobacco smoke (ETS)
Indicators:
% of adult nonsmokers who report they have not been exposed to cigarette smoke during the previous 7 days
8/23/2019

5. Indicator Examples
% of adults who report they are never exposed to cigarette smoke in restaurants  
% of adults who report they are not exposed to cigarette smoke at work during a typical work day
Example taken from
8/23/2019

6. A Note on % Indicators Indicators involving %’s need a denominator
Often the denominator is implied – such as all program participants or the total population of an area
8/23/2019

7. A Note on % Indicators
We could also write the indicator to make the denominator, and thus our calculation of %, explicit:
# of participants who report making new friends through the program
____________________________ All program participants
8/23/2019

8. Qualitative Indicators
Indicators have traditionally been numerical (quantitative)
Qualitative indicators give us more in-depth information on the meaning of change for people, rather than simply
counting whether a change did or did not happen.
8/23/2019

9. Qualitative Indicators
For example, in addition to looking at percentage of stay-at-home parents who make new friends, we can define what the friendships might look like (what people do together, how they involve children or not, etc)
8/23/2019

10. More Indicator Examples
Outcomes:
Young adults recovering from alcohol or drug addiction will have more connections with (a) non-using peers, (b) adults who focus on their strengths, and (c) supportive neighbourhood organizations, within three months of contact with an outreach worker
8/23/2019

11. More Indicator Examples
Indicators:
(a) % [or #] of recovering youth reached who do recreational or social activities with people their age who are not using alcohol or drugs; youth descriptions of types of activities they do with these other youth (qualitative)
8/23/2019

12. More Indicator Examples
(b) % [or #] of youth reached who reporting spending time with at least one adult who recognizes what they are good at or have potential to learn; youth describe type of contact with such adult(s) – e.g., encouragement to pursue their talents, coaching or mentoring of some type (academic, sport, arts)
8/23/2019

13. More Indicator Examples
(c) % [or #] of youth reached who make at least one visit a week to a neighbourhood organization that offers positive alternatives to alcohol and drug use; youth descriptions of the types of activities they do at the organization (e.g., arts, sports, recreational physical or social, activities, education in an area of interest to them, volunteer work with younger children, youth leadership
8/23/2019

14. More Indicator Examples
Outcome:
Children in our program achieve
developmental milestones appropriate to their age and/or developmental ability
(In this case, timelines may vary for each
child)
8/23/2019

15. More Indicator Examples
Indicators
Number (%) of children will demonstrate age/developmentally appropriate:
Communications skills (e.g. speaks, responses verbally to questions)
Gross motor skills (e.g. throws a ball, climbs, hops
8/23/2019

16. More Indicator Examples
Fine motor skills (e.g. draws with a pencil, does puzzles, uses scissors)
Problem solving skills (e.g. understands size and direction)
Personal/social skills (serves food, washes hands, knows names of others) 
Above examples based on Ages and Stages Questionnaire (ASQ)
8/23/2019

17. Application # 2: Indicators
Take one of the outcomes you developed in Application #1
What would be a good indicator for that outcome? Might there be more than one indicator for it?
Would your indicator(s) be quantitative, qualitative, or both?
8/23/2019

18. Limits of Performance-Focused Evaluation
Logic models and performance indicators have come to dominate evaluation of publicly funded projects/programs
Fits with government accountability focus
8/23/2019

19. Limits of Performance-Focused Evaluation
This is helpful for upfront planning, but can lead to an excessive focus on outputs and quantifiable outcomes, at the expense of an in-depth look at process and context, and of qualitative data
Can encourage monitoring without more in-depth evaluation
8/23/2019

20. Limits of Performance-Focused Evaluation
“Preoccupation with measuring a few indicators creates a kind of tunnel vision that shuts out the complexity of reality. Things appear simpler and more controllable than they are.” (Westley et al, , p. 240)
From: Getting to Maybe (Random House)
8/23/2019

21. Limits of Performance-Focused Evaluation
We get preoccupied with ‘did we achieve our outcomes?’ – without asking why or why not, what else happened or did not
What gets measured gets done – can lead to narrow focus on pre-determined performance criteria, rather than being open to evolving needs/capacities and unexpected findings
8/23/2019