1 Nemours Health Planning and Evaluation Collaborative Learning Session I: Designing Effective Health Programs

2 Learning Objectives How to plan child health programs with outcomes in mind How to plan child health programs with outcomes in mind How to apply sound theory to the planning of child health programs How to apply sound theory to the planning of child health programs How to develop a program logic model that specifies program outcomes How to develop a program logic model that specifies program outcomes

3 Planning with outcomes in mind

4 What do we mean by outcomes? Outcomes are changes in the people, organizations, or systems that we aim to serve Outcomes are changes in the people, organizations, or systems that we aim to serve

5 What do we mean by outcomes? Changes in people Changes in people –Knowledge –Attitudes & beliefs –Skills –Behavior –Health status

6 What do we mean by outcomes? Changes in organizations Changes in organizations –Vision –Mission –Strategies –Capacity –Capabilities –External Relationships –Internal Culture

7 What do we mean by outcomes? Changes in systems Changes in systems –System capacity –System cost –System organization –System coordination –System quality –System accessibility

8 What does it mean to “plan with outcomes in mind?” Begin by specifying what outcomes we want to achieve for people, organizations, or systems Begin by specifying what outcomes we want to achieve for people, organizations, or systems Work backward to design a program that could achieve these outcomes Work backward to design a program that could achieve these outcomes Make sure we are operating under sound program theory Make sure we are operating under sound program theory

9 Program theory

10 What is “program theory?” Program theory is the set of causal assumptions between your inputs, activities, outputs, and outcomes Program theory is the set of causal assumptions between your inputs, activities, outputs, and outcomes In other words: In other words: –What makes you so sure this will work?

11 Why is program theory important? Every child health program in the world has a program theory –Some are more explicit than others –Some are more sound than others –Flawed program theory is one of four basic reasons why programs fail:  Flawed program theory  Flawed program design  Flawed program implementation  Flawed program evaluation

12 Examples of program theory Discuss whether the following statements are sound or unsound Discuss whether the following statements are sound or unsound –If you immunize a child against MMR, then that child will avoid MMR –If you educate adolescents about the risks of tobacco, they will avoid smoking –If you provide a family with Medicaid, they will take their children in for EPSDT visits –If you teach a family how to make better food choices, their children will better control their weight

13 Logic models

14 What is a logic model? In its simplest conception, it is simply a graphic or tabular description of your program. In its simplest conception, it is simply a graphic or tabular description of your program. Looking deeper, the best logic models illustrate the underlying theory of the program, or "program theory." Looking deeper, the best logic models illustrate the underlying theory of the program, or "program theory."

15 Acme Child Health Enrollment Program AssumptionsInputsActivitiesOutputsOutcomes Theoretical assumptions about why a program will work The resources needed to deliver the program Key actions of program staff and clients Products of the program Changes in the target population Uninsured children who are eligible but not enrolled will become enrolled if their parents/ caregivers: Know they are eligible Know they are eligible Value coverage Value coverage Know how to enroll Know how to enroll Have assistance with enrollment process Have assistance with enrollment process Complete enrollment process Complete enrollment process 10 CBOs 10 CBOs 20 outreach workers 20 outreach workers 2 training experts 2 training experts Client families with children Client families with children $5,000 mini-grant to each organization $5,000 mini-grant to each organization Train CBO staff CBO staff: Interview parents/ caregivers about child health insurance status Interview parents/ caregivers about child health insurance status Educate about Medicaid-S-CHIP Educate about Medicaid-S-CHIP Offer assistance with application Offer assistance with application Guide through application process Guide through application process Monitor enrollment Monitor enrollment Parents/ caregivers of 500 children interviewed Parents/ caregivers of 500 children interviewed 100 educated and offered assistance 100 educated and offered assistance 70 accept application & assistance 70 accept application & assistance 50 are enrolled 50 are enrolled Families have peace of mind Children have access to needed health services Children have better health status Target Population: Uninsured children eligible for Medicaid or S-CHIP in Anytown, USA

16 Acme Child Health Enrollment Program Assumptions Uninsured children who are eligible but not enrolled will become enrolled if their parents/ caregivers: Know they are eligible Value coverage Know how to enroll Have assistance with enrollment process Complete enrollment process 10 CBOs with total of 20 outreach workers 2 training experts Inputs $5,000 mini- grant to each organization Client families with children Target Population Uninsured children eligible for Medicaid or S-CHIP Interview parents/ caregivers Educate eligible parents/ caregivers Activities Train outreach workers Provide application assistance 500 interviewed 100 educated Outputs 50 enrolled 70 accept assistance Families have peace of mind Children have access to needed services Outcomes Children have better health status 20 trained Monitor enrollment

17 Why are funders so interested? Logic models promote program improvement Logic models promote program improvement –The process of constructing a logic model forces the funded organization to develop more theoretically sound programs Logic models are efficient Logic models are efficient –When properly constructed, a logic model can convey the basic blueprint of a program in one or two pages. –When properly constructed, a logic model can convey the basic blueprint of a program in one or two pages. Logic models promote understanding and communication Logic models promote understanding and communication –Once funding organizations adopt the terminology of logic models -- terms like assumptions, inputs, activities, outputs, outcomes -- it gives them a common for understanding and discussing the relative strengths and weaknesses of diverse programs. –Once funding organizations adopt the terminology of logic models -- terms like assumptions, inputs, activities, outputs, outcomes -- it gives them a common for understanding and discussing the relative strengths and weaknesses of diverse programs.

18 Why are funders so interested? Logic models provide the foundation for practical program evaluation Logic models provide the foundation for practical program evaluation –A properly constructed logic model helps the funder identify the most important focal points for process evaluation and outcome evaluation. –A properly constructed logic model helps the funder identify the most important focal points for process evaluation and outcome evaluation. Logic models promote organizational learning Logic models promote organizational learning –As funders review and compare logic models, they learn valuable information about the theory and practice of health and human service programming. –As funders review and compare logic models, they learn valuable information about the theory and practice of health and human service programming. –This helps the funder make better decisions and become a more valuable partner in their ongoing relationship with community health and human service programs.

19 Why are grantees so interested? Because funders are interested! Because funders are interested! –Programs with a well constructed logic model may give themselves a competitive advantage in fundraising. –Programs with a well constructed logic model may give themselves a competitive advantage in fundraising. Because logic models improve program planning Because logic models improve program planning –The process of constructing a logic model forces programs to ask themselves a number of questions about their target population, assumptions, desired outcomes, etc. To the extent that these questions are raised and answered during the planning phase, the chances of successful implementation are increased. –The process of constructing a logic model forces programs to ask themselves a number of questions about their target population, assumptions, desired outcomes, etc. To the extent that these questions are raised and answered during the planning phase, the chances of successful implementation are increased.

20 Why are grantees so interested? Because logic models support program implementation. Because logic models support program implementation. –Many organizations that develop logic models for funders also find that periodic review of the logic model in staff or board meetings can help the organization stay focused on its mission assure that its resources are being directed toward the most important outcomes. –Many organizations that develop logic models for funders also find that periodic review of the logic model in staff or board meetings can help the organization stay focused on its mission assure that its resources are being directed toward the most important outcomes. Because logic models support practical program evaluation Because logic models support practical program evaluation –By using the logic model as the basis for evaluation objectives, both the funder and the funded organization can assure themselves that the evaluation is focused on the most essential aspects of the program.

21 If you can answer these questions.. –Who is your target population? –Who is your target population? –What outcomes, or personal changes do you want to help this population achieve? –What level of output do you hope to achieve, in terms of people served, products produced, etc. –What level of output do you hope to achieve, in terms of people served, products produced, etc. –What activities will be required to produce these outputs? –What activities will be required to produce these outputs? –And what inputs, or resources, will be required to complete the necessary activities? –And what inputs, or resources, will be required to complete the necessary activities? –What are the theoretical assumptions that lead you to believe that people exposed to this program will achieve the desired outcomes? –What are the theoretical assumptions that lead you to believe that people exposed to this program will achieve the desired outcomes?

22..you can construct a logic model! A logic model is simply a way of illustrating what you should already know about your program or plan A logic model is simply a way of illustrating what you should already know about your program or plan The most important purpose of the logic model is program improvement. The most important purpose of the logic model is program improvement. –If, in the course of constructing a logic model, you identify gaps in your knowledge or planning, you simply problem-solve until you can fill those gaps as best you can

23 Constructing your own program theory and logic models (Transition to Logic Model Development Guide)