Helping Our Clients Identify and Measure Outcomes Edward M. Haugh Jr. ESC Consultant ESC Brown Bag # 2, April

Introduction By the end of this session: Define outcomes Differentiate outcomes from outputs Define the logic model Describe several uses of a logic model Develop a logic model Define “indicators” Develop a logical framework

Impact You will love or at least tolerate outcome- based planning You will be able to discuss outcomes based planning, monitoring and evaluation with your ESC clients

What is an outcome Outcomes are benefits or changes for individuals or populations during or after participating in a program:

Outcomes Often expressed as verbs in past tense – Improved (nutrition, morale) – Increased (in employment) – Reduced (HIV transmission) – Enhanced (knowledge) – Maintained (vaccination coverage)

Outcomes  Reduced smoking among teenagers  Changed attitudes towards smoking (by non-smoking teenagers  School readiness among children attending preschool program

Other examples  Improved board performance after an ESC engagement  Improved strategic focus after an ESC engagement  Employment after attending a dress-for- success program

Improving the Delivery of Prenatal Care and Outcomes of Pregnancy: A Randomized Trial of Nurse Home Visitation

Outcomes of Nurse Visit Program Became aware of more community services Attended birth classes more frequently Made more extensive use of the nutrition supplementation program for women, infants and children Made greater dietary improvements,

Outcomes of Nurse Visit Program Reported that their babies’ fathers became more interested in their pregnancies Were accompanied to the hospital by a support person during labor more frequently Reported talking more frequently to family members, friends and service providers about their pregnancies and personal problems Had fewer kidney problems

Outcome Indicators Outcomes sometimes confused with outcome indicators – Outcome - girls participating in a training program obtain formal employment – Outcome indicators – the # and % of participants employed within 3 months of completing the program

Importance of Focusing on Outcomes Outcomes are what matter to stakeholders Focusing on outcomes promotes accountability Government, and funders demanding real tangible results on the ground Measuring outcomes helps us know if we are we really getting value for our money

Outcomes help us Strengthen existing services Target effective services for expansion Identify staff and training needs Develop and justify budgets Prepare long-range plans Focus board on programmatic issues

Funder Asked ESC the Following: What are your measurable outcomes, i.e. number of communities impacted, number of children served? Mark each category that applies to your grant measures. Increased visibility/awareness of issue Effected policy change Engaged constituents & beneficiaries and participation Increased resources for issue Educated constituents Improved condition/ quality of life in community Influenced stakeholders

Outcomes vs. Outputs Outcomes are the results Outputs are the tangible products produced by an intervention – Usually expressed as nouns – Can be counted

Outputs Classes taught Brochures distributed Clients served Contracts completed Board retreats completed Strategic plans prepared

Many Nonprofits Report Outputs Rather than Outcomes Main Street Homeless Shelter provided 2,903 nights of safe shelter for 173 men. 23,661 hot, nutritious meals were served at the Maple Avenue Kitchen. 100 residents received free health care and 168 residents received free mental health care.

What Wrong with Using Outputs Mosquito bed nets might be used for wedding dresses or fishing nets Loans repaid might not mean income generated Job training might not lead to jobs

So Why Don’t We Focus on Outcomes Outputs are tangible Outcomes are intangible Its easier to count tangibles than to describe intangibles Outcomes take longer to produce Measuring outcomes is challenging Measuring outcomes can be expensive and time consuming

But Tracking Outcomes will Not…… Explain why an outcome is or is not being achieved – Still need to track inputs, activities, outputs Prove that the outcome is the result of your program

Outcome Model Logic Model Inputs Processes Outputs Outcomes Impact

Logic Model Definitions Inputs are the resources required to undertake activities Activities (or processes) are what the agency, program or strategy does to fulfill its mission Outputs – tangible products Outcomes – changes, benefits Impact - a longer-term outcome

Inputs Processes Outputs Short-term Outcomes M edium- term Outcome Money Staff time Facilities Equipment Laws Regulations Funders requirements Educate consumers Educate providers Distribute vaccines Establish data base Establish reminder systems Monitor Coverage Brochures Vaccine supplies Doses of vaccines delivered Data bases established Providers monitoring Parents aware of vaccine benefits Provider awareness Changed attitudes Children immu nized Reduced incidence of HPV Reduced number of girls who are infected with HPM Long-term Reduced incidence of cervical cancer Reduced mortality for cervical cancer A logic model for a program to distribute HPV Vaccine Impact Reduced premature death

InputsActivities Outcomes Outputs Impact It is important to note the broad environmental context of the intervention: Political environment Macroeconomic picture Political environment Other “players” in field Political environment Public attitudes Environment Policy context Source: Imas and Rist (2009)

How can we use the logic model ESC Project planning Strategic Planning Program Monitoring Program Evaluation

Inputs Processes Outputs Short-term Outcomes Medium- term Outcome Staff time CRM Time Lead time Consultant time Materials Car and gasoline Knowledge Experience Place to meet Contracting Analysis Study Meetings w client Meetings with stakeholders Consultant meetings Progress reports Draft report Final report Board presentation ?? Impact ? Logic model for an ESC Engagement Deliverables

Inputs Process Outputs Short-term Outcomes Medium-term Outcome Strategic Plan Approved Impact Example of a logic model for an ESC Strategic Planning Engagement Strategic Plan in regular Use Longer- term Outcome Strategic Plan improves agency performance Community benefits from agency’s improved performance

Inputs Programs Output s Short-term Outcomes Money Staff time Facilities Equipment Laws Regulations Funders requirements Program A Impact Non-profit Logic Model Program B Program C Program D Output A Output C Output B Output D Outcome 1 Outcome 2 Outcome 3

Outcome 1: Improved Financial Position Outcome 5: New Training Program Developed Outcome 2: New Facility Developed Outcome 3: Strengthened Volunteer force Outcome 4: Improved Public Recognition Improved Ability to Fulfill Mission

Outcome 1: Improved Financial Position Improved Ability to Fulfill Mission Strategy 1: Capital Campaign Completed Strategy 2: Increased donor base Strategy 3: Client Fees Increased

Indicators Measure the quantity or quality of the outcome

Indicators Outcome: Participants in a job training program obtain a job in a timely manner # (%) of participants employed within 6 weeks # (%) of participants still employed 3 months after completing the program

Indicators Outcome: Homeless people obtain secure housing # (%) of homeless moved into temporary quarters and stayed for 6 weeks # (%) of homeless who obtained a job and were still employed after 3 months

Indicators Targets # (%) of participants employed within 6 weeks # (%) of participants still employed 3 months after completing the program 60% of participants will gain employment within 6 weeks 95% of participants who obtained employment will still be employed in 3 months

Logical Framework ComponentIndicatorBaselineTarget Outcome Output Program/ Activity/ Process Input

1. Choose the Outcomes to Measure 2. Specify the Indicators for the Outcomes 3. Prepare to collect data on the Indicators 4. Try out the Measurement System 5. Analyze and Report Findings 6. Improve the System What Can Our Clients Do to Measure Outcomes 7. Use the Findings

Two Great Resources Measuring Outcomes, A Practical Approach, The United Way Foundation Logic Model Development Guide, WK Kellogg Foundation

Thank you for participating in ESC Brownbag No. 2. Look for more Brownbags to come