Introduction to Evaluation

Objectives Introduce the five categories of evaluation that can be used to plan and assess ACSM activities. Demonstrate how each evaluation category can be used to answer different evaluation questions. Review examples of each type of evaluation.

Evaluation is Part of the M&E Plan M&E Plan FrameworkIndicators Data Collection Data Quality Data Use and Reporting Evaluation Strategy Budget

Monitoring versus Evaluation Monitoring Data collected on program activities Ongoing, routine Focus on process and output, compared to target Are we doing the work we planned? Evaluation Data collected to answer specific questions Periodic Focus on outcome, impact How should we design our activities? How effective were they?

Categories of Evaluation 1.Baseline evaluation (needs assessment) 2.Formative evaluation 3.Process evaluation 4.Outcome evaluation 5.Impact evaluation

Baseline Evaluation/Needs Assessment Identifies our ACSM starting point Who do we need to reach and how? What activities are most important or most needed? Where do we need to go? What is the baseline value for key indicators?

Formative Evaluation Helps design activities for the target group Strengthens interventions by getting the right message to the right people Pre-testing Do we have the right message? Is this the right communication method? Are we reaching the right people? Do they understand the message?

Process Evaluation Documents what happens in a program What is going well, what is not, and why Often overlaps with monitoring Did we do what we planned? Is the program being implemented effectively? What challenges do we need to address? What best practices can we share? Lessons learned?

Outcome Evaluation What was the quality of our work – were we effective? Did we contribute to changes in key outcomes?

Impact Evaluation Contribution of ACSM interventions to high- level goals and objectives Generates evidence about effectiveness of ACSM activities Very rigorous, time consuming, and expensive Relies on high-quality baseline data collection. MUST be planned from the very beginning. In practice, rarely done.

Match the Question and Category! Do focus group members understand this message? FORMATIVE Did our program contribute to higher case detection in our country? IMPACT How many people know the symptoms of TB before we launch our media campaign? BASELINE

Match the Question and Category! Did pharmacists refer more customers for DOTS after our training? OUTCOME What did we learn about conducting effective advocacy work this year? PROCESS

Desig n ImplementationEnd BASELINE FORMATIVE PROCESS OUTCOME IMPACT When do we evaluate?

Creating a Framework BASELINE FORMATIVE PROCESS OUTCOME IMPACT

Questions?