1. 1 What are Monitoring and Evaluation? How do we think about M&E in the context of the LAM Project?

2. 2 Objectives of Monitoring & Evaluation Collect, analyze and use accurate and reliable information Brazil

3. 3 Objectives of Monitoring & Evaluation Improve program planning Improve program performance Improve program management Determine outcomes of activities Evaluate program impact on beneficiaries

4. 4 Characteristics of Monitoring Occurs throughout project on a regular basis Provides early indication of progress, or a lack of progress Provides periodic oversight of activity implementation

5. 5 Characteristics of Monitoring Can be tasked to any staff, not a specific evaluation staff person Informs a project about strengths & weaknesses: do we refine our strategy?

6. 6 Characteristics of Monitoring Determines whether activities are being performed correctly  To what extent are planned intervention activities being realized?  What services are provided, to whom, when, how often, for how long and in what context?

7. 7 Is Monitoring Supervision? We monitor an intervention We supervise an individual

8. 8 Characteristics of Evaluation Time-bound Systematic and objective Assess performance and impact Carried out selectively – costly & time consuming Process & Impact  Process: organization & management  Impact: effects on beneficiaries

9. 9 Types of Evaluations 1.Baseline Surveys: provides project with information about the target population at beginning of project intervention  Knowledge about breastfeeding, complementary feeding, maternal nutrition, and LAM (BCM/LAM)  Practices related to BCM/LAM  Cultural beliefs related to BCM/LAM  Household-based in the community  Large, representative sample

10. 10 Types of Evaluations 2.Endline Surveys: provides project with information about the target population at the end of project intervention  The endline survey is compared to the baseline and like the baseline it is: - Household-based in the community - Large, representative sample

11. 11 Types of Evaluations Endline Surveys: The endline asks the following questions:  Did we meet our objectives?  Did we have an impact: is there a measurable difference in people’s knowledge & practice?  Where were we strong and where were we weak?  What problems remain to be addressed?

12. 12 Types of Evaluations 3.Performance Monitoring: measures knowledge & skill levels of individuals trained in the field  Results are used to refine training strategy & define future training needs  Results are used for better supervisory visits

13. 13 Types of Evaluations Performance Monitoring:  Ideally, performance monitoring evaluations are not necessary ─ An effective supervisory system would include on-going evaluation of field worker’s knowledge & skills (counseling, methodologies) following training

14. 14 Monitoring & Evaluation Indicators How do we track & measure progress over time? Guatemala

15. 15 What is an Indicator? A variable That measures One aspect of a program

16. 16 What is an Indicator? A measure of whether you are heading in the right direction A measure of what your program has achieved A measure of the behavior you’re monitoring

17. 17 An appropriate set of indicators will include at least one for each significant element of the program

18. 18 Characteristics of Indicators Quantitatively or qualitatively measurable Relevant to the goals they represent Objectively verifiable & reliable Meet international professional standards Understandable & appreciated by project participants and other stakeholders Clarity on the appropriate levels: clinic- specific, country wide

19. 19 Characteristics of Indicators Clarity on the appropriate levels  Is the indicator clinic-specific?  Is the indicator region-specific?  Is the indicator country wide?

20. 20 Developing Indicators Establish during initial program planning Reflect overall objectives Link to behaviors Include with baseline

21. 21 How To Measure Indicators? Knowledge, practice and coverage surveys (KPC) Health Facility Records Health Information Systems Formative research to identify behaviors & indicators

22. 22 LAM Indicators 1.Exclusive Breastfeeding Rate, Infants 0 - < 6 months Guatemala

23. 23 LAM Indicators Exclusive Breastfeeding Rate, Infants 0 - < 6 months  Age range for LAM to be applied  Eligible LAM users  Optimal infant feeding message  Includes ALL infants in the age range  Can also be written 0 – 5 months

24. 24 Exclusive Breastfeeding Rate, Infants 0 - < 6 months WHO indicator using 24 hour recall Reflects women eligible to use LAM 1995: 46.1% 1998 – 1999: 38.8% Decrease of 7.3% Source: Guatemala DHS Reports

25. 25 LAM Indicators 2.LAM Rate: Proportion of eligible women (infants less than 6 months), who choose LAM as a method of birth spacing Altiplano, Bolivia

26. 26 LAM Rate Formula: Current LAM users divided by all women with infants 0 < 6 months Accuracy: A true estimate of LAM use by eligible women Data Source: Service statistics (MIS) & Household-level survey (DHS)

27. 27 LAM Indicators 3.LAM User Rate: Proportion of all women of reproductive age who use a modern contraceptive method (few eligible to use LAM) who choose LAM as a method of birth spacing Llano, Bolivia

28. 28 LAM User Rate Formula: Current LAM users divided by all women of reproductive age who use a modern method of family planning Accuracy: Underestimates actual LAM use by eligible women Data Source: Service statistics (MIS) & DHS

29. 29 LAM Indicators 4. Appropriate LAM Rate: Proportion of women who give birth in a given period of time who consciously & deliberately accept LAM as a birth spacing method and meet the 3 LAM criteria Guatemala

30. 30 Appropriate LAM Rate Formula: Number of women who use LAM as a birth spacing method and meet the 3 LAM criteria divided by the total number of women with infants 0 < 6 months

31. 31 LAM Indicators 5. Model LAM Rate: Proportion of women who give birth in a given period of time who consciously & deliberately accept LAM as a birth spacing method, meet the 3 LAM criteria, and know the 3 LAM criteria Jordan

32. 32 Model LAM Rate Formula: Number of women who use LAM as a birth spacing method, meet the 3 LAM criteria, and know the 3 LAM criteria divided by the total number of women with infants 0 < 6 months

33. 33 Denominators Exclusive Breastfeeding Rate:  Infants 0 < 6 months LAM Rate:  Women with infants 0 < 6 months LAM User Rate:  Women of reproductive age using a modern contraceptive method Appropriate LAM Rate:  Women with infants 0 < 6 months Model LAM Rate:  Women with infants 0 < 6 months

34. 34 LAM Monitoring & Evaluation Strategies Integration of LAM as a contraceptive method into the national MIS LAM acceptors integrated into national MOH/FP data

35. 35 Lessons Learned Increase and sustain LAM Users  Optimal breastfeeding practices  Integrated & supportive environment

36. 36 Lessons Learned Data Collection  Monitor clinics for accurate data collection  Absolute LAM numbers do not reflect LAM success – need proportion  Additional information - Age of infant - Family Planning Transition information - Length of time using LAM

37. 37 Lessons Learned Integrate the concept of Monitoring  Utilize results of supervisory checklist for program management

38. 38 Next Steps Review monthly and quarterly data for changing LAM trends Review MIS and supervision data on a regular basis Conduct regular review meetings and analysis for program management