1. Impact Measurement why what how 9-14-09 Atlanta

2. Today Imperatives Questions Why Now? Significant Challenges Breakthroughs in the field CARE’s Long-Term Strategy This Year How do we meet the challenges?

3. Imperatives …can only flow from a clearly- defined program approach that is predicated on a clear framing of the problem. …systems must actually add value for the organization and produce reliable data that it can be utilized to inform strategic decisions. Measurement …

4. Questions 1.If you do not measure our impact and demonstrate the change, then does it matter? 2.Can we measure the most important changes or do we emphasis what we can measure? 3.How do you actually measure social impact and do those type of changes inherently defy measurement - an increase in someone’s level of opportunity or their dignity 4.Does spending on measurement and building measurement systems mean less time and resources for the actual work that we do?

5. Most Significant Challenges What does ‘impact’ even mean? How do we even begin to define metrics that can cut across a plethora of disparate issues that CARE is addressing? Do we need counter-factual, experimental designs and benchmarks within the same issue areas to do so? How do you measure outcomes rather than inputs? Especially when the intervention is removed from the ultimate desired change - i.e., girl’s education programs with the goal of increasing women’s empowerment.

6. Most Significant Challenges (2) How do we measure impact for the wide variety of constituents that CARE is accountable to? How do we build the capacity across the organization to make sure that the measurement is consistent through a well implemented system.

7. Breakthroughs in the Field Shared Measurement Platforms Allow organizations to choose from a set of measures within their fields Comparative Performance Systems Require all participants within a field to report on the same measures Adaptive Learning Systems Allow people working on different aspects of a single complex issue to engage in an ongoing, facilitated process

8. Why Now ?  The program approach across CARE  Critical to demonstrate intended benefits  for organizational performance management  Support fundraising  Provide evidence for advocacy for LIFT-UP  Proactively meets future compliance

9. Levels of Measurement The proposed system covers all levels of measurement and all stages of the program cycle: Outputs Effects Impact Activities Inputs

10. Levels of Measurement The proposed system covers all levels of measurement and all stages of the program cycle: Outputs Effects Impact Activities Inputs

11. Levels of Measurement LEVEL 1 Input: Resources needed to undertake a set of activities Activity: The actions that convert inputs into outputs Output: The changes that are a direct result of activities

12. Levels of Measurement LEVEL 2 Effect:  Changes in human behaviors and practices  Improvement in services through increased competency (e.g. more effective extension systems)  Changes in systems and policies (enabling environment) LEVEL 3 Impact:  Long-term sustainable change  Contribution to the MDGs

14. Components Component 1 Program Data Collection System Sectoral Portfolio Review Component 2 Annual Impact Report Strategic Impact Inquiry Component 3 Meta-Evaluation Proposal Analysis Program Causal Analysis

15. Components of IM System Inputs Activities Outputs Impac t Effects

16. Access Africa How are we measuring our impact

17. Increased general well being of participants household Increased in economic, social and political empowerment participants Monitoring & VSLA quality Activity/Output Level R2. Retention rate R5. Annualized return on savings R8. Portfolio at risk R16. Cost per member assisted Outcome Level % of VS&L members who have invested in productive assets % women holding leadership position at community level Level of implementation of policies for women right on land and other productive assets Impact Level % population below $2 per day (MDG ) Impact Measurement System

18. Activity & Output level: MIS System  Excel based system to Manage Field Officer performance & VSLA Quality  Measures 4 categories of performance ratios (SEEP): oMembers Satisfaction oVSLA Financial Performance oVSLA operating efficiency oCARE operating efficiency  Quarterly data collection by projects in the field  Access Africa aggregates global data quarterly  Data enables AA to engage in Program Quality discussions with the COs  Common system used by OXFAM, CRS, PLAN, Aga Khan, FfH, etc.

19.  Common indicators/tool across COs  Option of methodology:  Client Survey: data collection on participants household  Household Survey: data collection on community members (participants and non-participants)  RCT, with IPA in Uganda and Malawi  Mixing quantitative and qualitative methods  Capacity building of COs – TDY, direct support from AA  Support COs in hiring relevant staff and/or consultants for M&E What about Outcome and Impact?

20. Making the information Available

24. The Directory Name Program # Goal Manager Contact Information Associated Project Program Information Name PN/FC # Manager Contact Information Name PN/FC # Manager Contact Information Name PN/FC # Manager Contact Information Name PN/FC # Manager Contact Information Project Information Searchable information in both categories: Start Dates and End Dates of Projects and Programs  Impact Population  Target Group  Signature Program component  MDG Impact Area program or project works in  Number of direct participants in the program  Gender and Age  Activities and Outputs  CARE partners  Advocacy Goals, Partners and Targets of Advocacy Efforts

25. This year  Work with Country Offices to create and disseminate appropriate guidance on impact measurement  Work with Technical Units in Atlanta to help put impact measurement systems in place in select Learning Labs.  Focus on organizing consultations on and outlining a business process for impact measurement at CARE, complete with a resource development plan  Working with the Policy Advocacy Unit and Communications on CARE’s Annual Impact Report.

26. Finished Finally