Equity focused m&e Ngonidzaishe Marimo

Equality Focused M&E What is equity? Equity focused monitoring and evaluation? Managing, implementing and using equity focused evaluations?

What is Equity? Reduces poverty Economic growth Why is it important Peace and stability Equity refers to disparities that are avoidable, unfair and unjust Equity is embedded in the principle of social justice and fairness Comes from the idea of moral equality, that people should be treated as equals

What is Equity The proportion of teenagers who have begun childbearing decreases as wealth increases: five times more teenagers in the lowest wealth quintile (34 percent) have begun childbearing compared with teenagers in the highest wealth quintile (6 percent) (ZDHS, 2015) Older people above 60 years of age are more likely to have heart diseases

What is Equity Trends in childhood mortality for the 5-year period preceding the survey, Zimbabwe 1988-2015

What is Equity?

What is equity Of the children who were of the official primary school entry age, 73.3 percent were in the first grade of primary school, see Table ED.3. Minimal sex differentials were noted with 74.8 percent of girls in first grade compared to 71.8 percent for boys. Variations were noted by province and urban- rural areas. In Bulawayo Province, for instance, the value of the indicator reached 88.5 percent while it was 66.4 percent in Mashonaland West Province. (MICS, 2014)

What are the determinants of Equity Issues Location Remote areas are usually left out. Rural versus urban divide Gender Due to gender roles, gender power relations – women often excluded and therefore poorer than men Income Poorer households have limited access to services, opportunities and Disability Socially excluded, limited access to services,

Equity in M&E Equity focused M&E systems support delivery of equity focused programmes and policies M&E systems that interrogate equity enhance achievement of results on equity Be clear about the challenge of equity Political Cultural Power dynamics The M&E system needs to be able to capture and many of the underlying causes for inequity Equity is about disaggregated indicators Understand the drivers of equity: Location, gender, income, disability? Equity focused M&E system is inclusive and bottom up

Equity in M&E Equity focused M&E systems require data collection methods that provide information on status of indicator and reasons for such status Equity focused M&E systems require adequate financing as approach is more expensive than conventional national aggregated approaches Equity focused M&E Systems have a feedback loop to the bottom

Steps in Designing an Equity Focused M&E system Step 1: Establish an appropriate M&E system development structure that is inclusive Core team Reference Group Step 2: Understand the drivers of inequity Step 3: Selector indicators based on sectoral and national development plans disaggregated by the drivers of inequity Step 4: Select qualitative indicators for assessing the quality of context, power relations, social norms and cultural principles and values that undermine equity Step 5: Develop M&E framework (institutional arrangements, data flow, feedback, outputs, users and information they require, risk management matrix etc.) Step 6: Cost the M&E system and ensure provision in sector budgets Step 7: Develop tools, guidelines, reporting templates and baselines

Equity Focused Evaluations Preparation for the EFE Preparing TOR for the EFE Designing the EFE Collecting and analysing evidence Findings Conclusion Utilising the Evaluation Step 6 Step 5 Step 4 Step 3 Step 2 Step 1

Step 1: Preparation for the EFE Determining evaluability of the intervention’s equity dimension Clear Theory of Change Clear baselines Disaggregated indicators Clear Means of verification Identify evaluation stakeholders including worst off Steering committee/reference group (relevant to worse groups) - helps culturally sensitive, access to communities Ensuring an appropriate budget – more expensive Hard to reach areas Disaggregated data more expensive than national/province averages

Step 1: Preparation for the EFE Identify intended use by intended users Identify potential challenges in promoting and implementing equity focused evaluation Political Methodological Social challenges Develop strategy for dealing with challenges

Step 2: Preparing the TOR for an EFE Define the purpose and scope of the evaluation Selecting technically strong and culturally sensitive evaluation team Framing evaluation questions focusing on equity

Step 3: Designing EFE Select appropriate evaluation framework Select evaluation design Select methods Outcomes are difficult to define Conventional evaluation designs do not work Need for innovative designs Understanding the complex context in which EFE are implemented – scope for realist evaluation? Studying sensitive issues Vulnerability Political power relations Culture

Step 3: Design of an EFE- What is Credible Evidence Experimental designs Issue: - Can numbers capture complexity of equity focused interventions? In-depth analysis of lived experience Can narrative help explain the many factors Is it possible to generalise from qualitative studies? Roles of mixed methods?

Step 3: Designing an EFE – the challenge of attribution and assessing causality Causality and the definition of the counterfactual The difficulty of finding a comparison group The danger of positive bias Reaching beneficiaries and non-beneficiaries The human desire to find causality The danger of retrospective construction of causality Alternatives to the conventional counterfactual E.g Comparison with other countries with closely similar context etc

Step 3: Designing and EFE – importance of mixed methods Combining depth plus depth Combines narrative reporting of lived experience and cultural context, with the ability to generalise Integrated approaches draw on tools and techniques from a broad range Broadening the range of credible evidence Triangulation

Step 3: Designing and EFE – Evaluations framework Experimental or qualitative Theory based evaluation Use of the theory of change - challenge no alternative reason for change or rival hypothesis – potential for improving theory of change Systems based approach New complex thinking in development evaluation Try to understand actors and agencies, boundaries and historical conditions Understand development as a complex system embedded in a diverse context Expenditure and cost effectiveness Cost effectiveness Benefit incidence analysis - % of budget reaching bottom quintiles Public expenditure tracking Tracking % of resources reaching the frontline

Step 3: Designing and EFE – Evaluations framework Bottleneck analysis Factors that affect delivery of services (supply side) and utilisation of services (demand side) Supply side factors: Coverage, budget, cultural sensitive approaches (including staff) Demand side factors: Knowledge and attitudes Time poverty Out of pocket expenses for transport or paying for services Community ownership (ability to complain for example) Contextual factors (political, economic, legal and administrative) Design of the evaluation framework depends on the type of programme/policy being evaluated: Complex equity focused policies or programmes Simple localised programmes

Step 3: Designing an EFE – Complex equity focused policy/programme Systems analysis Unpack complex policies into components that can be more easily be evaluated Speaks to complexity and size of team Pipeline design Using other countries as comparison e.g changes in HIV rates Policy gap analysis which groups fall through the cracks

Step 3: Designing an EFE – Simple programmes Experimental/quasi experimental Creative design to define comparison group Naturalistic experiments Theory based approaches Non-experimental design Simple case studies Qualitative/narrative methods Longitudinal design Panel surveys Interrupted time series Case study design using mixed method sample selection Holistic analysis

Step 4: Collecting and analysis data Multiple methods Structured questionnaires Observations Key informant interviews Focus Group Discussions Transact walks etc. Be clear to link method and the desired information Be clear of the link between your quantitative and qualitative data before analysis.

Step 5: Findings and Conclusions Findings should demonstrate Equity of results Qualitative analysis of the status Conclusions Policy or programme effect on equity Factors underpinning success or failure

Step 6: Utilisation of findings Parliament Advocate for improved delivery on equity Advocate for policy revisions Disseminate evaluation findings to the public Government Improve or change programmes Undertake policy reform Development partners

Step 6: Utilisation of findings Under utilisation of results Not addressing client concerns Poor timing Not using stakeholder’s preferred communication style Only selected stakeholder receive reports Important to have a utilisation focused and participatory evaluation Alternative forms of reporting Agreeing on Action Plan Disseminating the evaluation and preparation of a management response

Exercise Review the following PICES executive summary and answer the following: Does the summary provide adequate information for equity focused programming? As an MP or government official what recommendations can you come up with improve equity of programmes based on the PICES survey? What recommendations can you come up with to improve equity in the presentation of results – what would you have wanted to see that is not presented?