Module 13 Integrating EFGR evaluations into the SDG country-level review Technical Assistance on Evaluating SDGs: Leave No One Behind EvalGender+ Network together with UN Women, UNEG and EvalPartners

Presentation developed by Michael Bamberger and Asela Kalugampitiya based on Chapter 5 of ”Evaluating the Sustainable Development Goals within a “No-one left behind” lens through equity-focused and gender-responsive evaluations”

Outline What is a national evaluation policy (NEP)? Integrating an EFGR lens into SDG national evaluation policies/systems Identifying and working with stakeholders Developing an EFGR evaluation system Developing an advocacy strategy Example of an advocacy campaign focused on parliamentarians

1. What is a National Evaluation Policy? National Evaluation Policies (NEPs) refer to formal policies defining the purpose, responsibilities, functions and organization of the public sector evaluation function in a particular country. (Bamberger, Segone and Reddy 2014)

Global Mapping on NEPs First study in 2013 and update in 2015 http://www.pfde.net/index.php/publications-resources/global- mapping-report-2015 109 countries were studied Only 20 countries have formal National Evaluation Policies Only 3 countries have reference to gender equality: Nepal, Kenya and Zimbabwe. Afghanistan draft results based M&E policy includes equity and gender (section 3.3) Documented six case studies on NEP with selected countries Documented four case studies on EFGR M&E systems with selected countries

Ways of incorporating evaluation at national level Evaluation is included in the constitution: Morocco and Nepal National Evaluation Policy endorsed by the government: Canada, Japan, Uganda, Malaysia, Colombia, Zimbabwe National Evaluation Policy Framework endorsed by the government: South Africa National Evaluation Policy Act approved by the parliament: Nepal (draft, awaiting approval) Sector wise evaluation policies approved by designated ministries: eg. Evaluation policy for HIV/AIDS programmes

Building gender into the NEP- Different scenarios Scenario 1: Gender equality is fully integrated in to the NEP itself. Eg. Afghanistan draft Results Based M&E Policy – sec 3.3, Zimbabwe NEP. Scenario 2: Countries where the NEPs integrate gender equality thanks to a national gender policy. Eg. Ethiopia Scenario 3: Countries where the national development strategy integrates gender equality. Eg. Nepal Scenario 4: Countries where the NEP does not include gender equality, but where gender-responsive evaluations are regularly conducted. In some countries gender equality is considered important by government, but the NEP is presented on a high level of generality which does not address sector, and cross-sector specific issues. Eg. Mexico

2. Integrating an EFGR lens into SDG evaluation policies and systems Challenges Political challenges The more in-depth analysis of gender and equity issues can reduce the rating of country programs on indices such as the Human Development Index EFGR analysis often shows equity gaps have increased or some groups are left behind EFGR focuses on socially marginalized groups that society may prefer to ignore Laws sometimes forbid collecting data on some vulnerable groups (e.g. ethnic minorities)

Integrating an EFGR lens into SDG evaluation policies and systems B. Methodological challenges Requires disaggregation of data – difficult and expensive Difficult to study subtle processes of social exclusion Social exclusion analysis may require the integration of different data sets which can be difficult – resistance from some agencies Difficult to obtain precise measures of income and wealth

Reasons why many national evaluations do not address gender In 2014 only 14 developing countries had a national evaluation policy (NEP) – probably required to facilitate complex and ambitious EFGR evaluations Reasons why gender often not incorporated into the NEP Many NEPs only provide high level framework Many NEPs do not address cross-cutting themes such as gender Often gender focus limited to monitoring women’s access to services or representation on community committees Many stakeholders do not consider gender a relevant issue for national policy evaluation

Lessons from the development agencies on the incorporate of gender equality into evaluations Gender equality must be considered a central development objective in its own right, as well as essential to achieving other priorities and sectoral objectives There is a need for a string organizational commitment with direct senior management responsibility The should be a clearly defined rational for a gender-responsive development strategy. Different agencies prioritize different issues (for example a business case or a human rights focus) and NEP should clarify which rationale, or set of rationales are prioritized. The importance of guidelines, checklists and practical example of gender- responsive evaluation Importance of alliance building and inter-agency coordination Recognizing that gender interventions are frequently broad-based and complex and that gender policies and evaluations must recognize this complexity.

Reasons many national evaluations do not address equality issues Political resistance to addressing equality Reluctant to challenge their positive image of development progress on conventional indices An equity focus is technically more challenging Increased cost of data collection and analysis Many indicators such as ethnicity, religion of physically disability are not included in national statistics so secondary data not available Many voulnerable groups are largely invisible and difficult to study

Integrating an EFGR focus into the country-led evaluation strategy Define responsibilities: often no agency is responsible for ensuring gender and equity are addressed in the evaluations Coverage and selection of evaluations: Evaluations must cover many issues so gender and equality can easily be ignored Develop guidelines to ensure gender and equality are always included where appropriate

Integrating equality and equity into national evaluation policies and systems…… Incorporate equality into the NEP guiding principles for the selection, design and use of evaluations. Coverage/selection of evaluations. Ensure that equality issues receive an appropriate priority in the criteria used to select the evaluations to include in the annual NEP evaluation program. Methodological techniques. Methodological guidelines should be developed for conducting equality-focused evaluations.

Elements of a gender-responsive evaluation system can include: National evaluation steering committees and similar bodies should include members with gender expertise. National evaluation offices should recruit staff with gender expertise and ideally with specific experience in gender-responsive evaluation. All staff in the national evaluation office and related bodies should attend training on gender responsive evaluation Gender ministries or other gender focal points and gender advocates should be consulted when developing gender plans and policies and terms of reference.

Elements of a gender-responsive evaluation system can include……. Government bodies should draw on the gender expertise of VOPEs. Gender-balanced national evaluation systems are encouraged. Parliaments and other government budget making bodies should provide adequate budgets for collection and analysis of sex- disaggregated data and gender-responsive monitoring. Donor and UN commissioned evaluations should include guidelines to ensure all evaluations (where appropriate) should be gender responsive.

Ethics Ensure social, economic, political and logistical factors affecting access of women and other vulnerable groups are identified and addressed in the evaluations Ensure required data is collected Address emerging ethical issues relating to mobile phones and big data Require ethical guidelines for research on survivors of gender-based violence

Consultation and dissemination Ensure all stakeholders including women and vulnerable groups are consulted on the planning and design of evaluations and have access to evaluation reports Use of evaluation findings Overall use of evaluation findings is often weak but use of EFGR findings can be particularly problematic Develop guidelines to ensure effective dissemination and use of EFGR evaluations Often special dissemination mechanisms will be required User friendly reports Non-conventional dissemination (e.g. through theater and dance)

3. Identifying stakeholders concerned with gender and equality Ensure that stakeholders concerned with gender and equality are identified and actively involved Broaden the range of stakeholders to ensure that civil society, community organizations and academics are involved

4. Developing an EFGR national evaluation system Linking the evaluation policy to national gender policy or strategy Highlighting international agreements and conventions on gender and equality to which the country is a signatory Integrating gender and equality into national development strategies Ensuring gender and equality indicators are built into the national development strategies Integrating a gender and equality focus into poverty programs and poverty analysis Building gender and equality into social accountability strategies Incorporating mechanisms for citizen feedback

5. Developing an advocacy strategy to promote EFGR evaluation use Typical questions in the design of an advocacy strategy What do we want to achieve? (goals) Who can give it to us? (audiences, power-holders, opinion leaders Who can deliver the information most effectively? (messengers) What is the most effective way to reach target audiences? (delivery) What resources do we have? (resources, strengths) What do we need to develop? (challenges, gaps) How do we begin? (first steps) How do we know if it is working? (M&E)

An advocacy strategy must identify Stakeholders with direct responsibility for policy formulation The mechanisms through which they can be influenced Carrots, sticks and sermons Stakeholders who can influence NEPs The types of influence they can exert Funding Technical support International standards Advocacy

Stakeholders with direct responsibility Types of influence An EFGR advocacy strategy: stakeholders with direct policy responsibility and stakeholders with influence Stakeholders with direct responsibility Types of influence Types of influence Stakeholders tht can influence NEPs Presidents Office Lead ministry Parliament Line ministries Research and technical agencies Regional government Carrots Sticks sermons EFGR focused NEPS Funding Technical support International standards Advocacy UN agencies Bilateral agencies Civil society Academia Private sector VOPES

Advocacy tools include Social media Mass media (TV, radio) Lobbying Demonstrations and protests Workshops and conferences Alliance building

“No more violence against women” The campaign in action in El Salvador.

6. Example: developing an advocacy strategy focused on parliamentarians Identifying the possible entry points: Parliamentary leadership Identifying key committees Staff assistants Caucuses Elections: Pending legislation Parliamentary research and documention services

Resources Case studies on National Evaluation Policies http://www.pfde.net/index.php/publications-resources/2014-02-28- 19-08-54 Case studies on EFGR M&E systems http://www.pfde.net/index.php/publications-resources/case-studies- on-ef-gr-m-e-systems Ten reasons why National Evaluation Policy is important http://www.pfde.net/index.php/publications-resources/ten-reasons- publications National Evaluation Policies for sustainable and equitable development http://www.mymande.org/sites/default/files/files/NationalEvaluation Policies_web-single-color(1).pdf