1. Module 13 Integrating EFGR evaluations into the SDG country-level review
Technical Assistance on Evaluating SDGs:
Leave No One Behind
EvalGender+ Network together with UN Women, UNEG and EvalPartners

2. Presentation developed by Michael Bamberger and Asela Kalugampitiya based on Chapter 5 of ”Evaluating the Sustainable Development Goals within a “No-one left behind” lens through equity-focused and gender-responsive evaluations”

3. Outline What is a national evaluation policy (NEP)?
Integrating an EFGR lens into SDG national evaluation policies/systems
Identifying and working with stakeholders
Developing an EFGR evaluation system
Developing an advocacy strategy
Example of an advocacy campaign focused on parliamentarians

4. 1. What is a National Evaluation Policy?
National Evaluation Policies (NEPs) refer to formal policies defining the purpose, responsibilities, functions and organization of the public sector evaluation function in a particular country.
(Bamberger, Segone and Reddy 2014)

5. Global Mapping on NEPs
First study in 2013 and update in mapping-report-2015
109 countries were studied
Only 20 countries have formal National Evaluation Policies
Only 3 countries have reference to gender equality: Nepal, Kenya and Zimbabwe. Afghanistan draft results based M&E policy includes equity and gender (section 3.3)
Documented six case studies on NEP with selected countries
Documented four case studies on EFGR M&E systems with selected countries

6. Ways of incorporating evaluation at national level
Evaluation is included in the constitution: Morocco and Nepal
National Evaluation Policy endorsed by the government: Canada, Japan, Uganda, Malaysia, Colombia, Zimbabwe
National Evaluation Policy Framework endorsed by the government: South Africa
National Evaluation Policy Act approved by the parliament: Nepal (draft, awaiting approval)
Sector wise evaluation policies approved by designated ministries: eg. Evaluation policy for HIV/AIDS programmes

7. Building gender into the NEP- Different scenarios
Scenario 1: Gender equality is fully integrated in to the NEP itself. Eg. Afghanistan draft Results Based M&E Policy – sec 3.3, Zimbabwe NEP.
Scenario 2: Countries where the NEPs integrate gender equality thanks to a national gender policy. Eg. Ethiopia
Scenario 3: Countries where the national development strategy integrates gender equality. Eg. Nepal
Scenario 4: Countries where the NEP does not include gender equality, but where gender-responsive evaluations are regularly conducted. In some countries gender equality is considered important by government, but the NEP is presented on a high level of generality which does not address sector, and cross-sector specific issues. Eg. Mexico

8. 2. Integrating an EFGR lens into SDG evaluation policies and systems
Challenges
Political challenges
The more in-depth analysis of gender and equity issues can reduce the rating of country programs on indices such as the Human Development Index
EFGR analysis often shows equity gaps have increased or some groups are left behind
EFGR focuses on socially marginalized groups that society may prefer to ignore
Laws sometimes forbid collecting data on some vulnerable groups (e.g. ethnic minorities)

9. Integrating an EFGR lens into SDG evaluation policies and systems
B. Methodological challenges
Requires disaggregation of data – difficult and expensive
Difficult to study subtle processes of social exclusion
Social exclusion analysis may require the integration of different data sets which can be difficult – resistance from some agencies
Difficult to obtain precise measures of income and wealth

10. Reasons why many national evaluations do not address gender
In 2014 only 14 developing countries had a national evaluation policy (NEP) – probably required to facilitate complex and ambitious EFGR evaluations
Reasons why gender often not incorporated into the NEP
Many NEPs only provide high level framework
Many NEPs do not address cross-cutting themes such as gender
Often gender focus limited to monitoring women’s access to services or representation on community committees
Many stakeholders do not consider gender a relevant issue for national policy evaluation

11. Lessons from the development agencies on the incorporate of gender equality into evaluations
Gender equality must be considered a central development objective in its own right, as well as essential to achieving other priorities and sectoral objectives
There is a need for a string organizational commitment with direct senior management responsibility
The should be a clearly defined rational for a gender-responsive development strategy. Different agencies prioritize different issues (for example a business case or a human rights focus) and NEP should clarify which rationale, or set of rationales are prioritized.
The importance of guidelines, checklists and practical example of gender- responsive evaluation
Importance of alliance building and inter-agency coordination
Recognizing that gender interventions are frequently broad-based and complex and that gender policies and evaluations must recognize this complexity.

12. Reasons many national evaluations do not address equality issues
Political resistance to addressing equality
Reluctant to challenge their positive image of development progress on conventional indices
An equity focus is technically more challenging
Increased cost of data collection and analysis
Many indicators such as ethnicity, religion of physically disability are not included in national statistics so secondary data not available
Many voulnerable groups are largely invisible and difficult to study

13. Integrating an EFGR focus into the country-led evaluation strategy
Define responsibilities:
often no agency is responsible for ensuring gender and equity are addressed in the evaluations
Coverage and selection of evaluations:
Evaluations must cover many issues so gender and equality can easily be ignored
Develop guidelines to ensure gender and equality are always included where appropriate

14. Integrating equality and equity into national evaluation policies and systems……
Incorporate equality into the NEP guiding principles for the selection, design and use of evaluations.
Coverage/selection of evaluations. Ensure that equality issues receive an appropriate priority in the criteria used to select the evaluations to include in the annual NEP evaluation program.
Methodological techniques. Methodological guidelines should be developed for conducting equality-focused evaluations.

15. Elements of a gender-responsive evaluation system can include:
National evaluation steering committees and similar bodies should include members with gender expertise.
National evaluation offices should recruit staff with gender expertise and ideally with specific experience in gender-responsive evaluation.
All staff in the national evaluation office and related bodies should attend training on gender responsive evaluation
Gender ministries or other gender focal points and gender advocates should be consulted when developing gender plans and policies and terms of reference.

16. Elements of a gender-responsive evaluation system can include…….
Government bodies should draw on the gender expertise of VOPEs.
Gender-balanced national evaluation systems are encouraged.
Parliaments and other government budget making bodies should provide adequate budgets for collection and analysis of sex- disaggregated data and gender-responsive monitoring.
Donor and UN commissioned evaluations should include guidelines to ensure all evaluations (where appropriate) should be gender responsive.

17. Ethics
Ensure social, economic, political and logistical factors affecting access of women and other vulnerable groups are identified and addressed in the evaluations
Ensure required data is collected
Address emerging ethical issues relating to mobile phones and big data
Require ethical guidelines for research on survivors of gender-based violence

18. Consultation and dissemination
Ensure all stakeholders including women and vulnerable groups are consulted on the planning and design of evaluations and have access to evaluation reports
Use of evaluation findings
Overall use of evaluation findings is often weak but use of EFGR findings can be particularly problematic
Develop guidelines to ensure effective dissemination and use of EFGR evaluations
Often special dissemination mechanisms will be required
User friendly reports
Non-conventional dissemination (e.g. through theater and dance)

19. 3. Identifying stakeholders concerned with gender and equality
Ensure that stakeholders concerned with gender and equality are identified and actively involved
Broaden the range of stakeholders to ensure that civil society, community organizations and academics are involved

20. 4. Developing an EFGR national evaluation system
Linking the evaluation policy to national gender policy or strategy
Highlighting international agreements and conventions on gender and equality to which the country is a signatory
Integrating gender and equality into national development strategies
Ensuring gender and equality indicators are built into the national development strategies
Integrating a gender and equality focus into poverty programs and poverty analysis
Building gender and equality into social accountability strategies
Incorporating mechanisms for citizen feedback

21. 5. Developing an advocacy strategy to promote EFGR evaluation use
Typical questions in the design of an advocacy strategy
What do we want to achieve? (goals)
Who can give it to us? (audiences, power-holders, opinion leaders
Who can deliver the information most effectively? (messengers)
What is the most effective way to reach target audiences? (delivery)
What resources do we have? (resources, strengths)
What do we need to develop? (challenges, gaps)
How do we begin? (first steps)
How do we know if it is working? (M&E)

22. An advocacy strategy must identify
Stakeholders with direct responsibility for policy formulation
The mechanisms through which they can be influenced
Carrots, sticks and sermons
Stakeholders who can influence NEPs
The types of influence they can exert
Funding
Technical support
International standards
Advocacy

23. Stakeholders with direct responsibility Types of influence
An EFGR advocacy strategy: stakeholders with direct policy responsibility and stakeholders with influence
Stakeholders with direct responsibility
Types of influence
Types of influence
Stakeholders tht can influence NEPs
Presidents Office
Lead ministry
Parliament
Line ministries
Research and technical agencies
Regional government
Carrots
Sticks
sermons
EFGR focused NEPS
Funding
Technical support
International standards
Advocacy
UN agencies
Bilateral agencies
Civil society
Academia
Private sector
VOPES

24. Advocacy tools include
Social media
Mass media (TV, radio)
Lobbying
Demonstrations and protests
Workshops and conferences
Alliance building

25. “No more violence against women” The campaign in action in El Salvador.

26. 6. Example: developing an advocacy strategy focused on parliamentarians
Identifying the possible entry points:
Parliamentary leadership
Identifying key committees
Staff assistants
Caucuses
Elections:
Pending legislation
Parliamentary research and documention services

27. Resources
Case studies on National Evaluation Policies
Case studies on EFGR M&E systems on-ef-gr-m-e-systems
Ten reasons why National Evaluation Policy is important publications
National Evaluation Policies for sustainable and equitable development Policies_web-single-color(1).pdf