1. Session 5 The new indicators
Rationale and approach for the new indicators
UNDP-OECD Joint Support Team

2. Refining the new indicators
5 new indicators were introduced to reflect broader scope of Busan
Indicator 8 (gender equality) – successfully monitored in 1st round.
Indicators 1,2,3,4 - still under development during 1st round  Methodologies refined during 2015, in consultation
with a broad range of stakeholders.
New indicators: incorporating new dimensions from Busan
We will go into the details of each of these new indicators in the Indicator clinics tomorrow,
but this session aims to give you an introductory overview of these indicators: rationale, why do they matter, how did you develop / refine the methodologies over the past 6 months

3. Indicator 1: Use of country results frameworks

4. Indicator 1 looks at the extent to which "development co-operation focuses on results that meet country's priorities"
Why is this Busan commitment important?
Overwhelming evidence shows that country ownership of development co-operation efforts ensures greater effectiveness and sustainability;
Focus on country priorities helps to aid coordination and harmonisation;
Monitoring providers’ behavior  Aims to prevent parallel results frameworks, or creating the need for additional M&E efforts not related to the country’s own priorities.

5. The process of defining & refining indicator 1 underwent several rounds of consultations & country testing
Initial approach piloted in countries
Budget support as a proxy to measure the extent of use of country results frameworks;
However:
Other aid modalities can also rely on country results frameworks;
Some providers emphasize some aid modalities vs others.
Refined approach light tested in additional countries
Assessment of providers’ use of CRFs in two selected sectors, at project design stage and at project reporting stage;
However, sampling issues and heavy process at the country level.
Final revised approach for 2nd Round
Basic assessment of use of CRF and/or other government planning tools by providers in approving new projects, with providers’ inputs;
Focus on all new 2015 projects above US$ 1 million, and look at project objectives, results indicators, M&E systems used.

6. Indicator 1: Main characteristics of the refined approach
Highlights of the current methodological approach
Indicator emphasizes a focus on providers’ behavior (+country context)
Focus on sector level instead of national (operational CRFs often at sector level)
Broad concept of “CRF”  more accurate picture of different country and sector-level realities
Data useful as entry point for country-level dialogue on CRF use & on needs for planning/M&E strengthening.
How is it measured?
Country-sourced data (country-led process):
1a. The degree to which (and the ways in which) providers use Country-led Results Frameworks (CRFs)  project level measurement
1b. Existence and characteristics of CRFs  qualitative assessment

7. For each new dev. project (2015) above US$ 1 million in the country
Indicator 1: Focus on use of Country Results Frameworks, complemented by country context info
1a. How is “use of CRFs” being measured?
1b. How is the “country context” assessed?
Brief qualitative self-assessment;
Complemented by:
Evidence from section 1a;
A quick mapping of existing planning tools.
For each new dev. project (2015) above US$ 1 million in the country
Objectives/Focus
Gov. Sector Plans
Results Indicators
% Sector Planning RFs
Indicator Sources
% Use of Gov. Sources
Final Evaluation
Gov. Participation

8. Indicator 2: CSO enabling environment and development effectiveness

9. Why is it relevant to monitor CSO enabling environment and CSO development effectiveness?
CSOs – key development actors
Promoting rights-based approaches
Shaping dev. policies and partnerships
Implementing dev. programmes and projects (raising funds)
Complementing the action of the states by delivering services to citizens
Busan (building on Accra):
recognition of CSOs as independent dev. actors in their own right
call for:
an EE in which CSOs can maximise their contribution to dev.
CSOs to strengthen their accountability & contribution to DE
2030 Agenda and AAAA
engagement of all stakeholders to achieve SDGs, incl. CSOs
CSO EE not explicitly mentioned, but embedded as a component of Goals 16 and 17
Inclusive follow-up and review mechanism of the SDGs  key role to play for CSOs
SDG 16 on governance, particulaly target 3 on rule of law and equitable access to justice, target 6 adressing effective, accountable and transparent institutions, target 7 calling for inclusive and participatory decision making and target 10 concerned with public access to information and the protection of fundemental freedoms.
The GPEDC proposed indicator 2 to be a part of the SDG indicator framework, which currently lacks indicators to track progress on goals 16 and 17
17.17 encourage and promote effective civil society partnerships // or complementary approach
Ambitious SDG agenda will require all stakeholder to engage
While the 2030 Agenda and AAA make no specific reference to the CSO EE, the concept can be seen embedded Importance of CSOs, as a means and an end, to acheive the SDGs
SDG 16 – peaceful and inclusive societies
SDG partnerships with CSOs
Indicator 2’s focus on shared responsibility, dialogue, learning and improvement corresponds to the multistakejolder approach underlying the SDGs

10. CSO Enabling Environment
What does it refer to?
Political, financial, legal and policy conditions (national and international) within which civil society operates. It can include:
Law, policy and practice respecting freedom of association, the right to operate without state interference, the right to pursue self-defined objectives, and the right to seek and secure funding from national & international sources
Institutionalised, inclusive and transparent multi-stakeholder dialogue fora
Effective support from development providers to empower CSOs
What is the state of play? (CPDE, 2015)
Progress:
basic rights inscribed in constitutions
laws enabling access to public information
creation of institutionalised spaces for dialogue
Challenges:
Complex registration processes
limitations on foreign funding access
restricted right to work on certain topics
shrinking space for CSO advocacy
The political, financial, legal and policy conditions within which civil society works, at the country and international levels. It can include:
Law, policy and practice that respect freedom of association
The right to operate free from unwarranted state intrusion or interference
The right to pursue a broad range of self-defined objectives
The right to seek and secure funding from national and international sources
The respect of other basic rights (freedom of peaceful assembly, of expression)
Institutionalised, inclusive and transparent multi-stakeholder dialogue fora for the participation of CSOs in determining, implementing and monitoring development policies and programmes
Effective support from development providers to empower CSOs
international levels
e.g. the UN Human Rights’ resolution on “Civil society space” which urges “States to create and maintain, in law and practice, a safe and enabling environment in which civil society can operate free from hindrance and insecurity”
Source: CPDE country reports (2015)

11. CSO development effectiveness
What does it refer to?
Istanbul Principles – CSOs are effective as development actors if they:
Respect and promote human rights and social justice
Embody gender equality and equity while promoting women’s and girls’ rights
Focus on people’s empowerment, democratic ownership and participation
Promote environmental sustainability
Practice transparency and accountability
Pursue equitable partnerships and solidarity
Create and share knowledge and commit to mutual learning
Commit to realising positive sustainable change
What is the state of play? (CPDE, 2015)
Progress
e.g. Cambodian CSOs adopting a code of conduct in the spirit of Istanbul Principles
Challenges
lack of timely access to information
capacity issues
misappropriation and inadequacy of funds
minimal collaborations between CSOs

12. How was the indicator developed?
Since Busan, efforts to build a meaningful indicator:
Step 1. First monitoring round – lessons learned ( )
CIVICUS originally envisaged
Step 2. Refining the indicator (Dec July 2015)
Close collaboration with the Task Team on CSO DE and EE (incl. CPDE)
Consultation: informal working group, light country testing, broad consultation
Step 3. Second monitoring round ( )
Further strengthening the indicator, on the basis of feedback from round 2 for subsequent rounds:

13. Key features of the indicator
Key objectives: sparking dialogue around CSO EE and DE, building a multi-stakholder assessment, identifying progress made and room for improvement
Grounding the indicator at the country level – contextual approach
Holding each stakeholder group accountable (governments, providers and CSOs), through a questionnaire:
qualitative assessment (primary data)
15 questions structured around 4 modules :
Space for multi-stakeholder dialogue on national policies;
CSO dev. effectiveness;
Official dev. cooperation with CSOs;
Legal and regulatory framework.
Building on the work of the CPDE, Task Team, Istanbul Principles, OECD/DAC 12 lessons for partnering with CSOs
Government-led exercise, through a multi-stakeholder process for data collection & validation.
This four module framework is unique in that it looks not only at what governments and development cooperation providers can do to enhance the CSO EE, but also encourages examination of what CSOs can do to strengthen their own accountability and improve their effectiveness
This is in line with the multi-stakeholder approach underlying the SDGs
In recognising that responsibility for a CSO EE is shared across stakeholders, indicator 2 aims to bring different stakeholders around the table and to foster progress
The methodology is also unique, un that it involves a questionnaire to be filled out at the country level in a multistakeholder fashion, involving governments, CSOs, providers
The joint exercise is intended to share information and spark dialogue amongst stakeholders on opportunities in creating an EE for CSOs in a particular country context. This contextual approach is in line with the 2030 Agenda that recognises that there are different approaches, visions, models and tools available to each country.

14. Indicator 3: Quality of public private dialogue

15. Why is it relevant to monitor the quality of public- private dialogue?
The private sector – key development actors
advancing innovation
creating wealth, income and jobs
mobilising domestic resources
in turn contributing to poverty reduction
Busan
The for-profit private sector is a central driver of dev.
Importance of inclusive dialogue
Call for the participation of the private sector in the design and implementation of dev. policies & strategies to foster sustainable growth and poverty reduction.
2030 Agenda and AAAA
Vital role of the private sector to finance the 2030 Development Agenda
Increasing role of ODA as a catalyzer of private finance
SDG 17 – effective partnerships with the private sector
Indicator 3 can inform the SDG monitoring framework, particularly SDG 17
SDG 16 on governance, particulaly target 3 on rule of law and equitable access to justice, target 6 adressing effective, accountable and transparent institutions, target 7 calling for inclusive and participatory decision making and target 10 concerned with public access to information and the protection of fundemental freedoms.
The GPEDC proposed indicator 2 to be a part of the SDG indicator framework, which currently lacks indicators to track progress on goals 16 and 17
17.17 encourage and promote effective civil society partnerships // or complementary approach
Ambitious SDG agenda will require all stakeholder to engage
While the 2030 Agenda and AAA make no specific reference to the CSO EE, the concept can be seen embedded Importance of CSOs, as a means and an end, to acheive the SDGs
SDG 16 – peaceful and inclusive societies
SDG partnerships with CSOs
Indicator 2’s focus on shared responsibility, dialogue, learning and improvement corresponds to the multistakejolder approach underlying the SDGs

16. Public-Private Dialogue
What does it refer to?
Structured interaction between the public and the private sector to:
Promote right conditions for PSD
Ensure more inclusive and sustainable policy reforms
Example:
The Philippines’ National Competitiveness Council (NCC)
“Promoting a more competitive Philippines and instill a culture of excellence, through PP sector collaboration as means to reduce poverty through inclusive growth.”
The political, financial, legal and policy conditions within which civil society works, at the country and international levels. It can include:
Law, policy and practice that respect freedom of association
The right to operate free from unwarranted state intrusion or interference
The right to pursue a broad range of self-defined objectives
The right to seek and secure funding from national and international sources
The respect of other basic rights (freedom of peaceful assembly, of expression)
Institutionalised, inclusive and transparent multi-stakeholder dialogue fora for the participation of CSOs in determining, implementing and monitoring development policies and programmes
Effective support from development providers to empower CSOs
international levels
e.g. the UN Human Rights’ resolution on “Civil society space” which urges “States to create and maintain, in law and practice, a safe and enabling environment in which civil society can operate free from hindrance and insecurity”
Source: CPDE country reports (2015)

17. How was the indicator developed?
Since Busan, efforts to build a meaningful indicator:
Step 1. First monitoring round ( )
Step 2. Refining the indicator (Nov June 2015)
Close collaboration with the World Bank
Piloting in 3 countries
Consultation: PPD international workshop in Copenhagen, discussions with private sector representatives of the GPEDC’ SC
Step 3. Second monitoring round ( )
Further strengthening the indicator, on the basis of feedback from round 2 for subsequent rounds

18. Key features of the indicator
Focus on the quality of PPD (proxy to capture private sector engagement in improving public policies)
Key objectives: sparking multi-stakeholder dialogue, building a joint assessment of the state of quality of PPD, incentivising behaviour change
Rather than a single indicator, a PPD country profile, combining global data (i.e. module 1) and country-sourced data (i.e. module 2&3)
Overview of the country-level context for PPD and assessment of a given dialogue platform, through 3 modules:
Legal and regulatory context for PPD
Existing quantitative indices – compiled by the JST
Country’s readiness to host, create or sustain a dialogue process
Country-level qualitative questionnaire / assigned score
Government-led exercise, through a multi-stakeholder process for data collection & validation
Effectiveness of a given platform
In-depth assessment on the quality of a selected PPD Platform (optional, commissioned by the Government)
This four module framework is unique in that it looks not only at what governments and development cooperation providers can do to enhance the CSO EE, but also encourages examination of what CSOs can do to strengthen their own accountability and improve their effectiveness
This is in line with the multi-stakeholder approach underlying the SDGs
In recognising that responsibility for a CSO EE is shared across stakeholders, indicator 2 aims to bring different stakeholders around the table and to foster progress
The methodology is also unique, un that it involves a questionnaire to be filled out at the country level in a multistakeholder fashion, involving governments, CSOs, providers
The joint exercise is intended to share information and spark dialogue amongst stakeholders on opportunities in creating an EE for CSOs in a particular country context. This contextual approach is in line with the 2030 Agenda that recognises that there are different approaches, visions, models and tools available to each country.

19. Indicator 4: Transparency

20. Transparency Dimensions
Indicator 4: Transparency Indicator
What is being measured?
Degree of transparency of development co-operation flows and information
How is it being measured?
3 dimensions in the common approach, as agreed in Busan:
Classifies providers within broad categories, depending on degree of progress in implementing the common approach.
What global sources of information? +
Transparency Dimensions
Timely
Comprehensive
Forward-looking
The preliminary proposed methodology is based on the refined methodological approach for assessing the CRS and FSS data proposed by the DCD Secretariat, and a proposed refined methodological approach for assessing the IATI data proposed by the IATI Secretariat (to be approved in the upcoming IATI SC).
Taking note of the recommendations from the Monitoring Advisory Group, this slide highlights key proposed elements of the composite approaches, which requires further sufficient consultation with WP-STAT, IATI SC, and GPEDC stakeholders – internalizing the implications of this proposed approach.
JST proposes to use overall composite scales on aggregated scores, presented for CRS/FSS and IATI separately. Scales to include: excellent, good, fair and major improvement needed. This scale approach seems to be reasonable. BUT – strong preference has been raised not to combine the scores for CRS and FSS together. IATI however sees where possible, combined scores for CRS and FSS will be consistent with the three component approach of the common standard. JST proposes to combine CRS/FSS where applicable (providers reporting to both CRS and FSS), but present separate scores/scales for those providers reporting only to CRS, or to FSS).
Acknowledging the inherently different natures of information captured by the common standard reporting systems means that key Busan dimensions of the common standard are interpreted in significantly different ways. Therefore, different dimensions form the basis of a composite approach. For example, timeliness of CRS data is interpreted as timeliness of submission of complete DAC/CRS data in accordance to the CRS reporting cycle, while IATI timeliness will assess frequency of reporting and freshness of information.
Similarly, CRS/FSS proposes to add accuracy dimension as part of the assessment for this indicator (meaning their composite scores/scales will be based on timeliness, comprehensiveness, and accuracy of data), while IATI proposes to work on a joint reliable methodology to assess the accurate content of the information in post March 2016.
Results are proposed to be presented based on three categories of countries (Group A – providers reporting to both CRS/FSS and IATI, Group B – providers reporting only to CRS/FSS, and Group C – providers reporting only to IATI). This approach seems to be feasible.
Data for assessment – JST proposes to assess the latest data and assessment available from CRS/FSS and IATI. This means CRS data will be based on 2014, IATI based on 2015 (the latest published to IATI at the time of assessment i.e. March 2016). FSS assessment will cover CPA , IATI’s forward looking information will look at
Assessment process – composite scales drawing on assessment of each dimension by Secretariats – this means methodology for assessing each component/dimension is to be approved by relevant constituencies of the system. JST assessment will be based on scores calculated by each secretariat submitted to JST – JST producing aggregate scales as per an agreed methodology, and presented with sufficient detailed information.
Different nature – inevitable that different dimensions forms the basis. However, to present a coherent outcome will mean succinct analysis of what this scales/scores mean for information that is for accountability purpose, and for management purpose.
Proposed timeline for stakeholder consultation:
WP-STAT Nov 2015; IATI SC Dec 2015
Use of composite scales to provide information on the status of implementation the common standard (excellent, good, fair, improvement needed).
Drawing on assessment of information reported to CRS/FSS and AIT against the three components (timeliness, comprehensiveness and forward looking). For CRS/FSS, accuracy component proposed to be added.
Composite scores for each dimension for each system, aggregated by system based on the own methodology by Secretariats supporting the system
Different dimensions assessed for each component.
Presentation of composite scales by each provider, grouped under the three categories (Group A-CRS/FSS and IATI) (Group B – CRS/FSS) (Group C – IATI)
Provider profile – composite scales with associating composite scores

21. Indicator 4: Transparency Indicator
What is the current challenge with this indicator?
Instead of converging towards the common standard, OECD & IATI methodologies are starting to diverge (e.g. type scoring, indicator composition, reporting, year).
Proposed GPEDC approach for consultation
Proposal reflecting technical consensus.
Ongoing consultation (Sept-Dec): IATI, OECD Working Party-Statistics, GPEDC stakeholders.
GPEDC Steering Committee endorsement (in early 2016 meeting or virtual)
Data becomes available in Dec 2015  included in 2nd Monitoring Round
The preliminary proposed methodology is based on the refined methodological approach for assessing the CRS and FSS data proposed by the DCD Secretariat, and a proposed refined methodological approach for assessing the IATI data proposed by the IATI Secretariat (to be approved in the upcoming IATI SC).
Taking note of the recommendations from the Monitoring Advisory Group, this slide highlights key proposed elements of the composite approaches, which requires further sufficient consultation with WP-STAT, IATI SC, and GPEDC stakeholders – internalizing the implications of this proposed approach.
JST proposes to use overall composite scales on aggregated scores, presented for CRS/FSS and IATI separately. Scales to include: excellent, good, fair and major improvement needed. This scale approach seems to be reasonable. BUT – strong preference has been raised not to combine the scores for CRS and FSS together. IATI however sees where possible, combined scores for CRS and FSS will be consistent with the three component approach of the common standard. JST proposes to combine CRS/FSS where applicable (providers reporting to both CRS and FSS), but present separate scores/scales for those providers reporting only to CRS, or to FSS).
Acknowledging the inherently different natures of information captured by the common standard reporting systems means that key Busan dimensions of the common standard are interpreted in significantly different ways. Therefore, different dimensions form the basis of a composite approach. For example, timeliness of CRS data is interpreted as timeliness of submission of complete DAC/CRS data in accordance to the CRS reporting cycle, while IATI timeliness will assess frequency of reporting and freshness of information.
Similarly, CRS/FSS proposes to add accuracy dimension as part of the assessment for this indicator (meaning their composite scores/scales will be based on timeliness, comprehensiveness, and accuracy of data), while IATI proposes to work on a joint reliable methodology to assess the accurate content of the information in post March 2016.
Results are proposed to be presented based on three categories of countries (Group A – providers reporting to both CRS/FSS and IATI, Group B – providers reporting only to CRS/FSS, and Group C – providers reporting only to IATI). This approach seems to be feasible.
Data for assessment – JST proposes to assess the latest data and assessment available from CRS/FSS and IATI. This means CRS data will be based on 2014, IATI based on 2015 (the latest published to IATI at the time of assessment i.e. March 2016). FSS assessment will cover CPA , IATI’s forward looking information will look at
Assessment process – composite scales drawing on assessment of each dimension by Secretariats – this means methodology for assessing each component/dimension is to be approved by relevant constituencies of the system. JST assessment will be based on scores calculated by each secretariat submitted to JST – JST producing aggregate scales as per an agreed methodology, and presented with sufficient detailed information.
Different nature – inevitable that different dimensions forms the basis. However, to present a coherent outcome will mean succinct analysis of what this scales/scores mean for information that is for accountability purpose, and for management purpose.
Proposed timeline for stakeholder consultation:
WP-STAT Nov 2015; IATI SC Dec 2015
Use of composite scales to provide information on the status of implementation the common standard (excellent, good, fair, improvement needed).
Drawing on assessment of information reported to CRS/FSS and AIT against the three components (timeliness, comprehensiveness and forward looking). For CRS/FSS, accuracy component proposed to be added.
Composite scores for each dimension for each system, aggregated by system based on the own methodology by Secretariats supporting the system
Different dimensions assessed for each component.
Presentation of composite scales by each provider, grouped under the three categories (Group A-CRS/FSS and IATI) (Group B – CRS/FSS) (Group C – IATI)
Provider profile – composite scales with associating composite scores

22. তোমাকে ধন্যবাদ Gracias Thank you Dankjewel Hvala Merci Asante مننه
ありがとう
Gracias
Thank you
Dankjewel
Hvala
Merci
Asante
مننه
شكرا
Obrigado
Salamat