2. Developing a National Framework for Monitoring the Iraqi NDP MEASURING GOVERNANCE Marie Laberge Global Programme on Country-Led Governance Assessments UNDP Oslo Governance Centre

3. Examine the limitations of global composite indicators Discuss the rationale for using complementary indicators (input vs. Process vs. outcome – de jure vs. de facto) Identify indicators that are ‘actionable’ Identify 4 ways in which indicators can be made pro-poor and gender sensitive Identify common sources of governance data Types of indicators & data sources

4. Part One International sources of governance indicators

5. 908886 82 84969800020492940678 76 1974 Global composite indicators of governance 08 80 CPIA Freedom in the World Commitment to Development Bertelsmann Transformation Index Global Accountability Report Index of Economic Freedom Journalists killed Open Budget Index Polity International Country Risk Guide Corruption Perceptions Index Opacity Index Integrity Index Human Rights Indicators Bribe Payers Index GAPS in Workers’ Rights BEEPS Press Freedom Survey Political Terror Scale Global Competitiveness Index World Governance Assessment World Values Survey State Failure Dataset Weberian Comparative State Project CIRI Human Rights Databse Women in Parliament Governance Matters Gender Empowerment Measure Index of Democracy World Democracy Audit Countries at the Crossroads Failed States Index Press Freedom Index Democracy Index Institutional Profiles Database Governance and Democracy Processes Global Peace Index

6. Transparency International Freedom HouseWorld Bank Governance Indicators Bertelsmann Foundation Global Integrity (Syria, Saudi Arabia not included) TopJordan Saudi Arabia Kuwait Morocco “Partially free”: Jordan, Morocco, Lebanon, Yemen, Kuwait Kuwait, Jordan, Saudi Arabia, Morocco Lebanon, Kuwait, Algeria Jordan, Kuwait, Egypt, Palestine Bot- tom Lebanon Yemen “Not free”: Algeria, Egypt, Saudi Arabia, Syria Palestine Syria Yemen Palestine Syria Saudi Arabia Morocco Algeria Yemen Lebanon

7. 1) What is each index measuring?  TI CPI  Public sector only  WB ‘Control of Corruption’ indicator  public & private sectors  Global Integrity Index  anti-corruption mechanisms 2) What types of indicators make up each index?  High-level corruption vs. petty corruption  Frequency of bribes vs. size of bribes How to explain these discrepancies in rankings?

8. Selected sources (out of 13 in 2009) Subject asked (based on perceptions) RespondentsCoverage World Economic Forum (WEF) Undocumented extra payments or bribes connected with various government functions Senior business leaders; domestic and international companies 131 countries Bertelsmann Transformation Index (BTI) The government’s capacity to punish and contain corruption Network of local experts / local business people 125 countries Economist Intelligence Unit (EIU) The misuse of public office for private (or political party) gain International experts170 countries 3) What data sources are used?

9. And… 4) Are data sources the same every year?

10. TI Corruption Perception index (CPI) 2009126 th Syria5 surveys 176 th Iraq3 surveys 179 th Afghanistan4 surveys 2008147 th Syria5 surveys 176 th Afghanistan4 surveys 178 th Iraq4 surveys 2007138 th Syria4 surveys 172thAfghanistan4 surveys 178 th Iraq4 surveys

11. Part two National governance indicators

12. Monitoring the input, process and outcome sides of governance reforms

13. Outcome indicators will tell us: What do we want to achieve? Have we achieved it? Process indicators will tell us: How are inputs used to generate outcomes? What are the ‘best’ processes? When & how do processes need adjustment? Input indicators will tell us: What is needed to create these processes? Whether it is available? Three levels of indicators

14. De jure (input) indicator: In law, is there an agency with a legal mandate to address corruption? Does it receive regular funding, a professional & full-time staff? Measuring a change in law, in procedures, in resources De facto (process / outcome) indicator: In practice, is the anti-corruption agency effective? Measuring improved governance in practice (how are citizens benefiting from this new institution?) Complementarity in the use of indicators

15. Combining indicators to show discrepancies  De jure indicator: In law, is there an agency with a legal mandate to address corruption? (input)  De facto indicators: In practice, is the anti-corruption agency effective? (outcome) When necessary, is the ACA able to independently initiate investigations? (process) Does the ACA make regular public reports (e.g. to the parliament)? (process) Can citizens complain to the ACA without fear of recrimination? (e.g. whistle-blower mechanisms) (process) Does the ACA act on citizen complaints within a reasonable time period? (process)

16. Outcome indicators are useful to assess progress towards the desired objectives of governance programmes (new laws / increased expenditures mean nothing in and of themselves) But they lack ‘actionability’ (they don’t tell us what needs to be fixed / points of interventions) Example: A government cannot ‘choose’ to lower a crime rate (an outcome indicator) It can, however, choose to put more police on the streets (input) or toughen penalties for offenders (prosecution process) Input / process indicators are more ‘actionable’

17. Outcome indicators (non-actionable) Input / process indicators (actionable) % citizens who believe there is a lack of transparency in the operations of local governments  Number of provinces which have a public forum for citizens to discuss with locally elected officials  Number of provinces which formally publish contracts, tenders, local budget and local development plan % citizens who do not believe that legal protection is ensured equally to all citizens regardless of their material status, ethnic, religious affiliations, political/party affiliations  Number of provinces where an awareness-raising programme on citizens’ rights to seek legal protection has been conducted  % citizens who say they know where to / how to seek legal protection Actionable indicators: Experience from Morocco

18. “Not everything that counts can be counted, and not everything that can be counted counts.” – Albert Einstein A word of caution:

19. “Not everything that counts can be counted, and not everything that can be counted counts.” – Albert Einstein Measuring is not an end in itself, but rather a means to an end (actual governance reforms). So what we measure should help us implement reforms – if not, why measure it? A word of caution:

20. Risk of measuring things because they are easily measurable, leading to ‘reform illusion’ Example: Measuring the number of corruption cases brought to trial (as an indicator of the efficiency of the judicial system in combating corruption): Does an increase in this indicator mean an increased level of confidence in the reporting mechanism, and in the courts? Or rather, does it indicate a higher incidence of corruption? Or both...?  In other words, are you really measuring what you intended to measure? A word of caution

21. Balanced baskets of indicators Do you have a balanced basket of indicators that measures progress towards a single aim? Is the ambiguity inherent in each indicator reduced by the presence of the others?

22. Balanced baskets of indicators –What is the aim you want to achieve? –What are the many reasons why your aim might NOT be achieved? –For each “reason”  develop one indicator

23. Unbalanced baskets of indicators Aim: “Equal access to justice” Indicator 1: Number of new courts opened in rural and urban areas Indicator 2: Number of courts per 100,000 residents Indicator 3: % citizens who say that they have access to court systems to resolve disputes Indicator 4: % accused not represented at trial

24. Indicator 1: Number of new courts opened in rural and urban areas Measures the activity of a court-building programme, but does not tell us the result of the programme: do people have their fair share of courts? Indicator 3: % citizens who say that they have access to court systems to resolve disputes Tells us nothing about equality Indicator 2: Number of courts per 100,000 residents Tells us nothing about the equality of the distribution of courts (serving mainly one group / one region?) Indicator 4: % accused not represented at trial The basket as a whole is unbalanced: 3 indicators relate to the courts, and one to legal aid

25. Balanced baskets of indicators Checklist for building a well-balanced basket of indicators Avoid narrow activity indicators What is the strategic aim you want to achieve? Draw one indicator from each institution/service that contributes to the overall aim Or... What are the many reasons why your aim might NOT be achieved? For each “reason”  develop one indicator

26. Balanced baskets of indicators 4 key services/institutions for achieving “equal access to justice” 4 indicators: 1. Court system 2. Legal aid service 3. Police service 4. Prosecution service

27. Indicator 1 – Court system % citizens who say that they have access to court systems, disaggregated by gender, ethnicity, region, etc. Adapted from version 1: Disaggregated to reveal issues of inequality Indicator 3 – Police service % citizens who say that the police will respond to them without requiring a bribe if called to resolve a dispute, disaggregated by gender, ethnicity, region, etc. Adds balance: Will alert you to problems in police services that may block access to justice more for some groups than for other. Indicator 2 – Legal aid service % accused persons legally represented at one or more court appearances, disaggregated by gender, ethnicity, region, etc. Adapted from version 1: Disaggregated to reveal issues of inequality Indicator 4 – Prosecution service Ratio of prosecution caseloads in courts serving wealthier communities to those in courts serving marginalized communities Adds balance: Will alert you to problems in prosecution services that may block access to justice more for some groups than for other.

28. How to make governance indicators ‘pro-poor’ and gender sensitive

29. What makes a governance indicator ‘pro-poor’ or gender sensitive? 4 ways to make indicators sensitive to vulnerable groups: 1.Disaggregating by poverty/gender 2.Specific to the poor/women 3.Implicitly poverty/gender sensitive 4.Chosen by the poor/women

30. 1. Disaggregating by poverty/gender (Parliament)  Information is collected for the general population, then disaggregated by sex or income % seats held by women in parliament Level of confidence among female citizens that the Parliament represents their interests % of Parliamentarians from poorer districts that have functioning and accessible local offices to meet with constituents

31. 2. Specific to the poor/women  Measuring governance issues that are specific to the poor or to women Size of funds allocated to legal aid in provincial budgets (per capita) Number of attorneys as % of citizens in need of one % of local governments practicing gender-sensitive budgeting

32. 3. Implicitly poverty/gender sensitive Backlog of small cases of little financial value  Makes no explicit reference to poverty status or gender, but by its nature, clear that the indicator is of particular relevance to low-income groups / women The number of hours that polling booths are open during on election day Frequency of engagement of CSOs in consultations on the legislation-making process

33. 4. Chosen by the poor/women  Identified & measured by using participatory techniques (surveys, focus groups, etc.) Acceptance of documentation other than birth certificates in the process of voter registration Women’s trust in the police and its ability to provide women with redress if they file a complaint

34. Sources of governance data

35. Important to monitor both the ‘supply’ and ‘demand’ for good governance Parallel with private sector: –Assessment of the company’s charter, balance sheet, internal processes and assembly line OR / AND ? –Assessment of whether consumers are actually buying their product, whether they were satisfied with it, and likely to keep on buying it Complementarity of data sources

36. Implications for data sources? 1) Objective data – Qualitative: Administrative sources: Policy and legal documents, codes of conduct, organizational set-up and management systems, processes for decision-making Other narrative reports: Newspaper reports, reports by CSOs 2) Objective data – Quantitative: Government statistics, data gathered by domestic NGOs, international organizations and academics, expenditure tracking and budgetary information, audit reports, court records 3) Opinion and experience-based data from citizens, specific groups (MPs), specific institutions, or private sector through surveys, focus groups, etc.

37. Assessing the ‘supply side’ of governance Assessing the ‘demand side’ of governance IHEC has a permanent voter registry with safeguards to prevent fraud and mechanisms for inclusion of all Iraqis in elections  -% of people prevented from registering (experience-based)  -% of people prevented from voting because name not on voter list (experience-based)  -% of people prevented from voting due to intimidation (experience- based)  -% of people who say results accurate reflection of vote  -% of people who feel elections produce legislature representative of people  -% of people who feel elections enable them to get rid of bad leaders

38. Assessing the ‘supply side’ of governance Assessing the ‘demand side’ of governance Number of criminal codes and criminal procedure codes harmonized with international Human Rights Law  % who say ordinary people can commit crimes without punishment  % who say high level officials can commit crimes without punishment  % who say people fear wrongful arrest

39. Opportunity to draw from ‘fresh’ survey data New ‘Arab Democracy Barometer’  Joint initiative by Arab Reform Initiative & Arab Barometer  Nationally representative public survey  In 10 countries – including Iraq  Data to be collected by country teams (starting autumn 2010)  75 questions

40. The Arab Democracy Index First governance index produced by an Arab institution (“The Arab Reform Initiative”) Data collected by consortium of Arab research centres & universities Covers 10 countries 40 indicators  Some measure the ‘means’ of democratic transition (legislation) – ‘de jure’ indicators  Some measure the ‘results’ of democratic transition – ‘de facto’ indicators Sources of data:  Government & non-government  Citizen’s impressions (public survey) used for 25% indicators 4 themes: 1)Strong & accountable public institutions 2)Rule of law 3)Respect for rights & freedoms 4)Equality & social justice