1 Actionable Governance Indicators: Progress and Prospects DEC Seminar on Measuring Governance May 1-2, 2008

2 Purposes of broadly-defined indicators Research on development impact of civil liberties, corruption, rule of law, etc. Research on development impact of civil liberties, corruption, rule of law, etc. Research on determinants/correlates of good governance Research on determinants/correlates of good governance Tool for reformers in government Tool for reformers in government Country-level diagnosis: more sensitive to random measurement error Country-level diagnosis: more sensitive to random measurement error

3 Purposes of narrowly-defined indicators Monitoring progress of public sector reform programs: are reforms being implemented? (e.g. pay de-compression) Monitoring progress of public sector reform programs: are reforms being implemented? (e.g. pay de-compression) Measuring results: are intermediate-level outcomes improving? (e.g. number of qualified applicants for high-skill positions increasing) Measuring results: are intermediate-level outcomes improving? (e.g. number of qualified applicants for high-skill positions increasing) Research on impacts of specific reforms on performance of governance sub-systems (budget management, civil service, tax administration) Research on impacts of specific reforms on performance of governance sub-systems (budget management, civil service, tax administration) Input into actionable parts of CPIA criteria Input into actionable parts of CPIA criteria

4 Actionable Governance Indicators are the Missing Middle Inputs and outputs Bank operations consistently track project inputs and outputs Bank operations consistently track project inputs and outputs A growing number of Bank operations and client countries have begun to track program and policy outputs A growing number of Bank operations and client countries have begun to track program and policy outputs Outcomes Broad governance outcomes are measured by Freedom House, ICRG, etc. and by TI, WGI aggregate indexes Broad governance outcomes are measured by Freedom House, ICRG, etc. and by TI, WGI aggregate indexes The Missing Middle: Direct impacts of institutional reform efforts on how particular governance sub- sub-systems function: institutional arrangements, procedures, and practices Direct impacts of institutional reform efforts on how particular governance sub- sub-systems function: institutional arrangements, procedures, and practices Examples include PEFA, Doing Business, Global Integrity, Open Budget Index, Statistical Capacity Index, OECD procurement index Examples include PEFA, Doing Business, Global Integrity, Open Budget Index, Statistical Capacity Index, OECD procurement index

5 "Effective implementation, including further development of actionable and disaggregated indicators, will now be critical to achieving the GAC strategy's desired results." Development Committee Communiqué, Washington DC, 15 April, 2007

6 Actionability implies greater clarity regarding the steps governments can take to improve their score on an indicator implies greater clarity regarding the steps governments can take to improve their score on an indicator is reduced to the degree extraneous factors can affect indicator scores is reduced to the degree extraneous factors can affect indicator scores in general requires that indicators have narrow and explicit definitions in general requires that indicators have narrow and explicit definitions does not preclude subjective or perceptions-based indicators (if they have explicit and narrow definitions) does not preclude subjective or perceptions-based indicators (if they have explicit and narrow definitions)

7 Specific indicators enhance: Actionability: clear link between actions and indicator values Actionability: clear link between actions and indicator values Replicability: independent informed observers should arrive at same score Replicability: independent informed observers should arrive at same score Contestability: debates over narrower concepts are more tractable and meaningful (DB) Contestability: debates over narrower concepts are more tractable and meaningful (DB)

8 Some specific indicators may also be Manipulable (design reforms narrowly for sole purpose of improving score) Manipulable (design reforms narrowly for sole purpose of improving score) Irrelevant (rule of law matters for development, but some best practice advice on how to improve rule of law may be wrong) Irrelevant (rule of law matters for development, but some best practice advice on how to improve rule of law may be wrong)

9 examples of narrow indicators PEFA PEFA Open Budget Index Open Budget Index OECD Procurement Assessment OECD Procurement Assessment Doing Business Doing Business Global Integrity Global Integrity Statistical Capacity Index Statistical Capacity Index Human Resource Management Human Resource Management Enterprise surveys Enterprise surveys Household surveys Household surveys

10 PEFA 28 high-level indicators measuring performance of PFM systems (3 more on donor practices)28 high-level indicators measuring performance of PFM systems (3 more on donor practices) a performance report based on the indicatorsa performance report based on the indicators applied in approximately 80 countries and by 19 donors (mostly WB and EC)applied in approximately 80 countries and by 19 donors (mostly WB and EC) common pool of information to promote country leadership, coordinated and aligned donor support (the strengthened approach)common pool of information to promote country leadership, coordinated and aligned donor support (the strengthened approach) about 30 assessments available at pefa.orgabout 30 assessments available at pefa.org

11 Open Budget Index Using questionnaire with 122 items, the OBI assesses: public availability of key budget documents public availability of key budget documents quality of information they provide quality of information they provide timeliness of their dissemination timeliness of their dissemination 59 countries assessed in 2006 study; goal of 80 or more in 2008

12 OECD Assessment Methodology for Public Procurement Systems 22 pilot countries Baseline indicators 12 indicators 12 indicators 54 sub-indicators 54 sub-indicators Based on review of regulatory framework and practices Based on review of regulatory framework and practices Compliance and performance indicators Based on sample of procurement transactions and other performance information Based on sample of procurement transactions and other performance information Assessment report

13 Doing Business Contract enforceability Contract enforceability Registering property Registering property Paying taxes Paying taxes Number of procedures Number of procedures Time Time Cost Cost

14 statistical capacity indicator (DECDG) Countries scored against specific criteria, using information available from the World Bank, IMF, UN, UNESCO, and WHO. Countries scored against specific criteria, using information available from the World Bank, IMF, UN, UNESCO, and WHO. Overall score based on 3 dimensions: statistical practice: ability to adhere to internationally recommended standards and methods statistical practice: ability to adhere to internationally recommended standards and methods data collection: frequency of censuses/surveys and completeness of vital registration data collection: frequency of censuses/surveys and completeness of vital registration indicator availability: availability and frequency of key socioeconomic indicators indicator availability: availability and frequency of key socioeconomic indicators

15 Enterprise Surveys 107 countries, firms 107 countries, firms unofficial payments as share of firm sales unofficial payments as share of firm sales time required to resolve dispute in courts time required to resolve dispute in courts costs of providing security against crime costs of providing security against crime time spent meeting with tax officials time spent meeting with tax officials percentage of firm sales reported for tax purposes percentage of firm sales reported for tax purposes major constraints to doing business major constraints to doing business

16 Household surveys Gallup World Poll Gallup World Poll TI/Gallup International Global Corruption Barometer TI/Gallup International Global Corruption Barometer Regional barometer surveys Regional barometer surveys World Values Survey World Values Survey is corruption widespread is corruption widespread paid bribes to police, tax officials, etc. paid bribes to police, tax officials, etc.

17 in-country diagnostic tools PER, CFAA, CPAR, fiscal ROSC PER, CFAA, CPAR, fiscal ROSC WBI GAC diagnostic survey WBI GAC diagnostic survey PETS (Uganda & others) PETS (Uganda & others) Citizen scorecards (Bangalore) Citizen scorecards (Bangalore) Road composition (Indonesia) Road composition (Indonesia) Municipal audits (Brazil) Municipal audits (Brazil) Pharmaceuticals (Argentina) Pharmaceuticals (Argentina) HRM (Albania, Macedonia, Romania) HRM (Albania, Macedonia, Romania)

18 Global Integrity Assesses existence of laws and institutions designed to curb corruption, and their implementation Assesses existence of laws and institutions designed to curb corruption, and their implementation Covers most UNCaC provisions Covers most UNCaC provisions Assessment by lead researcher in country; two peer reviewers per country Assessment by lead researcher in country; two peer reviewers per country Six broad categories, 23 sub-categories, 81 questions and numerous sub-questions Six broad categories, 23 sub-categories, 81 questions and numerous sub-questions Assessments in 2007 for 55 countries; 43 countries in 2006 Assessments in 2007 for 55 countries; 43 countries in 2006

19 Egypt 2007 strengths procurementprocurement political financingpolitical financingweaknesses public access to informationpublic access to information election integrityelection integrity accountability of public officialsaccountability of public officials external auditexternal audit

20 Hierarchy of Global Integrity Index Overall index Government Accountability (1 of 6 major categories) Executive Accountability (1 of 23 total sub-categories) Regulations governing conflict of interest by executive branch (1 of 81 questions) Are heads of state/government required to file regular asset disclosure forms? Can public access them?

21 not all categories are highly inter-correlated

22 implementation not as easy as passing laws

23 enforcement: a function of income per capita

24 How can ratings on broad indicators be improved? little understanding of what actions most effective in reduce incidence of corruption little understanding of what actions most effective in reduce incidence of corruption actionable indicators may improve this understanding actionable indicators may improve this understanding criteria in broad expert ratings often not detailed or even available criteria in broad expert ratings often not detailed or even available unclear how different aspects (e.g. grand v. petty corruption) are weighted unclear how different aspects (e.g. grand v. petty corruption) are weighted unclear what information sources are used by experts unclear what information sources are used by experts ratings may lag actions or events ratings may lag actions or events

25 ICRG corruption ratings for Ireland

26 timeline Dec. 1996: story breaks 5 to 4 Oct. 1997: tribunal convenes 4 to 3 Jan. 1999: hears 1 st witnesses 3 to 2 Feb. 1992: Haughey resigns as PM Nov. 1992: retires from politics

27 some hints from the data Examine correlations of 23 anti-corruption policy areas from Global Integrity with outcomes (TI, WEF) for up to 66 countries Examine correlations of 23 anti-corruption policy areas from Global Integrity with outcomes (TI, WEF) for up to 66 countries High: conflict of interest & asset disclosure rules for executive branch, public access to information High: conflict of interest & asset disclosure rules for executive branch, public access to information Low: conflict of interest & asset disclosure rules for judiciary, A-C laws & agencies, ombudsman Low: conflict of interest & asset disclosure rules for judiciary, A-C laws & agencies, ombudsman

28 TI weakly related to GI in law

29 TI strongly related to GI in practice

30 Global Integrity: not all categories are highly inter-correlated