1. 1 Indicators based on representative surveys of firms, households and service providers Javier Herrera (IRD - DIAL) Mireille Razafindrakoto (IRD - DIAL), and François Roubaud (IRD - DIAL) Seminar on “The Empirics Of Governance“ May 1 – 2, 2008, Washington DC

2. 2 PRESENTATION SCHEME DIAL Développement, Institutions et Analyse de Long terme I.Uses of household surveys II.Taking into account sources of bias III.Objective vs subjective corruption indicators IV.Conclusions & Perspectives

3. 3 I. Uses of household surveys Household survey as a « voicing » instrument; public awareness on corruption (particularly in of authoritarian political contexts). Complex links with policy

4. 4 Historical perspective  First estimation of the extent of corruption in 1995 in Madagascar (with Household survey)  Headlines in the press: « Outcry against corruption! »  It then became impossible to ignore the problem

5. 5 1995 It then became impossible to ignore the problem  the Ministry of Justice took steps to introduce a system of sanctions.  a draft law on the fight against corruption BUT Draft law rejected by the Government Council in 1999  Importance of context & characteristics of institution (~ authoritarian regime and problem of governance) 2002 Institutional change  Stress put on transparency  2003 Creation of an independent council (CSLCC)  2004 Independent anti-corruption office (BIANCO) MEASURE 1995  1998  DEFINITION OF POLICY QUALITY OF INSTITUTIONS / Political Regime Positive Impact Interactions

6. 6  Headlines in the press in may 2005: « More confidence & less corruption » after the public conference presenting the first results of the survey on « governance, democracy and fight against poverty »

7. 7 Sources : Razafindrakoto, Roubaud (2002) and 1-2-3 Survey 2002-2004, INSTAT, DIAL, authors calculations.  An improvement of civil servant wages  or an active anti-corruption policy  a sharp drop in the incidence of corruption Civil servant salaries and corruption levels in Madagascar 1995-2004 GOVERNANCE

8. 8 II. Taking into account sources of bias Who answers the question matters (random sample selection within the household) But also it matters to know more about who don’t answer (non respondent profile) Taking into account the working of corruption (middlemen, social norms, etc.) Robustness needs to be assessed Informal production units needs adapted survey approach (the 123 surveys)

9. 9 III. Taking into account sources of bias The wording and sequence of question matters (but not only when measuring corruption, ie. labor surveys). Pilot surveys are important. Non response to governance questions in HH surveys are lower than in standard questions (ie. Income) Unobserved heterogeneity (« over optimistic »/ »under optimistic », use panel data -ongoing research)

10. 10 The surveys Governance and Democracy Modules grafted onto official household survey conducted by National Statistical Institutes Survey support (stratified multi-stage sample surveys: area/HH) for the Modules (Individuals): In Africa In Africa: 1-2-3 Survey (LFS, informal sector, consumption and poverty) - Madagascar 1995-2004, Capital + urban areas (2000, 2001), rural, entreprises surveys (time series) - West Africa 2001/2003, in 7 WAEMU capital cities (Abidjan, Bamako, Cotonou, Dakar, Lomé, Ouagadougou and Niamey) 35 594 persons interviewed In Latin America In Latin America: Standard National Household surveys (ECH, Bolivia; SIE-ENEMDU, Ecuador; ENAHO, Peru). National and regional inference for Peru and Ecuador. - Peru 2002-, national level (18 000 HH sample in 2002; continuous survey from 2003 to date). - Ecuador 2004, (20 000 HH) - Bolivia 2004, national (1 700 HH). -The survey is taking place in Colombia during the 2nd term of 2005. More than 50 000 persons interviewed

11. 11 Strong points / Basic principles : Ownership Reliability Sustainability Conducted by National Statistical Institute  Integration in the National Statistical System Relevance Systematic Presentation / Publication of the result  wider public  Validation  Democratic debate  demand Bottom-up approach Investment in capacity building Light, flexible tool  Reconductible  time-series Marginal cost HH survey  « Voicing »  empowerment, accountability Supply sideDemand side Interactions

12. 12 Non response There is (often) more willingness to answer questions on governance and democray than usual economic questions Non response rate to selected questions % West AfricaMada Coton ou Ouag a- dougo u Abidj an Bama ko Niame y DakarLoméAntan a- narivo Total Opinion on governement functioning 03.60.91.12.75.23.50.12.2 Opinion on democracy02.80.50.91.93.10.30.21.3 Income level (continuous) 40.254.340.943.351.656.337.247.3 Income level (continuous or discrete) 2.36.53.26.514.19.21.60.1 Sources : 1-2-3 Surveys, Phase 1, Governance and Democracy module, 2001/2003, National Statistical Institutes, AFRISTAT, DIAL, authors’ calculations.

13. 13 Robustness Probabilistic surveys allows a quantitative assessement of indicators’ precision (also relevant for group and intertemporal comparaisons) ruralurbanTotal has increased31,039,136,6 95% confidence interval [29,0 - 33,0][37,4 - 40,9][35,3 - 38,0] has decreased8,4310,479,85 95% confidence interval [7,5 - 9,5][9,5 - 11,6][9,1 - 10,7] is the same49,148,848,9 95% confidence interval [47,5 - 14,0][46,9 - 50,6][47,4 - 50,3] don't know11,51,64,7 95% confidence interval [10,3 - 12,9][1,3 - 2,1][4,2 - 5,2] Total100 In your opinion, corruption since last year? Source: our estimation based on ENAHO july 2003-june 2004, INEI, Peru.

14. 14 Taking into account the working of corruption: middlemen; failed corruption; “accepted” corruption, amounts paid, access to services; links with inefficiency… Source: ENAHO 2004 HH survey Peru

15. 15 Incidence and cost of corruption (2002/2003 and 2004 in Antananarivo) En 2002/2003En 2004 TotalIncome per capitaTotalIncome per capita 1 st quartile4 th quartile1 st quartile4 th quartile Incidence (all households) 16,311,920,18,46,010,3 No contact with civil service 40,446,831,529,934,222,4 Incidence (households victims) 27,422,429,312,09,113,3 Total annual amount (1 000 Fmg) Mean (HH victims of corruption) 38121968230077484 Median (HH victims) 252050301450 % of income (HH victims) 3,37,83,11,21,81,2 Source : Enquête 1-2-3, modules qualitatifs, 2003 et 2004, INSTAT, authors calculations. Corruption incidence taking into account access to public services  Incidence (percentage of households victims) has decreased  Total amount paid has decreased in real terms  The poor as well as the rich had benefited from this improvement

16. 16 “Don’t know” answer is not random: it concerns mainly the poor, the rural, the less educated, the socially excluded Importance of democracy

17. 17 Corruption incidence for the informal sector production units can be measured CotonouOuagadougouAbidjanBamakoNiameyDakarLomeTotal %Incidence of Informal Units that had a problem with civil servants last year 4.75.07.03.56.28.56.2 % that settled the problem: By paying a fine 75.576.742.938.540.449.244.547.3 By giving a “gift” 13.95.750.232.624.738.145.141.3 other 10.517.66.928.934.912.710.411.4 Source: Alain Brilleau et al. « Le secteur informel : Performances, insertion, perspectives, enquête 1-2-3 phase 2 », STATECO n°99, 2005 p.82.

18. 18 III. Objective vs subjective corruption indicators Expert’s surveys vs. HH surveys Do they measure the same phenomena? (petty vs. big corruption) Minding the gap between :  -perceptions and objective indicators  -aspirations and perceived outcomes

19. 19 The mirror survey To round out the collection of surveys on Governance and Democracy in seven West African capitals and in Madagascar (Antananarivo), an additional survey to get the opinions of a certain number of Southern and Northern experts (researchers, development practitioners, decision-makers, senior civil servants, politicians, etc.). The aim --> to compare answers from the population surveyed in each country with the specialists’ point of view.

20. 20 The mirror survey (…) Two sets of questions: - what the experts think the respondents answered on average. - their own answer to these same questions (“What is your personal opinion?”) Questions : - Specificity of the answers of the population / experts, specialists - Knowledge of Northern or Southern experts on what happens and on people’s thinking in the South -Relevance and reliability of indicators based only on appreciation of panel of experts Sample size : 250 experts (30 per country in average) DIAL Développement, Institutions et Analyse de Long terme

21. 21 How far can we trust the experts’ opinion on corruption?  Discrepancies between real extent of corruption and experts’ perception in Francophone Africa  On the level of corruption / To what extent corruption acceptable  On the relative positions (rank) of the different countries Sources: General Household Survey (35,594 persons interviewed; 4500 for each country in average); Expert panel survey or Mirror survey (246 persons surveyed; 30 experts for each country in average). * In Madagascar, results are drawn from the 2003 survey. Authors calculations. The need for complementary approaches / indicators

22. 22 Perception of the main democratic principles by the population  Are they fundamental? Are they respected in the country?  Measuring the gap between aspirations and effectiveness as regards democratic principles DEMOCRACY Sources : Enquêtes 1-2-3, module Démocratie, 2001/2003, Instituts Nationaux de la Statistique, AFRISTAT, DIAL authors calculations.

23. 23  Measuring the gap between aspirations and effectiveness as regards democratic principles DEMOCRACY Sources : Enquêtes 1-2-3, module Démocratie, 2001/2003, Instituts Nationaux de la Statistique, AFRISTAT, DIAL authors calculations.

24. 24 No correlation between  Objective measure (incidence of corruption)  Subjective perception of the working of civil service The need for complementary approaches / indicators

25. 25 The global lessons Advantages of household sample surveys  Transparency of measurement procedures  Representativeness  Quantification,  Comparability of indicators over time. In-depth policy-oriented analyses More appropriate than international indicators and aggregates. Both objective (behaviour, actual experiences) and subjective information (perception, satisfaction) Monitoring and relating the two fundamental dimensions of these phenomena. Socio-economic disaggregation These two dimensions can be combined with traditional variables related to the socio-economic characteristics of individuals and households (income, occupation, sex, age, ethnic group, etc.). Possibility to disaggregate information between different population categories (gender, poverty, ethnic groups, discriminated people, etc. International comparability

26. 26 IV. Conclusions & Perspectives Measure different dimensions of governance A detailed understanding of different forms of corruption is necessary to analyse its determinants and consequences Different sources of potential bias should be considered in designing & analysing surveys Both, objective and subjective, governance indicators need to be collected to better understand aspiration/outcome gaps and why corruption may become or not a political issue.

27. 27 END For more works on Governance, Democracy and Poverty see our web site http://www.dial.prd.fr Our article: “Governance, Democracy and Poverty Reduction: Lessons drawn from household Surveys”, International Statistical Review (2007), vol. 75, issue 1, pp.70-95 is available upon request