1. 1 Measuring Progress in Democracy and Human Rights: Why? How? To whom? By Jan Robert Suesser and Raul Suarez de Miguel Second OECD World Forum on Statistics, Knowledge and Policy: “Measuring and Fostering the Progress of Societies” Istanbul, Turkey, 27-30 July 2007

2. 2 1.INDICATORS OF PROGRESS IN THE AGE OF GLOBAL HUMANISM

3. 3 AN EVOLVING PARADIGM OF PROGRESS Enlightenment: knowledge and freedom (18 th cent.) Modern science and technology (19 th –20 th cent.) Economic growth / economic integration (40s-60s) Social integration / fair distribution of wealth (60s) Quality of life (70s and 80s) Human development (80s and 90s) Sustainable development (90s and 00s) Millennium Development Goals (late 90s and 00s) Now, happiness ?

4. 4 A GLOBAL PARADIGM OF PROGRESS Universally shared values Universally shared goals Universally shared tools

5. 5 THREE KEY DIMENSIONS OF PROGRESS… Human rights Democratic participation Governance and accountability … THAT DESERVE PROPER MEASUREMENT

6. 6 A NEED FOR EVIDENCE-BASED ASSESSMENT IS EMERGING WORLDWIDE Huge demand for robust data and meaningful indicators (national policy actors and civil society, international institutions, development agencies). Many measuring initiatives are being implemented in different regions of the world. Partly identified through a global survey, these initiatives are being documented in an inventory, available on-line in: www.metagora.org.

7. 7

8. 8 ON-GOING MEASURING EFFORTS INCLUDE: Global initiatives to build indicators of democracy, human rights and governance Regional initiatives to set-up evidence-based monitoring mechanisms National and local initiatives to measure human rights and democratic governance

9. 9 THESE INITIATIVES PRESENT SEVERAL LIMITATIONS: International indicators are of rather limited use in assessing the impact of national policies. Their sources and methods are often opaque and their comparative scope (ranking) is broadly questioned. Regional initiatives are still incipient. National and local initiatives are too often based on rather rudimentary quantitative methods. They are implemented with very limited human, technical and financial resources.

10. 10 2. THE METAGORA PROJECT: APPROACH, METHOD AND LESSONS

11. 11 CAN HUMAN RIGHTS AND DEMOCRATIC GOVERNANCE REALLY BE MEASURED ? By whom? With which methods? Under which conditions? For which purposes?

12. 12 METAGORA FORMULATES RESPONSES, RELYING ON ITS 2004-2006 EXPERIENCE Several pilot national experiences were carried out in various regions of the world. These pilot experiences, which willingly addressed sensitive issues, were selected by national stakeholders in different political, social and cultural contexts

13. 13 A Decentralized Laboratory Content

14. 14 THESE PILOT EXPERIENCES WERE CONDUCTED BY SEVERAL ORGANIZATIONS gathering together and sharing their specific expertise within a North/South community : Human Rights Institutions Research Organizations Civil Society Organizations National Statistical Offices Governmental bodies

15. 15 ASSESSING STEERING Independen t Panel of Experts Committee of Donors IMPLEMENTING AAAS USA PCBS Palestin e HSR C South Africa Fundar Mexico DIAL France SGCA Andean Community CHR Philippine s Coordinatio n Team OECD / PARIS21 Group of NSIs Benin, Burkina Faso, Côte d’Ivoire, Mali, Niger, Senegal, Togo & Madagascar Group of NSIs Bolivia, Colombia, Ecuador, Peru & Venezuela Partners’ Group

16. 16 ADRESSING SOME BASIC QUESTIONS Can multidimensional human rights and democratic governance issues be measured through surveys? Will people respond to sensitive questions? Will the gathered information be statistically significant and politically relevant? Can official statistical agencies be involved?

17. 17 METAGORA METHOD OF WORK (1) A bottom-up approach consisting of: identifying with stakeholders national key issues for which evidence-based assessment could be policy relevant; applying statistical methods adapted to each particular context; assessing these methods for their capacity to provide policy-relevant results; providing stakeholders with a shared knowledge on the policy issues at stake; contributing to draw shared lessons from the pilot experiences.

18. 18 METAGORA METHOD OF WORK (2) The Metagora bottom-up approach complements the top-down global approaches to measure democracy, human rights and governance. It aims at generating measurement tools which can contribute at designing, implementing and evaluating national and local policies. Its primary goal is not to compare national performances or to make international ranking, but to address major national issues.

19. 19 ENHANCING NATIONAL CAPACITIES Metagora promotes and stimulates: participatory processes based on large consultative mechanisms; mutual learning among the national stakeholders and actors committed in national pilot experiences; international transfer of skills; fostering, through increased authoritative skills, the leading role of key national institutions and actors.

20. 20 1.Measuring human rights and democratic governance is technically feasible and politically relevant. Sensitive data on human rights, democracy and governance can be collected and analysed using statistical tools. 2.On the basis of this information, indicators can be produced that are relevant and useful for political decision and action. DRAWING LESSONS FROM THE METAGORA PILOT EXPERIENCES

21. 21 DRAWING LESSONS FROM THE METAGORA PILOT EXPERIENCES 3.Quantitative data and qualitative information can and should interrelate to properly inform assessment of human rights and democratic governance. 4.Official Statistical Agencies can be efficiently involved in various forms in evidence-based assessment of human rights and democratic governance. Nevertheless the decision on such an involvement should be taken along a national coherent strategy for the development of the national statistical system.

22. 22 5.Quantitative indicators and statistical analysis bring a significant value-added to the work of national Human Rights Institutions. 6.Statistical methods can substantially enhance the research and advocacy of civil society organizations in the fields of human rights and democracy. 7.Available statistical data should be used to structure and inter-relate relevant indicators of social, economic and cultural rights. DRAWING LESSONS FROM THE METAGORA PILOT EXPERIENCES

23. 23 DRAWING LESSONS FROM THE METAGORA PILOT EXPERIENCES 8.Indicators gain relevance when they: –Are related to specific public policies and programs; –make evident the accountability of identifiable public authorities; –allow to address specific human rights issues for specific populations; –serve as basis for democratic dialogue, policy design and decision-making processes.

24. 24 DRAWING LESSONS FROM THE METAGORA PILOT EXPERIENCES 9.To become sustainable, statistics and indicators must: be nationally based and owned (appropriation); Rely on independent and robust professional basis; enjoy a broad social and political legitimacy; be authoritatively institutionalised; be financed through ad-hoc headings of public budgets.

25. 25 3.TOPICAL EXAMPLES OF WHAT CAN BE MEASURED

26. 26 Example: measuring irregularities, abuse of power and ill-treatment in Mexico City (Federal District) Persons without abuse 47 % Persons with contact 24 % Persons with abuse 53 % Persons with non- physical abuse 93 % Persons with physical ill-treatment 7 % Incidence of contact with public security and procurement of justice authorities Incidence of abuse Type of abuse Target population: persons aged 15 or more living in the Federal District (6,400,000 persons) Reference period: events occurred between November 2003 and October 2004 Measuring method: random sample household survey, conducted through face-to-face interviews. Persons without contact 76 %

27. 27 Example: Non-physical abuse in contacts with law enforcement authorities (Survey results correspond to 2,300,000 contacts experienced by 1,520,000 persons)

28. 28 Example: Reasons for not reporting abuses (spontaneous multi-answer)

29. 29 Example: incidence of corruption within public agencies in Peru

30. 30 Example: levels of corruption and civil servants’ wages in Antananarivo, Madagascar

31. 31 Example: in Peru, support for democratic regime weakens as corruption perception increases

32. 32 Example: measuring dimensions of democracy: are they fundamental? Are they respected?

33. 33 Example: comparing expectations for and respect of democratic principles in French speaking Africa

34. 34 Example: comparing support to democracy in Andean countries

35. 35 Example: how important is democracy for people in Ecuador?

36. 36 Example: linking quantitative and qualitative approaches to assess indigenous peoples’ rights in the Philippines Quantitative approach: SURVEY FINDINGS Qualitative approaches: FOCUS GROUP DISCUSSIONS  High perception and awareness of rights to ancestral domain and land (Bago, 68%), Bugkalot (70.8%), Kankana-ey (60.8%)  Government is in second place as source of information of rights to ancestral domain and land: Bago 28.4%, Bugkalot 54.2%, Kankana-ey 22.4%.  Experienced violations of rights consisting of encroachment (Bago-6%, Bugkalot 30.9% and Kankana-ey 13.2%); pollution (Bago, 4.7% Bugkalot, 17.7%, Kankana-ey, 8.8%); illegal entry (Bago, 5.3%, Bugkalot, 46.3% and Kankana-ey, 13.2%). Existence of violations on land grabbing by private individuals (Bago 50%; Bugkalot, 56.5%, Kankana-ey, 55.6%), council of elders and others  Substantial awareness and availment of governmental programs and services  Enjoyment of land ownership and acquisition of right to ancestral domain  Average satisfaction (68-78%) on delivery of government programs and services  90% considered customary laws helpful in solving land issues; 52% of land issues are resolved by customary laws  Top five primary needs: adequate food, housing, water system, livelihood and education  Tribal leaders and women with higher awareness and perception of rights to ancestral domain and land. Youth has lowest awareness. Right of ownership, to develop lands and natural resources and to stay in territories well understood. Low or no awareness of other rights listed under IPRA. Apparent confusion of rights due to lack of knowledge about distinction or difference between rights to ancestral domain and land  Positive effect of IPRA on their rights to ancestral domain and rights  Existence of violations and sources are tribesmates, other tribes, private mining companies  Recognition of government efforts in fulfilling rights to ancestral domain and land  Customary law as primary source of dispute resolution affecting rights to ancestral domain and land LOCAL CONSULTATIONS  Demand for relevant and deeper human rights and IPRA education  Need for livelihood and organizing especially from women sectors  Development aggression of private sectors permitted/not controlled by government and co-opted by some tribal leaders  Lack of delivery of vital services  Discriminatory policies to access rights to education and other social services  Pollution of and inadequate water resources  Peace and order to ensure personal security

37. 37 COMPARING OFFICIAL STATISTICS WITH EXPERTS’ VIEWS Appended Module to Regular Household Surveys conducted by National Statistical Offices in French Speaking Africa (35,600 persons interviewed; 4,500 for each capital city in average);* Expert panel survey (“Mirror survey”) conducted by DIAL-France (246 persons surveyed; 30 experts for each country in average). * In Madagascar, results are drawn from the 2003 survey.

38. 38 How far can we trust experts’ opinion ?

39. 39 How far can we trust experts’ opinion ?

40. 40 ASSESSING ACHIEVEMENTS, DOCUMENTING PILOT EXPERIENCES AND PRODUCING TRAINING MATERIALS An independent panel of senior experts is monitoring the process and assessing the outcomes of the Metagora pilot experiences. Experiences, problems encountered and lessons learned were documented in the form of on-line training materials aimed at facilitating the replication and extension of the pilot experiences in other countries and other contexts.

41. 41 UPCOMING CHALLENGES Consolidating the working method Replicating and extending the pilot experiences Enlarging the Metagora community Enhancing the policy impact Enriching the methodology and tool box Financing field operations and analytical work

42. 42 Warm Thanks for Your Attention !