Decentralization Thematic Group/Public Sector Group Funding “Assessment of the Impact of Decentralization: The Case of Colombia ( )” Presentation by: Jonas Frank (LCSPS) March 10, 2004

The Puzzle: What is the impact of decentralization and can this impact be evaluated? Better Services? Economic Efficiency? Democratic Government?  Evaluation is about finding causality

The Colombia Case: The Study:  Sample of 148 municipalities, out of a total of 1100  70 indicators  1000 pages  It took ca. 2 years to complete the evaluation Appropriate time-frame: process started in 1986, evaluation covers years following 1991 Integral reform: political, fiscal, administrative Typical sequence: political reform first, then fiscal and administrative decentralization; municipal level

Three important questions: Why evaluate? What should be evaluated? How can one evaluate?

Objectives of Evaluation differ by actors of decentralization:  Minister of Finance, Indigenous People, Legislators, Mayors, Regional Governors, Donors,... There is no single objective, but evaluations can...  (i) improve decision-making  (ii) create accountability among actors involved I. The “Why” of Decentralization Evaluation

Implications:  Evaluation is a continuous exercise  Transformation of data into information  A participatory exercise vs. purely technocratic I. The “Why” of Decentralization Evaluation

Objective of the Colombia Study: Crisis: fiscal pressures, weak budget constraints, bailouts Inefficient services Slow democratization Lesson: Evaluate as early as possible Ownership Too many objectives of evaluation I. The “Why” of Decentralization Evaluation

Some guiding questions: Where is decentralization supposed to have an impact? Were the initial goals of decentralization met and to what extent? Dilemas: Decentralization goals remain vague They are not expressively formulated and agreed upon Decentralization is a “moving target” I. The “What” of Decentralization Evaluation

Decentralization Goals in Colombia:  In 1986: democratization  In 1991: efficiency in services, redistribution, services for the poor, improve popular participation  In 2001: fiscal discipline, efficiency I. The “What” of Decentralization Evaluation

Venezuela Ecuador Peru Bolivia Colombia ARG Brazil Rio Grande Chile LAC: Decentralization Objectives (1970 – 2002)

Colombia Study: “Comprehensive evaluation” (not only fiscal): 1. Outcomes within eight sectors  Education  Health  Water and basic sanitation  Fiscal performance  Political decentralization  Management capability  Economic development 2. Municipal Progress: (i) sectoral outcome (ii) aggregate outcome, II. The “What” of Decentralization Evaluation

1. Sectoral Evaluation: Results in Education “Decentralization has improved and contributed to:  Increase in coverage  Improved teacher/student ratio  Higher schooling levels  Reduction in illiteracy rate” II. The “What” of Decentralization Evaluation

Results in Education

Sectoral Evaluation: Results in Education

1. Sectoral Evaluation: Results in Health “Decentralization has improved and contributed to:  Increased coverage  Higher public spending  Greater equity  Lower infant mortality rates  Greater ratio of physicians per inhabitant”. II. The “What” of Decentralization Evaluation

Sectoral Evaluation: Results in Health

2. Performance Evaluation of Municipalities (i) Municipal Performance in Six Sectors (Health, Education,...) (ii) Aggregate Performance: Most municipalities have achieved an average performance; only 4% achieved and acceptable standard But:  What were the starting conditions?  Where there several observations in the past and when? II. The “What” of Decentralization Evaluation

Results in Economic Development

Lessons: Some important areas are excluded:  Allocative efficiency  The regional level: departmental government Conclusions about the causality of decentralization are primarily hypothetical “Less is more” II. The “What” of Decentralization Evaluation

1. Base-line of evaluation 2. Finding causality 3. Indicators 4. Selection of samples of local government 5. Periodicity of evaluation III. The “How” of Decentralization Evaluation

1. Base-line Usually no such base-line has been established before decentralization was implemented   The use of proxies   Select indicators that are available and mirror the starting conditions III. The “How” of Decentralization Evaluation

2. Finding Causality To date, impact studies of decentralization are either: (i) Evidence-based (ii) Subjective (iii) Hypothetical All of these three types of studies provide a useful understanding and appraisal at various stages of the decentralization process. III. The “How” of Decentralization Evaluation

2. Finding Causality The Colombia study is primarily hypothetical Statistical analysis is not sufficient to prove causality Lessons:  Discerning between decisions that are in hands of local government and which are not  Clarifying the value added of local government in carrying out new responsibilities?  Continuous monitoring III. The “How” of Decentralization Evaluation

2. Finding Causality: Example of the Education sector Local government defines curriculum content:   ”Instruction Indicator” Local government is responsible for construction and maintenance of school buildings   “Infrastructure Indicator” Local government develops education improvement plan   “Planning Indicator” Local government allocates budget   “Budget Indicator” Is this operationally possible? III. The “How” of Decentralization Evaluation

3. Indicators 70 indicators were used in the Colombia study Lesson:  Selection of indicators depends on “what” will be used, but:  use only a minimum set of indicators, and...  indicators that can be monitored regularly and with low cost III. The “How” of Decentralization Evaluation

4. Comparability of data and selection of local government Colombia: “representative sample” = most diverse municipalities were selected Lesson:  Use only most similar cases because diversity increases during decentralization process  Split them into groups: eliminate some structural factors that lie outside control of municipalities (population size) III. The “How” of Decentralization Evaluation

5. Periodicity of evaluation Colombia: evaluation after 10 years of time Lesson:  Time period sufficiently large, but...  Regular monitoring is necessary  It is important to separate between short-term, medium-term, and long-term goals  helps create accountability  helps better decision-making III. The “How” of Decentralization Evaluation

Conclusion: Risks and Opportunities Evaluation as a purely bureaucratic exercise (i) Evidence and (ii) subjective evaluation are still important: the only “early-warning system” that is operational Ready to propose and implement corrective measures: “exit” of decentralization process? Giving “erroneous” answers to the “wrong” questions: “Decentralization has not worked well enough because there was not enough of it”