1 Building Statistical Capacity to monitor the Progress of Societies Techniques for Evaluating Public Policies in Developing Countries Luiz Awazu Pereira da Silva Ministry of Budget and Planning (Brazil) OECD World Forum on Statistics, Knowledge and Policy on Measuring and Fostering the Progress of Societies Istanbul, Turkey from 27 to 30 June 2007 Based on: The Impact of of Economic Policies on Poverty and Distribution, Volume 1 by François Bourguignon, Luiz A. Pereira da Silva, eds. The World Bank, Oxford University Press (2003), and Volume 2 with Maurizio Bussolo (ed.) forthcoming (2007)

2 Measuring Progress of Societies changed from 50s (NA, aggregate data) to 90-00s specific economic policies and structural reforms in DCs (HHS, micro data) 1.Even macro-economic policies, monitoring post-stabilization requires specific surveys, not just NA aggregate data Fiscal monetary policy stance, exchange rate regime, public debt management strategy, etc. role of expectations (from market surveys) and of high frequency specific data (daily) 2.Evaluation of efficiency of public expenditures design of social policies requires targeting thru micro data Tax policy reform, composition of public expenditure, design of social programs (CCTs) civil service reform, pension reform, decentralization 3.Ex-ante analysis of structural reforms too Trade liberalization, liberalization of specific markets, financial sector reforms, improving the investment climate, land reform, privatization etc.

3 Measuring Progress in Brazil: New Data and Surveys (Monthly, Daily to monitor macro policies and efficiency of public spending) a)Credibility of Macro-Policies need to be monitored by high frequency macro data from specific surveys (daily, weekly) b)Where is really going public spending (40% of GDP of about $1trillion), especially health, education, social security, requires HHS and micro data

4 p inflation w wages L employt. Y growth G public spending. Macro data, aggregated, NA and specific surveys Measuring progress with macro economic policies Aggregate and first principle analysis used: assuming homogeneity of individuals and households, adds up into aggregate or average impact (e.g., higher growth, higher employment, lower inflation, higher social spending are all good on average for progress of societies) Policy objectives essentially macro concepts (e.g., IMF, World Bank of the s, old Governments, etc.) Macro data bases (NA) available as the natural analytical environment for aggregate, growth/social analysis

5 Scope and objective, challenges increase: evaluate the economic feasibility of public programs and policies and their overall development impact. Aggregate and first principle analysis insufficient: heterogeneity of individuals and households, microeconomic behavior do not add up into aggregate nor average, specificities of economic structures and local political economy, transmission of shocks and policies Policy objectives and social demand increasingly focusing on distributional effects and poverty reduction, essentially micro concepts (e.g., Post-WC IFIs, new types of Governments, etc.) Micro data bases (household surveys HHS) increasingly available as the natural analytical environment for distributional and poverty analysis Measuring progress with social policies

6 y i : real income w i : wage rate L i : labor supply E i : self-employment, non-wage income R i : net transfer income A i : socio-economic characteristics C i : consumption characteristics household-specific P price index p: general price index Measuring progress in social policies Define impact for individual i as the difference in income y i with and without the program, denominated y i :

7 Household Survey (HHS), i individual households Compare the distribution of y|P=1 with the distribution of y|P=0. Calculate changes in inequality or poverty across the two distributions Different tools/methods differ in how they construct the counterfactual distribution and the data that are needed Rank results according to some agreed upon rule and/or objective p prices w wage L employt. R transfers A households character. Program or policy will shock one or more components that explain the individual income y i Evaluation of Program and/or Economic Policy

8 Several Microeconomic techniques 1. Average Incidence Analysis a)Tax Incidence Analysis (Sahn & Younger) b)Public Expenditure Incidence Analysis (Demery) 2. Marginal Incidence Analysis a)Behavioral response to changes (Van de Walle) b)Poverty mapping (Lanjouw) 3. Impact Evaluation (randomization, matching, double- dif) a) Ex-post (Ravallion) b) Ex-ante (Bourguignon & Ferreira) 4. Data and Measurement (not covered here) a) Multi-topic Household Surveys (Scott) b) Qualitative surveys (Rao & Woolcock) c) Performance in Service Delivery (Dehn, Reinikka & Svensson)

9

10 New Macroeconomic techniques, from robust to more speculative…. 1. Standard RHG approaches to macro-micro linkage: a)"Micro-accounting"/RHG approach based on aggregate macro predictions (PovStat-SimSip-PAMS) b)The disaggregated SAM-CGE/RHG approach (Adelman-Robinson, Bourguignon and al. in the "Maquette, Loefgren, Robinson or Agenor and al. with IMMPA.) 2. Top-down "micro-simulation" approaches (micro-macro linkages) a)"Micro-accounting modules" linked to disaggregated macro models (Chen- Ravallion, McCulloch-Winters) b)"Micro-simulation modules" linked to disaggregated macro models (Bourguignon-Robilliard-Robinson, Ferreira-Leite-Pereira-Picchetti, Cogneau-Robilliard-van der Mensbrugghe) 3.Other issues for research and applications: a) Fully integrated models (Townsend, Heckman, Browning-Hansen-Heckman) b) Accounting for general equilibrium effects of public expenditure programs c) Dynamic modeling and the proper treatment of growth

11 Household Survey (HHS), i individual households p prices w wage L employt. R transfers A households character. Macro framework, general/partial equilibrium Evaluation of macro economic policies. Macro to micro linkages Instead of « exogenous and independent » shocks like before use « endogenous and dependent» shocks to 'microsimulate' the effect of policies on all individuals in the micro data sets, and the poor some consistency constraints will be « binding » (e.g., budget envelope for social programs, real GDP growth, etc.) LAVs =Linkage Aggregate Variables

12 Top-down "micro-simulation" approach within a macro-micro linkage approach Macro model Linkage AggregatedVariables (prices, wages, employment levels) Household income micro-simulation model

13 Top-down "micro-simulation" approach within a macro- micro linkage approach Two distinct approaches to micro-module: - "micro-accounting" : no explicit change in behavior (envelope theorem argument), e.g., Chen-Ravallion - "micro-simulation" : change in behavior, possibly linked to (labor) market imperfections, e.g., Robillard, Bourguignon, Robinson and Ferreira, Leite, Pereira da Silva, Picchetti LAVs from above Household income micro-simulation model

14 Example of Brazil Results using this technique: Aggregate Poverty and Inequality Indices (on aggregate, good results) after financial crisis 1999

15 Example: Brazil, 1999 Financial Crisis, Results of Simulation nominal changes in per capita income after floating ER, across centiles of income

16 Top-down "micro-simulation" approach Macro model Linkage AggregatedVariables (prices, wages, employment levels) Household income micro-simulation model Final Remarks: introducing feedbacks Feedback, e.g., micro-transfers, minimum wage

17 Concluding Remarks 1.Statistical capacity is needed more than ever and evolves with changes in policy-making practices and objectives (from aggregate growth to local quality & efficiency of public spending, targeting, evaluation ex-post/ex-ante) 2.Data needed evolved from aggregate-macro (National Acc.) to sectoral and mostly micro (HHS); continuity of time series is paramount to measure policy results 3.Information technology for collecting data, lower costs, allows applying techniques to MICs and LICs; international coordination (OECD, WB, etc.) important for cross-country and panel comparisons with policy-implications 4.Statistical capacity, data collection needs to be built-in project and policy design, ex-ante and not only ex-post 5.Macro policies need continuous micro evaluation of results to measure progress/efficiency (e.g., Bolsa-Familia in Brazil)

18 Measuring Progress in Society? Output Growth, quantitative indicators Effectiveness of Public Spending, Redistribution Poverty Reduction Kuznetstype-curve Aggregate growth then redistribution Effective social policies for growth and redistribution requires evaluation micro-data (HHS) & adequate policy design

19 END Caminante no hay camino, se hace camino al andar… (Antonio Machado)

20 Question? Usage of the techniques to meet DCs policy challenges, if, when, where and how Are these techniques for evaluation of public policies used? Some of these techniques are 'simple', yet not all of them are widely used, why? Costs (training) and institutional implications for Ministries and agencies (Finance vs. Planning, political economy of budget process, etc…). Aid agencies (IDA, DFID, AFD, etc…) promoting evaluation, When these techniques are used, are they useful for policy-makers? They capture only the 'between' (RHG) dimension of distributional changes, which empirically proves limitative. They are ill adapted to the distributional aspects of growth, but important in putting broader (poverty) perspective to decisions Where are these techniques for evaluation of public policies used? Examples below

21 Usage of these techniques in connection with policy-making (with external assistance) Average Incidence Analysis (Tax Incidence Analysis and Public Expenditure Incidence Analysis) Most OECD countries. Also in many DCs particularly IDA [Ghana (ISSER), Madagascar (INSTAT & Cornell Univ.), Uganda (EPRC)] Marginal Incidence Analysis Many OECD countries and India, using NSS 1994; Indonesia, using SUSENAS 1981 & 1987; Vietnam using panel from VLSS 1993 & 1998; Argentina,using public spending and census data, Ministry of labor team; Brazil using PNADs Poverty mapping Many OECD countries Ecuador, Bolivia, Mexico, Panama, Nicaragua, Guatemala, South Africa, Madagascar, Kenya, Uganda, Malawi, Mozambique, Tanzania, Bulgaria, Albania, Thailand, Vietnam, Cambodia, Indonesia, China., Brazil, etc. Ex-post impact evaluation methods(randomization, PSM, double-dif) Many OECD countries. Also in many DCs, Argentina, Brazil, Kenya

22 "Micro-accounting"/RHG approach based on aggregate macro predictions SimSIP/PAMS Latin America, Burkina-Faso, Thailand, Indonesia The disaggregated SAM-CGE/RHG approach IFPRI (US) country models, IMMPA-Cameroon, Brazil, many countries have GTAP+LAVs+HHS "Micro-accounting modules" linked to disaggregated macro models China, Colombia, Brazil, Many countries have GTAP+LAVs+HHS "Micro-simulation modules" linked to disaggregated macro models Indonesia, Brasil More sophisticated? Fully integrated models (Thailand, Madagascar) Accounting for general equilibrium effects of public expenditure programs (???) Dynamic modeling and the proper treatment of growth (???)