How can we know if EU cohesion policy is successful? Integrating micro and macro approaches to the evaluation of Structural Funds John Bradley EMDA (Economic Modelling and Development Systems) Copy of paper available on

National Development Plans and Structural Funds Large-scale investment aid for physical infrastructure, human resources, production incentives EC and local (public & private) co-finance Targeted at lagging EU member states Massively expanded after 1989 Implemented through multi-year National Development Plans

Regional GDP 2001 Part I Situation and trends GDP per head (PPS), 2001 < >= 125 No data Index EU 25 = 100 Source: Eurostat

Two aspects of cohesion policy Policy design and implementation: usually a pragmatic process driven by local political wishes with local and EC oversight Implementation success depends on institutional capabilities and fiscal constraints Policy evaluation: (ex-ante, mid-term and ex- post) Logically the two aspects should be inter- linked. In practice they tend not to be (but ESRI “Investment Priorities” an exception)

Structural Fund impact evaluation: micro versus macro techniques

Micro (bottom-up)Macro (top-down) Level of disaggregation High (individual projects)Low (sectoral aggregates, whole economy) Use of theoryWeak (judgemental, CBA) Strong (macroeconomics) Model calibrationJudgemental/informalScientific(?)/econometric Policy impactsInformal/implicit/ranking /some quantification Formal/explicit/quantified Treatment of externalities Limited or ignoredIncluded/explicitly modelled

How can we know if EU cohesion policy is successful? Integrating micro and macro approaches to the evaluation of Structural Funds (Bradley, Mitze, Morgenroth & Untiedt, March 2006) Paper available on:

Implementing a micro-based approach to evaluation Welfare economics and the underlying rationale for public expenditure 1.Public goods 2.Corrective pricing (due to presence of externalities) 3.Targeted interventions (information asymmetries) 4.Redistribution (agriculture, social housing: but mainly through tax and social welfare system)

Public goods: evaluation criteria

Corrective pricing: evaluation criteria

Targeted interventions: evaluation criteria

Redistribution: evaluation criteria

Two strands to the macro debate on cohesion policy effectiveness A political-economic literature that stands back from technical analysis, but argues in terms of theoretical paradigms An empirical literature that examines the issues empirically, using a variety of different analytical models.

Impact of recent research in economics New Trade Theory (Helpman & Krugman, 1985) New Growth Theory (Grossman and Helpman, 1991) New Economic Geography (Fujitsa, Krugman, Venables, 1999)

Empirical studies of cohesion: Two methodological approaches [1] “Testing” methodologies: Testing a null hypothesis (e.g., cohesion policy has no effect) [2] “Impact evaluation” methodologies: Tracing out complex causal chains of policy consequences, and quantifying impacts

Macro evaluation: [1] Hypothesis testing Ederveen et al, 2002: “Funds and games: the economics of European cohesion policy” “Fertile soil for Structural Funds? A panel data analysis of conditional effectiveness of European cohesion policy” Midelfart-Knarvik & Overman, 2002 “Delocation and European integration: is structural spending justified?”

Ederveen et al, 2002 Used Barro-type regresions over the period to for 13 EU countries. Found no statistically significant “cohesion policy” effect (except for Ireland!) Critique: Looks only for growth impacts (i.e., ignores “level” impacts); inappropriate data sample; crude panel-regression model)

Mitelfart-Knarvik & Overman, 2002 Focused on role of cohesion policy on industrial location, as it affects the interplay between agglomeration and dispersion forces Finds that cohesion policy influences endowments, but endowments do not appear to feed through to changes in production structure Ireland also an outlier, due to pre-cohesion policy investment in human capital.

Macro evaluation: [2] Modelling causality and impacts Be aware of the “built-in” limitations of the type of model selected: I-O, CGE, growth, macro-sectoral Implement an appropriate level of sectoral disaggregation on the production side Nest cohesion policy mechanisms within wider domestic and global drivers of growth Address the difficult issue of model “calibration”

The uses of the macro models Constructing internally consistent medium-term baseline scenarios or forecasts Analysis of conventional policy shocks (external environment, domestic policy, etc.) Analysis of complex policy shocks like a EU Structural Funds)

Key issues arising from the macromodel-based research Need to have a more explicit treatment of FDI Need to evaluate carefully fiscal and monetary crowding out mechanisms Need to incorporate migration mechanisms, and treat labour inputs in more detail.

Contexts for Structural Fund impact analysis The model as a global framework for economic analysis The model as an explanatory framework for the study of growth and cohesion The model as an action framework for SF impact analysis

Construction phase vs Use phase in cohesion policy During construction phase, there will be large demand-side impacts. These vanish after completion (i.e., after 2013/15 for next NSRF) Increased stocks of infrastructure and human capital can generate long-tailed supply-side impacts The size of the supply-side impacts depend on the appropriateness and effectiveness of the NSRF

Physical infrastructure: PI Demand-side impacts (implementation): PI  IG  I (total investment)  (Keynesian multiplier)  impact on GDP Supply-side impacts (mainly post-implementation): PI  increased stock of infrastructure (KPI)  boost to output/productivity

Human resources: HC Demand-side impacts (implementation): HC  Income & Public expenditure  Keynesian multiplier  GDP Supply-side impacts (mainly post-implementation): HC  stock of human capital (KHC)  boost to output/productivity

A serious methodological challenge Ex-ante impact analysis of “yet-to-be- implemented” NDPs Is the NDP appropriate? How effective will be the implementation? Strict monitoring and evaluation can help, but do not guarantee success

Infrastructure and human capital interaction effects The links between infrastructure and human capital are difficult to measure. A parallel improvement in both is probably necessary But we cannot say much about the optimum balance between them within an NDP

What macromodel? QUEST versus HERMIN QUEST: quarterly; one-sector; model- consistent expectations; no CEE models of new EU member states HERMIN: annual; four-sector (+); auto- regressive expectations; applied to “old” EU and new EU member states

HERMIN versus QUEST The issue of “crowding out” CEE economies operating below capacity Public goods and externalities Modest domestic co-finance requirement Quasi-euro zone, so no monetary impacts

Presenting model-based cohesion policy impacts Difficult to define an appropriate counterfactual baseline scenario. Difficult to assign values to the spill-over (or externality) elasticities to different countries in he absence of empirical research. Macro impacts are complex, and GDP is an imperfect indicator

Long –run impact of cohesion policy Policy impacts build up gradually over time, so use accumulated change in GDP relative to the no-policy baseline Big SF injection implies big shock, so normalise SF expenditure as a percentage of GDP Define the SF cumulative multiplier as the accumulated percentage change in GDP compared to the no-poicy baseline caused by a one-percent of GDP SFshock

Ordinary policy multiplier Change in GDP Change in public investment Cumulative policy multiplier Cumulative percentage change in GDP Cumulative percentage share of SFs in GDP

Evolution of accumulated SF injection (as % of GDP) and the accumulated percentage increase in the level of GDP: Czech Republic: NDP

Evolution of the Czech cumulative SF multiplier

Classifying performance: NDP Star performers: Czech Republic, Slovenia, Estonia, Poland, Portugal Average performers: Latvia, Romania, Spain, Hungary Under performers: East Germany, the Italian Mezzogiorno, Greece

What explains differences in outcomes? A common set of “implementation” and “effectiveness” parameters Nimble Small Open Economies? Estonia, Slovenia, Czech Republic Structures oriented towards growth (Polish “eagle”, Portugal) Need for a “bottom-up” analysis (measure => operational programme => CSF) Mix of measures vital; also institutional & organizational abilities

Some conclusions 1.Structural Funds, on their own, will never produce cohesion (for example, of the dramatic Irish variety) 2.However, returns to well-designed and effectively implemented NDPs are probably high 3.Micro-evaluation studies have not been systematic 4.The macro “testing” literature conclusions are probably overly negative and pessimistic 5.The HERMIN/QUEST macro-modelling studies and mechanisms may understate the potential for accelerated convergence

Towards a more constructive debate The Commission’s Cohesion Reports need to draw on available analytical research (micro and macro), even when critical Empirical approaches (“testing” and “impact evaluation”) can always be improved, but only examine limited aspects of cohesion Analysis needs to be broadened to include insights from industrial strategy and other policy frameworks (Vernon, Porter, Best),