1. (HERLIT model based on HERMIN)
Evaluation of the Impact of the EU Structural Funds on Gross Domestic Product
(HERLIT model based on HERMIN)
Ministry of Finance of the Republic of Lithuania
Presentation for
DIRECTORATE-GENERAL REGIONAL POLICY –
"EVALUATION NETWORK MEETING"
Brussels, 25 and 26 February 2010
Jonas Jatkauskas,
BGI Consulting, Public Policy Division
Dr. John Bradley,
Economic Modelling and Development Strategies (EMDS),

2. Aims of the Evaluation
General aim – to evaluate the impact of the EU Structural Funds on the Gross Domestic Product (GDP) of Lithuania. Firstly the impact of the SPD and secondly preliminary impacts of the OPs.
Tasks:
Economic overview;
To assess the level of achievement of the SPD key indicators (Increase of GPD according to the initial scenario);
Provision of recommendations for the period of (GDP increase emphasized).

3. Evaluation Criteria
Relevance – relevance of the Programme indicator “Real increase of GDP according to the initial scenario” was assessed.
Effectiveness – the level of achievement of the key SPD indicator “Real increase of GDP according to the initial scenario” was evaluated.
Impact – the impact of SPD on the economy and separate sectors was evaluated. This included evaluation of long term social and economic impacts of the investment.
Efficiency – the comparative analysis of efficiency of SPD investment was carried out. This included comparison of “cumulative” multipliers among convergence member states.

4. Methods of Evaluation
The central tool of the evaluation – Economic macro-model HERLIT.
HERLIT model is based on HERMIN methodics which was widely used in other evaluation exercises of member states.
The HERLIT model was created and applied by the key expert of the evaluation – dr. John Bradley (EMDS).
Micro (bottom-up)
Macro (top-down)
Level of disaggregation
High (individual projects)
Low (sectoral aggregates, whole economy)
Use of theory
Weak (judgemental, CBA)
Strong (macroeconomics)
Model calibration
Judgemental/informal
Scientific(?)/econometric
Policy impacts
Informal/implicit/ranking/ some quantification
Formal/explicit/quantified
Treatment of externalities
Limited or ignored
Included/explicitly modelled

5. The key assumptions
 The key assumptions were made in carrying out the HERLIT simulations of the SPD impacts:
The baseline is the “with-SPD” scenario, because it is an ex-post evaluation.
 The values of the supply-side SPD spillover parameters are selected from the mid range of the international literature, as being reasonably representative of a standard design and implementation of the SPD.
 A later sensitivity analysis can be used to probe different assumptions, and seek out how the impacts might be improved.
 If the SPD were absent, no substitute purely domestic investment package would be substituted.
 When SPD terminates in 2008, public investment is assumed to revert to the no-SPD situation. SPD evaluated in isolation from OPs)
 A preliminary ex-ante evaluation of combines both periods ( and ) together. 5

6. Stages of Evaluation (1)
Stage 1: Preparation of the baseline Lithuanian macroeconomic scenario.
This involves the preparation of a baseline projection for the Lithuanian economy to be used as an ex-post reference point for the subsequent analysis of the SPD policies. The period covered will extend to about the year 2015 in the case of SPD and to 2020, in the case of SPD
Stage 2: Preparation of the Lithuanian EU Funds data for use in the impact analysis
The SPD expenditure data was reclassified into the three standard economic categories: physical infrastructure, human resources, and direct aid to business (including R&D assistance).

7. Stages of Evaluation (2)
Stage 3: Applications of the model to Lithuanian SPD impact analysis
The new HERLIT model was applied to the impact analysis of the Lithuanian SPD , both in terms of aggregate effects and more detailed sectoral effects. In addition, an exploratory examination of the likely impacts of SPD was carried out.
Stage 4: Documentation of results of SPD impact analysis
The SPD impact results were documented, and the policy conclusions related to the wider socio-economic analysis carried out. The results of ex-post evaluation was compared with those of the ex-ante evaluation carried out in 2003 (particularly measuring the level of achievement of the key SPD indicators as GDP increase and jobs created).

8. Results of Evaluation: Input
SPD – Size of funding injection
Starting from a very low base in 2004 (less than 0.1 percent of GDP), the SPD expenditure build up, and peak in the final year 2008 at over 1 per cent of GDP (EU) and 1.5 per cent of GDP (total). This pattern of expenditure suggests that there were difficulties in initiating the SPD programmes, and the back-loading was a consequence. It should also be remembered that these two ratios use actual GDP, and that economic growth in Lithuania was very high during the years of SPD implementation. This relatively reduced the size of the SPD expenditures as a share of GDP.
Starting from a very low base in 2004 (less than 0.1 percent of GDP), the SPD expenditure build up, and peak in the final year 2008 at over 1 per cent of GDP (EU) and 1.5 per cent of GDP (total).

9. Results of Evaluation: Effectiveness
SPD – Impact on the level of GDP
Figure shows the impact of SPD on the level of real GDP. It shows the impact on the level of GDP, and not on the growth rate. The initial impact in 2004 is very small, and raises the level of GDP by only 0.07 per cent. The impact builds up over time, and peaks in the terminal year 2008 at an increase of over 2.1 per cent. Thus, in the year 2008 the level of GDP was over 2.1 per cent higher than it would have been if there had been no SPD.
Although the impact on the level of GDP declines dramatically from the 2008 peak, it does not go to zero. In the year 2009 (the first after termination of programme), the increase in the level is just under 0.6 per cent, and in the long run GDP remains higher by 0.13 per cent.
It can be summarised that SPD effectiveness goal (increasing GDP) set in the Programme (SPD) was achieved and slightly exceeded initial estimations (by 0.3 per cent points).
The initial impact in 2004 is very small, and raises the level of GDP by only 0.07 per cent. The impact builds up over time, and peaks in the terminal year 2008 at an increase of over 2.1 per cent.
It can be summarised that SPD effectiveness goal (increasing GDP) set in the Programme (SPD) was achieved and slightly exceeded initial estimations (by 0.3 per cent points). 9

10. Results of Evaluation: Impacts (1)
SPD – Impact on unemployment rate
SPD impact on unemployment rate was app. -0,3 % points in 2005, -0,8 % points in 2006, -1,17 % points in 2007 and peaked in 2008 by reducing unemployment by -1,5 % points. It can be stated that in no-SPD scenario unemployment rate would be 7,3 % in 2008, while with SPD it actually was 5,8 % in Lithuania.
SPD impact on the unemployment rate was app. -0,3 % points in 2005, -0,8 % points in 2006, -1,17 % points in 2007 and peaked in 2008 by reducing the unemployment by -1,5 % points. It can be stated that in no-SPD scenario the unemployment rate would be 7,3 % in 2008, while with SPD it actually was 5,8 %. 10

11. Results of Evaluation: Impacts (2)
SPD – Impact on sectoral output
The impacts on GDP in four sectors: manufacturing (Red line), market services (Green), building and construction (Blue) and public services (Violet) is shown in the figure. The most dramatic impact is on building and construction, where the increase in the level of real GDP peaks at about 7 per cent in the year 2008, and declines quickly to zero after termination. The implementational impacts on market services is also significant, peaking at about 2.3 per cent in 2008, and declining rapidly thereafter. The impacts on manufacturing GDP are slow to build up, peak at just under 0.75 per cent in 2010, and continue into the medium term. The impacts on public services GDP are modest and are affected by minor price and wage changes. 11

12. Results of Evaluation: Impacts (3)
SPD – Impact on sectoral productivity
Figure shows the impacts on labour productivity in the economy as a whole (Red line), in manufacturing (Green) and in market services (Blue). The key modernisation goal of the SPD is to enhance the performance and competitiveness of manufacturing, thereby generating increased export earnings and improved import substitution possibilities in the medium term. The effects shown in Figure, show this process in action. 12

13. Results of Evaluation: Impacts (4)
SPD – Impact on sectoral balances
Because of SPD impacts Lithuanian trade balance deteriorated by over 1.2 percentage points at most in the year 2008 (trade balance is expressed as a percentage of GDP). This deterioration occurred during the implementation years Immediately after the termination of the programme at the end of 2008, the trade balance deterioration caused by absorbing the SPD expenditure vanishes, and the improved supply potential of the economy means that there is a modest improvement in the trade balance of about 0.2 per cent of GDP in the post programme period.
The public sector balance is defined as a borrowing requirement (GBORR, measured as a percentage of GDP). It decreases during the implementation years, by about 0.4 per cent of GDP in the peak year 2008, and moves back to the original balance thereafter. This suggests that the EU SPD financial support generates sufficient extra revenue buoyancy to more than offset the need to raise the domestic public co-finance. 13

14. Results of Evaluation: Input
SPD plus OPs – Size of funding injection
injection
injection
Evaluating impacts of both and periods.
The SPD funding injection is relatively small, averaging 0.6 per cent of GDP per year for the EU contribution and 0.9 per cent per year for the total contribution (EU plus domestic public co-finance). The Programme is the first in which Lithuania fully participates, and as shown in Figure, the expenditures are considerably greater than during the SPD programme.
According to the planning data provided by the Ministry of Finance, the SPD expenditure peaks in 2011 at just over 4 per cent of GDP (EU element) and at just over 5 per cent of GDP (total expenditure). After the year 2011 the planned SPD expenditures decreases, and are just over 1 per cent of GDP in the terminal (“n+2”) year 2015. 14

15. Results of Evaluation: Impacts (5)
SPD plus OPs – Impact on the level of GDP
Figure shows the impacts on aggregate GDP, where both Programming periods overlap for the years 2007 and The impact on the level of GDP peaks in the year 2012, at just under 9 per cent. This declines to 5.7 per cent by the terminal year, 2015, after which the SPD expenditures are assumed to go to zero. After 2015 there remains a significant long-term increase in the level of GDP relative to the no-SPD scenario. By the year 2020, this is still about a 3 per cent increase, but would gradually decline as the new stock levels depreciate over time. 15

16. Results of Evaluation: Impacts (6)
SPD plus OPs – Impact on unemployment rate
The impacts on the unemployment rate are shown in the Figure. At its peak in 2011, the unemployment rate is reduced by almost 5 percentage points compared to the no-SPD scenario. The long-term reduction is about 1 percentage point. But it should be noted that the competitive performance of the economy is greatly enhanced, and it can benefit from other opportunities such as increased attractiveness to foreign direct investment and stronger indigenous firms. 16

17. Results of Evaluation: Impacts (7)
SPD plus OPs – Impact on sectoral output
Figure shows the much larger impacts of the combined Programmes on sectoral output. The increase in the level of building activity peaks at over 30 per cent in the year 2011, but declines thereafter. The maximum boost to output in market services is 10 per cent, in the year Public sector output is hardly affected. The boost to the level of manufacturing output builds up gradually to peak at 5.2 per cent in 2015, and remains at over 4 per cent out to the year 2020. 17

18. Results of Evaluation: Impacts (8)
SPD plus OPs – Impact on sectoral productivity
The impacts on sectoral productivity in manufacturing, market services and in the aggregate economy are similar to those of SPD in isolation, but considerably larger. In the case of manufacturing, productivity increases by over 5 per cent in the later years of SPD , and remains higher by over 3 per cent into the longer term. 18

19. Results of Evaluation: Efficiency
Comparing “Cumulative” multipliers
“Cumulative” multiplier, is the cumulative percentage increase in the level of GDP due to SPD divided by the cumulative funding injection, where the latter is expressed as a percentage of GDP.
The results are compared among objective 1 (convergence) economies.
Countries are divided into three groups, based on a ranking by the size of the cumulative multipliers:
High values (above 3.0): IE (4.0), ES (3.3), CZ (3.3) and MT (3.1)
 Medium values ( ): SK (2.8), EL (2.8), EE (2.8), PT(2.6), PL (2.5)
 Low values (below 2.5): LT (2.4), HU (2.4), SI (2.2), CY (2.2), LV (1.9)
Source: Study for the European Parliament. Bradley, Untiedt and Zaleski, 2009. 19

20. Recommendations
Recommendations were directed towards increasing Programme investment impact on GDP.
Recommendations for The Ministry of Finance of Lithuania (MA) were set three-fold:
To strengthen the Programming phase by incorporating detailed forecasting and sophisticated economy analysis tools in to the process. This should ensure higher relevance of the investment to the needs of economy and society.
To strengthen Implementation phase of the programmes by invoking micro level analysis tools to increase effectiveness (like Cost Benefit Analysis). The results should be further utilized in order to revise the set of interventions.
To strengthen national and regional strategic planning by prioritizing the key needs of the economy (according to stages of competitive development). National strategic planning should be strongly related to the Programmes funded by the EU structural funds. 20

21. Potential uses of HERLIT model
Preparation of economic forecasts (short and medium-term);
General public policy analysis;
EU investment related policy analysis (Ex-ante, Mid-term, Ex-post)
Impacts of the other EU policies (Single market, EMU, etc.)
Design/evaluation of national industrial strategies 21

22. Thank you! Ministry of Finance of the Republic of Lithuania
BGI Consulting Ltd.,