1 Ex post evaluation of the Cohesion Fund (including former ISPA) Work Package B: cost - benefit analysis of selected transport projects Jurate Vaznelyte, Evaluation Network Meeting Brussels, 20 October, 2011

2 Objectives of the study Cohesion Fund and ISPA (2000 – 2006): 254 individual transport projects co-financed Terms Of Reference for this study: 103 individual transport projects 10 schemes considered for ex post evaluation (38 individual projects out of 260 or 15%) Question 1: What were the impacts of these projects? Question 2: How can ex post CBA contribute to the practice of ex ante CBA? Question 3: What are the potential and limits of ex post CBA?

3 AVE Madrid- Barcelona A23 Motorway Lisbon – Algarve railway M1 Motorway IX B Corridor + Vilnius bypass A2 Motorway Bratislava Railway Upgrade M0 Motorway Agiou Konstantinou bypass Thriasso-Pedio-Eleusina- Korinthos railway Road Rail

4 Ex ante stated objectives Reduce travel time (6) Increase capacity/reduce congestion (4) Reduce operating costs (2) Improve safety (8) Improve connectivity (2) Projects

5 Ex ante impacts taken into account in CBA Impacts on safety (10) Impacts on the environment (4) Impacts on transport operators (3) Impacts on users (Time Savings, Vehicle Operating Costs) (10) Other impacts (2) Projects

6 Ex ante CBA – its role in decision making Compliance with EC CF application requirements (9) Ensure value for money (6) Choose alignment (2) Choose design standards (2) Prioritise elements of national transport strategy (1) Allocate budget and optimise timing (0) Projects

7 Ex post economic evaluation – benefit-cost ratio (BCR) Most projects show BCR larger than 1, i.e. the net project impact, as captured by the CBA is positive Due to high capital costs, the AVE shows a BCR < 1. However, the wider socio-economic impact (not captured in the CBA) are significant

8 Ex post financial analysis – financial NPV The financial NPV (before EC contributions) of seven of the ten projects is negative – not surprising as most projects do not generate direct commercial revenues. The fact that most projects have an economic BCR > 1, but a negative financial NPV indicates that that the EC contributions help unlock the economic benefits of these projects

9 Sources of benefits – railway projects

10 Sources of benefits – road projects

11 Other key findings Transparency and accountability The regular publication of ex post findings, whatever the outcome, generally increases transparency and accountability. It also help identify key learning points for future project planning Evidence base Feedback Average difference between ex ante and ex post was 13.4% - relatively low Factors explaining the difference: time lag, project delays, scope alterations and unforeseen circumstances Generally in line with objective to build in sufficient spare capacity to cover project lifetime. Two exceptions (one too high, one too low) Ex ante NPV generally higher than ex post: the main driver for discrepancy was the demand forecast Together with capital cost and utilisation rates findings, it is indicative of optimism bias Ex ante demand modelling would benefit from improvement in study area definition, modal coverage, inclusion of induced traffic and variable demand CBA practice would benefit from more consistency in scope, more granularity (e.g. business vs. non-work users) and harmonisations of assumptions, parameters and approaches Capital costs Utilisation rates NPV Modelling practice CBA practice

12 Key issues considered Strengths and weaknesses of CBA methodologies as applied by the Member States Effectiveness of CBA as a tool to support project generation and decision of Member States and Commission Utility of ex post CBA from the point of view of project promoters, Member States and the Commission Ex post CBA as a tool for evaluating project impacts Relevance and potential utility of CBA for macro-economic modelling

13 Strengths and weaknesses of CBA There is a basic level of consistency in CBA methodology used for transport infrastructure projects across EU Member States CBA framework used in the ten projects is both consistent with DG REGIOs guidelines and with good practice However Scope of the ex ante analysis sometimes narrower than what would be ideal (externalities not always considered) Different appraisal parameters used across EU Member States (e.g. HEATCO values) Weaknesses in demand modelling, counterfactual definition and modelling of network effects

14 Effectiveness of CBA in decision-making The current contribution of CBA in the decision making process is limited Mainly used to confirm projects value of money and to support funding application CBA results Regional development and environmental concerns CBA does not fully take into account other key impacts Time that would be required to fully embed CBA into planning process CBA offers more potential as decision-making tool TechnicallyPolitically CBA is strong in comparing projects with similar impacts. Could help in choosing design, alignment and in prioritization CAB can add transparency to the decision-making process and improves political accountability

15 Utility of ex post CBA Ex post CBA can add significant value to planning process Adds transparency Strengthens evidence base Provides feedback on methods and techniques used for ex ante design and appraisal 1 23 A key contribution of ex post evaluation has been a better understanding of optimism bias and its causes at the planning stage

16 Ex post CBA as a tool for impact evaluation Advantages Limitations Holistic approach Delivers an unambiguous indicator of the projects economic worth Considers both direct impacts (e.g. time savings) as well as associated externalities (e.g. safety benefits) Due to long life of infrastructure, ex post CBA should be carried out as late as possible Counter-balancing the above, there is a risk of institutional memory loss The analysis requires the definition of a counterfactual – for projects with a long expected economic life, this may be problematic 1 2 3

17 CBA relevance and utility for macro-economic modelling Ex post CBA covers a wider range of impacts than those focus of macro- economic models (i.e. non-work traffic savings, safety and environment) but It provides a net benefit indicator and does not indicate who actually and ultimately receives this benefit. It measures the added value of bringing new resources to use, while macroeconomic modeling measures the gross value. In practice Macroeconomic impacts are best modeled by linking transport accessibility models directly with macroeconomic models

18 Thank you for your attention Thank you for your attention