NETLIPSE Infrastructure Project Assessment Tool Stuart Baker, Deputy Director of National Rail Projects Department for Transport, UK Zagreb, November 10, 2009

Motivation (1) The European Commission and member states: address the need for a Trans European Transport Network (TEN-T) have already invested billions in the construction of several Large Infrastructure Projects (LIPs) to create the TEN-T and to deliver infrastructure for economic gain. But, these organisations: have limited possibilities for forecasting and monitoring the effectiveness of these projects; face large delays and cost overruns on the supported projects and experience local opposition; are aware that knowledge exchange between Large Infrastructure Projects (LIPs) is scarce.

Motivation (2) These organisations have a need for: Improvement of the current management and organisation of LIPs. Insight in the vitality of projects on certain moments, e.g. financing (gate review): to have a reliable insight in risks and opportunities before deciding; and if decision is go: as a basis to manage risks and opportunities As well as the allocation of budgets to the most vital projects to also take into account which projects are most likely to deliver results and stick to programme. Better insight in the progress of LIPs (risks, opportunities). Benchmark projects.

The objective of the IPAT The IPAT is a tool that can assess, monitor, benchmark, and evaluate project organisations of large infrastructure projects before, during, and after implementation in a competent and uniform way. In this way it also gives guidelines to a project organisation on the crucial factors in managing large infrastructure projects.

IPAT Infrastructure Project Assessment Tool

IPAT-Assessment – Assessing when? 1.Ex ante evaluation, go/no go-decision To indicate strengths and weaknesses of the project delivery organisation in advance. 2.Monitoring, during implementation: To indicate the abilities of the project delivery organisation during implementation. 3.Ex post evaluation To expand the knowledge on the project delivery organisations approaches of project planning and implementation (generates a comparative perspective on weaknesses and strengths on different implementation strategies). 4.Benchmark To create the opportunity to benchmark large infrastructure projects in different stages.

IPAT-Assessment – Whos involved?

IPAT - Criteria The scientific criteria for the IPAT: 1.Practical - Is the data available? 2.Reliable - Is the assessment carried out in a consistent manner? 3.Valid - Are all aspects measured that are supposed to be measured? Are the Assessors qualified? A practical criteria: 4.Applicability - Is the outcome understandable for clients and projects in a way that they can benefit from the conclusions and recommendations?

2.IPAT – Results The IPAT will allow: Project delivery organisations to increase the certainty of successful execution of projects, resulting in particular in reduced cost overruns and time delays, and; Clients and funders to understand the deliverability of projects by the project delivery organisations,and; EU, local governments and financial institutes such as The EIB and The World Bank to monitor and evaluate projects (ex ante and ex post) in a systematic way, and; the collection of information on research forecasts and future research demands.

IPAT- Building block (1)

IPAT - Building block (2) Interaction Control

IPAT - Building Block (3)

IPAT - Building Block (4)

IPAT - Assessment process

IPAT-Questionnaire - 12 Themes T 1 Political Context T 2 Objectives, Purpose and Business Case T 3 Functional Specifications T 4 Interfaces T 5 Stakeholder Management T 6 Finances T 7 Legal procedures T 8 Technology T 9 Knowledge T10 Organisation and Management T11 Contracting T12 Risks (Threats and Opportunities)

IPAT-Questionnaire – Project Phases Phases or Milestones: M 1Initiation of the project M 2Funding assembly M 3Official approval official planning authority M 4Start of execution M 5Completion M 6Start operation M 75 years after start of operation

M 1M 2M 3M 4M 5M 6M 7 T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T10 T11 T12 Levels of Importance by Milestone and by Theme Prioritisation: Level 1: Minimal importance Level 2: Little importance Level 3: Medium importance Level 4: Important Level 5: Crucial

IPAT-Questionnaire – Theme 1

Scoring Methodology - Criteria The objective of scoring the criteria is to identify the main strengths and weaknesses of the project and its organisation within each theme. Scoring table reflects weakness of the project (low score) and strength of the project (high score): ScoreQualificationThis reflects Score 1Very negative effect on the project being successful a vital need to improve immediately Score 2Negative contribution to a successful projectan need to improve a weak area Score 3Positive contribution to a successful projectFurther progress can be made to achieve a better result Score 4Very positive contribution to a successful project a clear strong area for the project (probably close to best practice)

Scoring Methodology - Themes The assessors will score the themes as follows: ΣImportance * Score criterion The pass score represents the score which the panel estimates an adequate but not brilliant project would get. The score for a theme should pass the pass score (ΣImportance * Score > pass score)

IPAT – Testing the IPAT Three pilot projects, tests in different project phases The objective of the pilot projects is to: 1 – validate and fine-tune the questionnaire 2 – further develop the scoring methodology 3 – further develop the final analysis of the IPAT and co consider the appropriate pass scores

Development of the IPAT – Delivery Completed – 2 pilot pilots to test the questionnaire Further steps: Q1, 2010: Three pilot projects Further development of the IPAT Q2, 2010: Evaluation of Pilot projects Further develop of the IPAT Q3, 2010: Finalize the IPAT Complete and deliver the IPAT Assessors manual. IPAT-Brochure IPAT-assessors Course.

IPAT – Your input! The IPAT needs to be widely understood and validated and improved Projects are needed to pilot the IPAT The IPAT needs buy in and acceptance by member states to achieve broad acceptance so that it may be used as an effective tool by the EC and other users The IPAT needs…...your experiences and input to make it a tool that will be used!

Time for your input now please... Can you really do a reliable and comparable assessment of Projects?