1. NETLIPSE Infrastructure Project Assessment Tool Stuart Baker, Deputy Director of National Rail Projects Department for Transport, UK Vilnius, May 18, 2010

2. Contents  Background IPAT  IPAT – The assessment tool  IPAT – The assessment process – pilot cases  Lessons learned  The future of IPAT  Discussion

3. What is the context of IPAT?  Predecessor: Netlipse I research  One of the main deliverables of Netlipse II  Funding by the European Commission / TEN-T Agency in order to:  Improve the current management and organisation of LIPs  Get insight in ‘the vitality’ of projects on certain moments, e.g. financing (gate review)  Get better insight in the progress of LIPs  Benchmark projects

4. What is the IPAT?  The IPAT is an assessment tool  The IPAT indicates the level of confidence the IPAT assessors have that the PDO and its Client have created the necessary conditions for successfully executing the next project phase  The IPAT may be used by the European Commission /remark MHE/ as one of its tools for deciding what projects will receive their support.

5. When to apply the IPAT?

6. What is the added value of the IPAT?  Project delivery organisations: to increase the certainty of successful execution of projects, resulting in particular in reduced cost overruns and time delays, and;  Clients and funders: to understand the deliverability of projects by the project delivery organisations  EU, local governments and financial institutes: to monitor and evaluate projects (ex ante and ex post) in a systematic way  NETLIPSE and research institutes: the collection of information on research forecasts and future research demands.

7. Focus of the IPAT A Clear Context and Purpose, with a competent Organisation will bring the Required Results Organisation

8. IPAT – Conceptual model and themes Project Delivery Organisation Client / official sponsor T7 T3 T10T9T8 T2 T11 Political context (Usually a Ministerial client) T1 T5 T6 T12 T4 T 1Political Context T 2 Objectives, Purpose and Business Case T 3 Functional Specifications T 4 Interfaces T 5 Stakeholder Management T 6 Finance T 7Legal procedures T 8Technology T 9Knowledge T10Organisation & Management T11Contracting T12 Risks Stakeholders Private companies (contractors, advisors, operators, etc.) Related projects

9. IPAT – model: Project Phases Phases of the Project (relate to Milestones): M 1Initiation of the project M 2Funding assembly M 3Official approval official planning authority M 4Start of execution M 5Completion M 6Start operation M 75 years after start of operation

10. M 1M 2M 3M 4M 5M 6M 7 T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T10 T11 T12 Levels of Importance by Milestone and by Theme Prioritisation: Level 1: Minimal importance Level 2: Little importance Level 3: Medium importance Level 4: Important Level 5: Crucial

11. IPAT – The assessment tool Three elements:  Questionnaire  Scoring model  Assessment process

12. A pilot assessment  EX ANTE EVALUATION  Femern Bælt, Denmark

13. A pilot assessment: Femern Bælt, Denmark  STEP 1: Search for background information

14. A pilot assessment: Femern Bælt one of a long term series of key Danish links Great Belt Øresund Fehmern Bælt

15. 19 km coast to coast connection (Bridge or Tunnel?)  Bridge  Tunnel

16. Project Organisation  Femern A/S: subsidiary of Sund & Bælt

17. Budget & Financing Budget (prices as of 2008) a cable-stayed bridge: 4.5 billion Euro an immersed tunnel: 5.5 billion Euro hinterland infrastructure on the German side: 840 million Euro (cost as of 2003). hinterland infrastructure on the Danish side: 1 billion Euro Financing70-75%: by Danish loans, to be refinanced by toll income 25-30%: European Commission (TEN-T) funds for the period from 2007 to 2013 of 339 million Euro. For the period from 2014 to 2020, EU support will be requested

18. Planning Present stage of project.Investigations: - route corridor - geotechnical - maritime - environmental Time of decision to build, i.e., the go/no-go decision 2012: Anticipated presentation of a construction law in the Danish Parliament Time of start of construction2013: start of construction phase for the coast-to-coast segment Time of delivery2018 Commencement of operations End of 2018

19. A pilot assessment: Femern Bælt, Denmark  Step 1: Search for background information  Step 2: Site visit - interviews

20. Site visit  Assessors team:  Stuart Baker  Marcel Hertogh  Ellen Gehner  Interviewees:  Peter Lundhus – Managing director Femern A/S  Steen Lykke – Project director Tunnel  Open interview on 12 themes  Duration: 2 x 2 hours

21. A pilot assessment: Femern Bælt, Denmark  Step 1: Search for background information  Step 2: Site visit – interviews  Step 3: Scoring

22. IPAT- Questionnaire (Theme 1) 1Political contextM1 1 Is there a political consensus about the need and general purpose of the project?0 2 Is there a political consensus on the defined outcomes from the project?0 3 Is there a political consensus on the type of infrastructure solutions proposed by the project?0 4 Is there a political consensus on the defined route of new infrastructure?0 5 Are there clear interfaces between the politicians and the clienting of the project?0 6 Is there political consensus on the funding solutions?0 7 Are there arrangements to handle any political interfaces, including differences of view between the various levels of (inter)national, regional and local, city level?0 8 How is the relation between the Sponsor/Client and the politics/ and how do you handle differences?0 Total score (Σ score 1-8)0 Average score (Total score / number of questions)0,0

23. Scoring process  Score all questions in the IPAT Questionnaire  on a scale from 1 to 4  individually  Questions that are not relevant to a project phase or not applicable to a project must be indicated as such.  Note relevant arguments and comments that support the scoring as input for the assessment report  Organise a meeting with the assessors team to create consensus about the scores

24. Scores  The objective of scoring the criteria is to identify the main strengths and weaknesses of the project and its organisation within each theme.  Scoring table reflects weakness of the project (low score) and strength of the project (high score): ScoreQualificationThis reflects Score 1Very negative effect on the project being successful a vital need to improve immediately Score 2Negative contribution to a successful projectan need to improve a weak area Score 3Positive contribution to a successful projectFurther progress can be made to achieve a better result Score 4Very positive contribution to a successful projecta clear strong area for the project (probably close to best practice)

25. IPAT – Scoring model PROJECT NAMEPROJECT PHASE: M1 ThemeWeightScoreMax. scoreWeighted scoreMax. weighted score 1 Political Context3,0…4,0= Weight x Score= Weight x Max. score 2 Scope3,6…4,0““ 3 Functional Specifications4,8…4,0““ 4 Interfaces4,8…4,0““ 5 Stakeholder Management4,2…4,0““ 6 Finances4,2…4,0““ 7 Legal Procedures3,6…4,0““ 8 Technology4,4…4,0““ 9 Knowledge4,6…4,0““ 10 Organisation and Management5,0…4,0““ 11 Contracting4,8…4,0““ 12 Risks4,4…4,0““ = Total Weighted Score = Total Maximum Weighted Score Total Weighted Percentage Score = Total Weighted Score / Total Maximum Weighted Score

26. IPAT – Scoring model PROJECT NAMEPROJECT PHASE: M2 Theme WeightScoreMax. scoreWeighted scoreMax. weighted score 1 Political Context4,03,34,013,016 2 Scope4,92,84,013,419,6 3 Functional Specifications4,23,54,014,716,8 4 Interfaces1,91,14,02,17,6 5 Stakeholder Management2,41,04,02,49,6 6 Finances4,13,84,015,416,4 7 Legal Procedures3,62,04,07,214,4 8 Technology3,43,14,011,013,6 9 Knowledge2,83,04,08,311,2 10 Organisation and Management3,32,84,09,413,3 11 Contracting2,61,34,03,3510,4 12 Risks4,23,04,012,616,8 = 112,9= 165,7 Total Weighted Percentage Score= 68,1%

27. A pilot assessment: Femern Bælt, Denmark  Step 1: Search for background information  Step 2: Site visit – interviews  Step 3: Scoring  Step 4: Analysis of the results

28. Result 1: Weight – score matrix (sub)themes

29. Result 2: Overall score total weighted percentage score

30. A pilot assessment: Femern Bælt, Denmark  Step 1: Search for background information  Step 2: Site visit – interviews  Step 3: Scoring  Step 4: Analysis of the results  Step 5: Writing the assessment report

31. Two pilot assessments  MONITORING / EX POST EVALUATIONS  Gotthard Base Tunnel  West Coast Main Line (Pre 2002)

32. Overall score: Total percentage score West Coast Main Line – Failed Project Pre 2002

33. (Sub)theme scores: Weight – score matrix West Coast Main Line – M1

34. (Sub)theme scores: Weight – score matrix West Coast Main Line – M1, M2

35. (Sub)theme scores: Weight – score matrix West Coast Main Line – M1, M2, M4

36. Overall score: Total percentage score Gotthard Base Tunnel

37. (Sub)theme scores: Weight – score matrix Gotthard Base Tunnel – M1

38. (Sub)theme scores: Weight – score matrix Gotthard Base Tunnel – M1, M2

39. (Sub)theme scores: Weight – score matrix Gotthard Base Tunnel – M1, M2, M3

40. (Sub)theme scores: Weight – score matrix Gotthard Base Tunnel – M1, M2, M3, M4

41. Lessons learned from (pilot) pilots  Validation and fine-tuning of the questionnaire  Questionnaire was comprehensive  Questions were relevant  Questions were recognisable for the interviewees  Questions could be answered by the interviewees  Some questions are reformulated  Development of the scoring methodology  The assessors encountered little problems in scoring the questions  The assessors were generally unanimous about the scoring  The methodology of calculating the overall score was considered and changed ahead of the pilots  Significance of levels of importance appears to be little  Development of the final analysis of the IPAT:  Still to do: what are appropriate ‘pass scores’???

42. What’s next until end 2010?  Pilots still to undertake:  Lock - Flanders (June 2010)  French Project possibly Figueras - Perpignan (Sep 2010)  Koper – Ljubljana (June 2010)  Operational version of the IPAT (including assessors manual)  List of competent assessors  Training programme

43. Discussion (1)  Role of the assessors during the assessment process: how to guarantee the quality of the assessment?

44. Discussion (2)  Pass score: what is a ‘good enough’ project?  (sub)theme score – position of green area???  Overall score > 50% ??? Bandwidth?  Extra hurdle: the overall score might be considered good enough if on each (sub)theme level the score is in or above the green area???

45. Discussion (3)  Added value of the application of IPAT: what makes the IPAT attractive to Clients and PDOs to do an assessment?

46. Time for your input now please... Having heard all this, do you broadly understand the IPAT and are you now convinced that you we can do a reliable, fair and comparable assessment of Projects with the IPAT?