Benchmarking regional maintenance costs on England’s Strategic Road Network ETC 2017, Barcelona Adam Spencer-Bickle, Principal Economist, ORR

Overview Background on Road Reform, Road Investment Strategy and ORR’s role A brief introduction to econometric efficiency benchmarking analysis Data, results and conclusions from our initial analysis Given the stage of the analysis, the results do not represent ORR’s view of Highways England’s efficiency

Background on Roads Reform and the RIS

Roads Reform and the Road Investment Strategy ORR’s highways role “Our Highways role has been established to place a greater level of scrutiny on the company than has been the case in the past. We hold Highways England to account for its management of the strategic road network – including delivery of performance and efficiency. We also advise the UK Government on the levels of funding and performance requirements for future road periods to help frame challenging and deliverable performance and efficiency requirements.” Strategic Vision Investment plan Performance Specification Statement of Funds Available

RIS2 Efficiency Review provide advice… on whether… proposed requirements are deliverable with the proposed financial resources, and the extent to which the Draft SBP is challenging and deliverable, including with regard to levels of efficiency Capability Reviews of Highways England’s: 1. procurement and contract management 2. project and programme management 3. asset management Benchmarking studies with an analytical focus on: 1. Regional intra-company comparisons 2. Comparisons to other UK regulated sectors Sample testing the draft SBP to assess whether individual expenditure lines are supported by robust analysis about efficient costs Bringing the evidence together with a review of the efficiency improvements built into SBP, an assessment of efficiencies achieved in Road Period 1 and evidence from phases 1 to 3 of our work

Overview of the programme of work to date Capability reviews and sample testing provide a more ‘bottom-up’ assessment of efficiency. So benchmarking focusing on: more ‘top-down’ methods for efficiency, as a sense check; and performance. Efficiency Performance

Brief intro to econometric efficiency analysis

Econometric efficiency benchmarking Efficiency gap

Econometric efficiency benchmarking 1. Estimate the cost function: 2. Establish the efficient frontier (minimum cost): 3. Measure the efficiency gap: COLS is very simple – all cost difference treated as inefficiency SFA models split the error term into uit (inefficiency) and vit (noise or unobserved heterogeneity) 𝑙𝑛𝐶 𝑖𝑡 =𝛼+ 𝛽 1 𝑙𝑛 𝑄 1𝑖𝑡 + 𝛽 2 𝑙𝑛 𝑄 2𝑖𝑡 + 𝜀 𝑖𝑡 𝑙𝑛 𝐶′ 𝑖𝑡 = 𝛼 +𝑚𝑖𝑛 𝜀 𝑖𝑡 + 𝛽 1 𝑙𝑛 𝑄 1𝑖𝑡 + 𝛽 2 𝑙𝑛 𝑄 2𝑖𝑡 + 𝑢 𝑖𝑡 𝑒𝑓𝑓 𝑖𝑡 = min 𝑐𝑜𝑠𝑡 𝑎𝑐𝑡𝑢𝑎𝑙 𝑐𝑜𝑠𝑡 = 𝑒 𝛼 𝑄 1𝑖𝑡 𝛽 1 𝑄 2𝑖𝑡 𝛽 2 𝑒 𝛼 𝑄 1𝑖𝑡 𝛽 1 𝑄 2𝑖𝑡 𝛽 2 𝑒 𝑢 𝑖𝑡 = 1 𝑒 𝑢 𝑖𝑡 =𝑒𝑥𝑝 − 𝑢 𝑖𝑡

Data, results and conclusions from initial analysis

Comparing the potential data sources Highways England’s Performance Monitoring Statements: Maintenance Renewals General operations Traffic management PFI payments Support costs Capital M&R By region, with separate “centrally managed” category From 2015-16 Highways Agency cost of maintaining the network KPI: All renewal of roads and structures Proportion of MAC routine and winter maintenance spending Proportion of PFI/DBFO payments All technology maintenance and renewal spending By region (with no “centrally managed” category) Consistent lane and vehicle (from an accompanying KPI) miles From 2010-11

Comparing the potential data sources Also sourced data for Scotland and Wales Balanced panel of 8 “i’s” and 7 years Lane miles and vehicle miles as only candidate cost drivers

Developing the cost function Preferred model and estimation results: Need to include Scottish and Welsh data uncertainty about cost comparability so tested a range model performs better with “low” cost data. Lane elasticities = 1 => constant returns to scale Vehicle density elasticities in line with existing evidence Link et al (2008) found a range of 0.6-1.0 for renewals / maintenance and renewals 𝑙𝑛𝐶 𝑖𝑡 =𝛼+ 𝛽 1 𝑙𝑛 𝐿𝑎𝑛𝑒 𝑖𝑡 + 𝛽 2 𝑙𝑛 𝑉𝑒ℎ𝑖𝑐𝑙𝑒𝐷𝑒𝑛 𝑖𝑡 + 𝑡𝑖𝑚𝑒 𝑡 + 𝜀 𝑖𝑡 Lane miles Vehicle density ‘pure' lane miles S&W ‘high’ S&W ‘low’ COLS 1.56* 1.42* 0.26 0.51* 1.30* 0.91* PSFA 1.92* 1.86* 0.38 0.73* 1.54* 1.13* P&L(81) 1.78* 1.68* 0.26* 0.54* 1.52 1.14* RE 1.60* 1.44* 0.39* 0.61* 1.21* 0.83* CSSRE 1.35* 0.33 0.53* 1.23* 0.82* Cuesta 1.84* 1.73* 0.24 0.76* 1.60 0.97*

Efficiency scores Given the stage of the analysis, the results do not represent ORR’s view of Highways England’s efficiency Average theoretical efficiency scores across Highways England regions Model 2010-11 to 2016-17 S&W ‘high’ S&W ‘low’ COLS NA 84% 82% Pooled SFA 80% 79% Pitt and Lee (’81) 78% Random Effects CSSRE 75% 73% Cuesta

Conclusions and development priorities Cost function and aggregate efficiency scores appear plausible Priorities for development include: Consistency of Scottish and Welsh data Disaggregating lane and vehicle mile variables Other measures of network ‘quality’ Potential to widen the panel

Questions and contact details ? Adam.spencer-bickle@orr.gsi.gov.uk