Learning the lessons 2012 and 2014 procurements of audit services

Introduction The Audit Commission undertook post-implementation reviews after its final two procurements of audit services: –2012 outsourcing (70% of the market) –2014 procurement (remaining 30% of the market) This slide pack takes the lessons learnt and summarises them as a list of factors that contributed to the delivery of successful outcomes for both procurements. These success factors may be helpful in generally informing future procurements of audit services undertaken by individual local public bodies or collective procurement vehicles under the Local Audit and Accountability Act However, it should be noted that the procurements undertaken by the Commission were unique to the Commission’s regime and some of the approaches taken may not necessarily be relevant to other procurements.

Background to the procurements 2012 outsourcing Covered 750 audited bodies Total value £89 million Ten lots awarded to four firms Savings achieved 40% 2014 procurement Covered 260 audited bodies Total value £25 million Five lots awarded to three firms Savings achieved 25%

Success factors The following four factors were key to the Commission achieving successful outcomes from its 2012 and 2014 procurements: –Robust project governance; –Thorough understanding of the market; –Effective quality assurance; and –Structured procurement process. Local public bodies, or collective procurement vehicles, undertaking procurements of audit services in the future may wish to note these success factors and consider how they could be adapted to suit their particular circumstances. The following slides take each factor in turn and explain how and why it contributed to the overall success of the procurements.

Success factor 1: Robust project governance CharacteristicsSuccessful outcomes Delegation by the Commission Board of strategic oversight of the process to a Procurement Panel made up of a small number of Board members with relevant sector and commercial experience. The Panel was chaired by the Commission's Deputy Chairman. The Panel was able to engage effectively with the detail of the process and oversee the project management arrangements, through interaction with the executive project board, in a way that would have been difficult for the full Board. The Commission Chairman was able to remain at arms length and lead the Board’s consideration of the Panel’s recommendations on the award of contracts. Application of formal project management techniques and a robust approach to risk management. The procurement was run as a PRINCE2 project with a dedicated project manager. This ensured that every key milestone was achieved and all risks were monitored and managed effectively. Use of expert external legal advice and specialist procurement support. The provision of high-quality legal advice and procurement support from an expert firm helped to mitigate effectively the significant legal risks attaching to both procurements. Clarity about the aims, objectives, timetable and process for the procurement set out in a high- level published strategy document. The development and publication of a procurement strategy ensured transparency and a level playing field for all interested parties.

Success factor 2: Thorough understanding of the market CharacteristicsSuccessful outcomes The internal teams on both procurements comprised long-serving Commission staff with extensive knowledge of the Commission’s audit regime and of the public audit market. An in-depth understanding of the Commission’s audit regime and of the market enabled both procurements to be designed efficiently and effectively to maximise the likelihood that the stated aims and objectives would be achieved. Exploiting the Commission’s bulk purchasing power to ensure competitive prices. The contract lots were packaged appropriately to ensure competitive prices. Lots were packaged on a geographical basis so that firms knew the general location of the work they were bidding for. There were ten lots in 2012 ranging from £12.5m to £5m, and five lots in 2014 all of which were around £5m. A lot size of £5m was considered by the Commission to be the minimum to attract competitive prices. Packaging lots to ensure enough firms would win a contract to enable independence issues to be effectively managed. Managing independence issues is a key aspect of the Commission’s appointments process. There needs to be enough firms with a contract to enable an alternative firm to be proposed in the event of a firm being conflicted. To achieve this, the Commission applied a bidding rule that a firm could only win one lot in each geographical region. From 2015/16, there will be only five firms with contracts which in the Commission’s view is the minimum to effectively manage independence issues across its regime..

Success factor 3: Effective quality assurance CharacteristicsSuccessful outcomes Effective internal and external quality assurance (QA) and challenge built in to the process. A layered approach was adopted to quality assurance comprising challenge to the PQQ and ITT evaluation teams from an internal QA team, internal audit review, and external gateway review by DCLG. The different reviews provided the project board, Procurement Panel, Commission Board and DCLG with independent assurance that: key aspects of the projects were being managed effectively; the projects were on track to deliver their objectives; and the information on which key decisions were based was accurate and reliable. Use of a small, expert evaluation team to review the quality aspects of the tenders independently, supported by a process for agreeing consensus scores. The evaluation team reviewed and scored the quality aspects of the tenders without knowledge of the prices so they could not be unduly influenced. Scores were discussed and challenged at a consensus meeting, facilitated by the project manager, at the end of which a set of consensus scores were agreed.

Success factor 4: Structured procurement process CharacteristicsSuccessful outcomes Clear pre-qualification questionnaire (PQQ) and invitation to tender (ITT) documentation, including: PQQ criteria and threshold that enabled a range of firms to bid, while ensuring that only those meeting the Commission's requirements were invited to tender; and imposition of word limits on responses to quality questions in the ITT. The PQQ and ITT documents were clearly set out and accessible. The PQQ criteria and threshold resulted in 13 firms in 2012 and nine firms in 2014 proceeding to the ITT stage, which meant a competitive tender process in both cases. Word limits placed a premium on clarity and ensured focused responses from tenderers which could be evaluated efficiently. Any words over the word limit were redacted and excluded from the evaluation. 60:40 weighting given to price relative to quality and the price scoring methodology. The 60:40 weighting given to price over quality ensured good quality but not ‘gold plated’ audits. The price scoring methodology deducted three points for every percentage point that price varied from the cheapest bid and provided an incentive for tenderers to bid competitive prices. Clear contract specification provided as part of tender process. A draft contract was provided as part of the tender process which enabled agreement of contract terms before the tender submission deadline. This also enabled the contracts to be prepared for signing relatively quickly after contract award.