PROPOSED CHANGES TO RATING METHODS Compatibility, Workload, Past Performance, and Category Weighting February 11, 2016

Purpose Provide consultant community and DOTD practitioners with an update on the progress of the DOTD/ACEC initiative Obtain feedback concerning the specific recommendations for implementation

Background In June 2015, DOTD Chief Engineer and ACEC BOD signed document approving recommendations for implementation Approved 35 recommendations for implementation from 57 in 6 categories (see next chart) Several (7) were pending, not ready Sub committee A (procurement) continued to work Many additional meetings (Jun –Dec 2015) Presented to steering committee on 28 Jan 2016 and approved to move forward with this briefing

DOTD/ACEC Initiative Status   Total Approved Pending Tabled Not Approved A. Procurement 22 6 7 2 B. Contracting 5 3 1 C. Contract Administration D. Quality Assurance/Control E. Claims F. Contract Provisions 18 57 35 10 4 8

DOTD-ACEC/L Round Table Group A: Procurement Committee of DOTD and Consultants Focus on consultant selection process Actively working for over 3 years Committee members: DOTD ACEC/L Ed Wedge Simone Ardoin Bob Boagni Charles Nickel Ray Mumphrey Bob Basinger Debbie Guest Masood Rasoulian Gordon Nelson Paul Vaught Lawrence Hamm Buddy Porta Heather Huval Connie Porter Nick Ferlito Jesse Rauser Suzanne McCain Hunter Lancaster

Goals Evaluate the existing consultant selection process and look for opportunities for improvement by: Ensuring that the process appropriately balances the desire to provide as many firms as possible with the opportunity to perform the work with the need to ensure that the most technically experienced firms are selected Increasing transparency by simplifying the process and making it less subjective and more consistent

Key Issues Compatibility and Workload scoring methodologies are unnecessarily complex Determine alternate approach to evaluate those categories Create and define new input variables Project Magnitude Firm Size Designation Better address how past performance scores for new firms are assigned

Key Issues Rating categories can be divided into “experience” and “non-experience” ratings “Experience” Categories: Staff Experience, Firm Experience, Past Performance “Non-Experience” Categories Compatibility, Workload, Location Greater variation between consultants in the “non- experience” categories leads to their over-weighting in the selection process

PROJECT MAGNITUDE

Project Magnitude Project Magnitude is proposed to be determined by: 1. Complexity 2. Contract Time or Schedule (Current Solicitation) 3. Amount of the Contract (Projected for all Phases/Stages) 4. Route Classification.

Project Magnitude Criteria Value 1 2 3 Criteria Complexity Simple Medium Complex Contract Time Typical Compressed Critical Contract Amount ≤ $250,000 $250,000 to $2,500,000 ≥ $2,500,000 Route Classification Non-NHS Local Non-NHS State NHS

Project Magnitude Project Magnitude Designation Combined Criteria Values Micro 4 to 5 Small 5 to 7 Medium 7 to 9 Large 9 to 11 Mega 11 to 12

FIRM SIZE

Number of Transportation Personnel Firm Size Firm Size designation is determined using the table below: Firm Size Designation Number of Transportation Personnel Micro ≤ 13 Small 10 to 22 Medium 20 to 33 Large 30 to 55 Mega ≥ 50

COMPATIBILITY RATING

Compatibility Rating Once the project magnitude and firm size have been determined, the proposed compatibility rating* can be obtained by the following table. *When a team is considered, the compatibility rating will be based on the prime consultant only.

Project Magnitude Designation Compatibility Rating   Project Magnitude Designation Micro Small Medium Large Mega Firm Size Designation 5.0 3.0 1.0 4.0 2.0

WORKLOAD RATING

Workload Rating Once the firm size and remaining DOTD work amount have been determined, the proposed workload rating can be obtained by the following table.

Project Magnitude Designation Workload Rating   Project Magnitude Designation 5.0 5.0 → 1.0 1.0 Remaining DOTD Work Firm Size Designation Micro ≤ $20,000 $20,000 to $1,625,000 ≥ $1,625,000 Small ≤ $110,000 $110,000 to $2,750,000 ≥ $2,750,000 Medium ≤ $210,000 $210,000 to $4,125,000 ≥ $4,125,000 Large ≤ $320,000 $320,000 to $6,875,000 ≥ $6,875,000 Mega ≤ $530,000 $530,000 to $12,500,000 ≥ $12,500,000

PAST PERFORMANCE RATING

Past Performance Existing: Firms who have not received a rating for a work category will be assigned a rating equal to the lowest of the following: The average rating of the firms submitting The statewide average rating for that category as of the date the advertisement was posted.

Past Performance Proposed: Firms who have not received a rating for a work category will be assigned a rating equal to the lowest of the following: The average rating of the firms submitting The statewide average rating for that category as of the date the advertisement was posted The satisfactory rating of 3.

CATEGORY WEIGHTING

Category Weighting The score is based on six categories: Firm Experience Staff Experience Past Performance Compatibility Workload Location

Category Weighting Three of the six categories can be thought of as “experience” based and are generally associated with technical ability to perform the work. Firm Experience Staff Experience Past Performance

Category Weighting The other three categories can be thought of as “non-experience” based and are generally associated with the distribution of opportunity for work. Compatibility Workload Location

Category Weighting The “experience” ratings are currently weighted as follows: Firm Experience 3 Staff Experience 4 Past Performance 6 Total: 13

Category Weighting The “non-experience” ratings are currently weighted as follows: Compatibility 3 Workload 5 Location 4 Total: 12

Category Weighting Although the “experience” scores are higher, the lower “non-experience” scores dominate the ranking, simply due to their larger variations.

Category Weighting This is a known consequence of the current system. In some cases, such as complex projects, where technical experience based selection is of critical importance, current practice is to neutralize some of the “non-experience” scores to help alleviate this issue.

Category Weighting Based on a sample set of contracts, the “experience” scores need to be weighted more to fully compensate for the larger variations typically observed in the “non-experience” scores.

Category Weighting A Sensitivity Analysis was performed using a sample set of contracts, which were scored using the proposed category weightings. This resulted in: 50% of the contract selections were governed by the “experience” factors. 75% of the time, non-experience factors were not of sufficient influence to move a firm ranking by more than one-third (when experience only scores were compared with the overall final scores).

Category Weighting The category weightings are as follows: Proposed Proposed Existing Typical Specialty Firm Experience 3 3 4 Staff Experience 4 4 5 Past Performance 6 6 7 Compatibility 3 2 1 Workload 5 3 2 Location 4 2 1 Total 25 20 20

Category Weighting It has been noted that the “experience” scores may still need to be weighted even more in order to achieve our goal. However, since this is based on a relatively small sample of contracts, it was felt that the proposed weightings should be used first and after one year, if it has been determined that additional weighting is required, then at that time they can be adjusted further.

One Year Measurables Percentage of number 1 ranked firm controlled by “Experience” categories (at least 50%) Firms final rankings not influenced by more than one third by the non-experience score (at least 75%)

Action Plan Receive feedback from meeting today by email to Consultant Contracts, heather.huval@la.gov by February 19, 2016 Group A evaluate feedback and make final recommendations Submit to Chief Engineer and Steering Committee for approval Goal is to approve process by 2016 Louisiana Transportation Conference and implementation by July 1, 2016

Consultant Contract Services Contact Information Masood Rasoulian 225-379-1433 masood.rasoulian@la.gov Contracts Administration Kathy Ward 225-379-1893 kathy.ward@la.gov Contracts Manager Wanda Crawford 225-379-1406 wanda.crawford@la.gov Agreements/Invoicing Manager Heather Huval 225-379-1733 heather.huval@la.gov Advertisement Specialist Larry Hamm 225-379-1457 lawrence.hamm@la.gov Engineer 5 -DCL

QUESTIONS ? Latest information to be posted on the Consultant Contracts Services website under Latest News