1. PROPOSED CHANGES TO RATING METHODS
Compatibility, Workload,
Past Performance,
and Category Weighting
February 11, 2016

2. Purpose
Provide consultant community and DOTD practitioners with an update on the progress of the DOTD/ACEC initiative
Obtain feedback concerning the specific recommendations for implementation

3. Background
In June 2015, DOTD Chief Engineer and ACEC BOD signed document approving recommendations for implementation
Approved 35 recommendations for implementation from 57 in 6 categories (see next chart)
Several (7) were pending, not ready
Sub committee A (procurement) continued to work
Many additional meetings (Jun –Dec 2015)
Presented to steering committee on 28 Jan 2016 and approved to move forward with this briefing

4. DOTD/ACEC Initiative Status
Total
Approved
Pending
Tabled
Not Approved
A. Procurement 22 6 7 2
B. Contracting 5 3 1
C. Contract Administration
D. Quality Assurance/Control
E. Claims
F. Contract Provisions 18 57 35 10 4 8

5. DOTD-ACEC/L Round Table Group A: Procurement
Committee of DOTD and Consultants
Focus on consultant selection process
Actively working for over 3 years
Committee members:
DOTD ACEC/L
Ed Wedge Simone Ardoin Bob Boagni
Charles Nickel Ray Mumphrey Bob Basinger
Debbie Guest Masood Rasoulian Gordon Nelson
Paul Vaught Lawrence Hamm Buddy Porta
Heather Huval Connie Porter Nick Ferlito
Jesse Rauser Suzanne McCain
Hunter Lancaster

6. Goals
Evaluate the existing consultant selection process and look for opportunities for improvement by:
Ensuring that the process appropriately balances the desire to provide as many firms as possible with the opportunity to perform the work with the need to ensure that the most technically experienced firms are selected
Increasing transparency by simplifying the process and making it less subjective and more consistent

7. Key Issues
Compatibility and Workload scoring methodologies are unnecessarily complex
Determine alternate approach to evaluate those categories
Create and define new input variables
Project Magnitude
Firm Size Designation
Better address how past performance scores for new firms are assigned

8. Key Issues
Rating categories can be divided into “experience” and “non-experience” ratings
“Experience” Categories:
Staff Experience, Firm Experience, Past Performance
“Non-Experience” Categories
Compatibility, Workload, Location
Greater variation between consultants in the “non- experience” categories leads to their over-weighting in the selection process

9. PROJECT MAGNITUDE

10. Project Magnitude
Project Magnitude is proposed to be determined by: 1. Complexity 2. Contract Time or Schedule (Current Solicitation) 3. Amount of the Contract (Projected for all Phases/Stages) 4. Route Classification.

11. Project Magnitude Criteria Value 1 2 3 Criteria Complexity Simple
Medium
Complex
Contract Time
Typical
Compressed
Critical
Contract Amount
≤ $250,000
$250,000 to $2,500,000
≥ $2,500,000
Route Classification
Non-NHS Local
Non-NHS State
NHS

12. Project Magnitude Project Magnitude Designation
Combined Criteria Values
Micro
4 to 5
Small
5 to 7
Medium
7 to 9
Large
9 to 11
Mega
11 to 12

13. FIRM SIZE

14. Number of Transportation Personnel
Firm Size
Firm Size designation is determined using the table below:
Firm Size Designation
Number of Transportation Personnel
Micro
≤ 13
Small
10 to 22
Medium
20 to 33
Large
30 to 55
Mega
≥ 50

15. COMPATIBILITY RATING

16. Compatibility Rating
Once the project magnitude and firm size have been determined, the proposed compatibility rating* can be obtained by the following table. *When a team is considered, the compatibility rating will be based on the prime consultant only.

17. Project Magnitude Designation
Compatibility Rating
Project Magnitude Designation
Micro
Small
Medium
Large
Mega
Firm Size Designation
5.0
3.0
1.0
4.0
2.0

18. WORKLOAD RATING

19. Workload Rating
Once the firm size and remaining DOTD work amount have been determined, the proposed workload rating can be obtained by the following table.

20. Project Magnitude Designation
Workload Rating
Project Magnitude Designation
5.0
5.0 → 1.0
1.0
Remaining DOTD Work
Firm Size Designation
Micro
≤ $20,000
$20,000 to $1,625,000
≥ $1,625,000
Small
≤ $110,000
$110,000 to $2,750,000
≥ $2,750,000
Medium
≤ $210,000
$210,000 to $4,125,000
≥ $4,125,000
Large
≤ $320,000
$320,000 to $6,875,000
≥ $6,875,000
Mega
≤ $530,000
$530,000 to $12,500,000
≥ $12,500,000

21. PAST PERFORMANCE RATING

22. Past Performance Existing:
Firms who have not received a rating for a work category will be assigned a rating equal to the lowest of the following:
The average rating of the firms submitting
The statewide average rating for that category as of the date the advertisement was posted.

23. Past Performance Proposed:
Firms who have not received a rating for a work category will be assigned a rating equal to the lowest of the following:
The average rating of the firms submitting
The statewide average rating for that category as of the date the advertisement was posted
The satisfactory rating of 3.

24. CATEGORY WEIGHTING

25. Category Weighting The score is based on six categories:
Firm Experience
Staff Experience
Past Performance
Compatibility
Workload
Location

26. Category Weighting
Three of the six categories can be thought of as “experience” based and are generally associated with technical ability to perform the work.
Firm Experience
Staff Experience
Past Performance

27. Category Weighting
The other three categories can be thought of as “non-experience” based and are generally associated with the distribution of opportunity for work.
Compatibility
Workload
Location

28. Category Weighting
The “experience” ratings are currently weighted as follows:
Firm Experience 3
Staff Experience 4
Past Performance 6
Total: 13

29. Category Weighting
The “non-experience” ratings are currently weighted as follows:
Compatibility 3
Workload 5
Location 4
Total: 12

30. Category Weighting
Although the “experience” scores are higher, the lower “non-experience” scores dominate the ranking, simply due to their larger variations.

31. Category Weighting
This is a known consequence of the current system. In some cases, such as complex projects, where technical experience based selection is of critical importance, current practice is to neutralize some of the “non-experience” scores to help alleviate this issue.

32. Category Weighting
Based on a sample set of contracts, the “experience” scores need to be weighted more to fully compensate for the larger variations typically observed in the “non-experience” scores.

33. Category Weighting
A Sensitivity Analysis was performed using a sample set of contracts, which were scored using the proposed category weightings.
This resulted in:
50% of the contract selections were governed by the “experience” factors.
75% of the time, non-experience factors were not of sufficient influence to move a firm ranking by more than one-third (when experience only scores were compared with the overall final scores).

34. Category Weighting
The category weightings are as follows: Proposed Proposed Existing Typical Specialty Firm Experience Staff Experience Past Performance Compatibility Workload Location Total

35. Category Weighting
It has been noted that the “experience” scores may still need to be weighted even more in order to achieve our goal. However, since this is based on a relatively small sample of contracts, it was felt that the proposed weightings should be used first and after one year, if it has been determined that additional weighting is required, then at that time they can be adjusted further.

36. One Year Measurables
Percentage of number 1 ranked firm controlled by “Experience” categories (at least 50%)
Firms final rankings not influenced by more than one third by the non-experience score (at least 75%)

37. Action Plan
Receive feedback from meeting today by to Consultant Contracts, by February 19, 2016
Group A evaluate feedback and make final recommendations
Submit to Chief Engineer and Steering Committee for approval
Goal is to approve process by 2016 Louisiana Transportation Conference and implementation by July 1, 2016

38. Consultant Contract Services Contact Information
Masood Rasoulian Contracts Administration Kathy Ward Contracts Manager Wanda Crawford Agreements/Invoicing Manager Heather Huval Advertisement Specialist Larry Hamm Engineer 5 -DCL

39. QUESTIONS ?
Latest information to be posted on the Consultant Contracts Services website under Latest News