1. CAPITAL DISTRICT TRANSPORTATION COMMITTEE Project Evaluation
AMPO Annual Conference 10/26/16
Quantitative and Qualitative Benefits

2. Agenda Previous Project Evaluations Issues Revision Objectives
Revision Process
New Project Evaluations
TIP Evaluation Results
Comments and Questions

3. Previous Project Evaluations
CDTC has been calculating quantifiable benefits in our benefit/cost (B/C) ratio calculation for many years.
We use our travel demand computer model to develop the costs of not improving the facility for each project, and calculate safety, travel time, operating, user costs, and facility life benefits.
For non-quantifiable benefits we were using a system of “filters” and “networks” that each project passed or failed, but that did not capture all of the non-quantifiable benefits.
Filters include B/C top half, on NHS, network points top half, pass/fail screening, advisory committee recommendation

4. Issues
In 2014 we undertook a project to better determine or score the non-quantifiable project benefits.
We understood members’ concerns and confusion regarding the use of “filters” and “networks” in previous TIP updates.
And we also recognized that because of this confusion we sometimes relied too heavily on the project B/C ratio.

5. Objectives
Provide a direct link between our long range plan principles, recommendations, and funding priorities and our TIP project selection so that project selection directly reflects the New Visions priorities.
Provide an easily adjustable evaluation system if New Visions priorities change.
Quantify the non-quantifiable project benefits that are not included in the B/C ratio.
Provide an explicit, objective, transparent, easily understood and complete evaluation system that better reflects the project value.
Replace the use of “filters” and “networks” that were not easily understood in project evaluations and potentially biased the process against rural projects.
Augment and not replace the project B/C ratio.
Provide incentives for sponsors to include beneficial project features in project scopes.

6. Process
Early in the development of this new merit evaluation system, staff examined several precedents and MPO examples of best practices.
We also examined the merit evaluation systems used by more than 20 large to medium MPOs.
Ten merit evaluation systems, which used a point scoring system and were relevant, explicit, and transparent, were further analyzed.

7. Process
Our new merit evaluation system is similar to the project evaluation systems used in:
Nashville, Tennessee
Boston, Massachusetts
Seattle, Washington
Syracuse, New York
San Diego, California
Richmond, Virginia
Winston-Salem, North Carolina
Northern New Jersey
Genesee Transportation Council (Rochester, NY)
Mid-Ohio (Columbus)
But CDTC’s system includes more categories and more specific descriptions for each score.

8. Process
CDTC’s new draft merit evaluation system was submitted to our Planning Committee for their review, and several adjustments were made.
The new system was tested using 15 representative past projects and 11 potential future projects.
The tests confirmed the objectivity and effectiveness of the new evaluation system.
There was considerable debate over the actual scores, negative scores, impacts on different projects, and weight of merit score v. B/C ratio.
11 potential projects were all NYS DOT projects.

9. New Project Evaluation
Our new merit evaluation system measures non-quantifiable project benefits by using a point scoring system for each of the following 11 categories:
Regional Benefit
Community Quality of Life & Equity
Appropriate Infrastructure
Multi-Modalism
Environment & Health
Economic Development
Safety & Security
Operations & Technology
Freight
Innovation
Project Delivery

11. New Project Evaluation
Each point in each category is clearly defined so that project sponsors can accurately estimate the score for each project, and project evaluators can accurately and consistently score each project.
Unlike most other systems, there is a very specific, logical, and related progression from one point score to another, which reduces judgement and uncertainty.
For CDTC it was originally a maximum score of 67 points, then scaled to 50 points.
The final merit category score assigned to each project can range from a -21 to +50 points.
This score is added to the benefit/cost ratio (maximum of 50 points) so that each project can receive a total score ranging from -21 to +100 points.

12. In spreadsheet form the rating scoresheet is connected to the summary sheet.

14. New Project Evaluation
A copy of our new merit evaluation system can be found at ( ).
If you would like an Excel version, give me your business card or me at

15. TIP Evaluation Results
This year CDTC used our new merit evaluation system and the merit scoresheet to evaluate 119 new projects for our new TIP.
Projects were broken into categories (bridge, pavement, etc.) with easily comparable scores.
There was little disagreement with final scores.
Selected highest rated projects!
The following is an example of the results submitted to the Planning and Policy Committees.

16. Read headings and total project scores.

17. 2016-21 TIP New Projects Project Type Cost ($M) Number of Projects
Pavement Preservation Projects 
9.536 9
Pavement Beyond Preservation Projects 
2.251 1
Bridge Preservation Projects 
34.729 7
Bridge Beyond Preservation Projects 
37.020 6
Bicycle and Pedestrian Preservation Projects
0.721 2
Bicycle and Pedestrian Beyond Preservation Projects
7.436
Other Beyond Preservation Projects 
2.857
Low Volume Local Projects 
2.907
Total
97.457 41
Sponsor Type
Cost ($M)
Number of Projects
Number of Sponsors
State 
75.825 15 1
County 
7.306 9 3
City 
9.485 10 5
Town 
3.285
Village 
1.556 2
Total
97.457 41 16
Other Beyond Preservation included intersection projects, slope repair, roundabouts, traffic signals, etc. Low volume = 2600 AADT or less.

18. Michael V. Franchini, Executive Director
Questions
Contact:
Michael V. Franchini, Executive Director