1 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Jerzy Wieczorek, Rafael J. Fernández-Moctezuma, and Robert L. Bertini, Portland State University Transportation Research Board 89 th Annual Meeting Washington, DC January 12, 2010

2 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data What is a bottleneck? Queueing upstreamQueueing upstream Freely-flowing downstreamFreely-flowing downstream Temporal and spatial variationTemporal and spatial variation Queued Unqueued Bottleneck Detectors Introduction Validation Visualization

3 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Research objectives Develop an automated algorithm to systematically detect freeway bottlenecks, and quantify and visualize their impactsDevelop an automated algorithm to systematically detect freeway bottlenecks, and quantify and visualize their impacts Implement this tool in PORTAL, our continuously-updated transportation data archiveImplement this tool in PORTAL, our continuously-updated transportation data archive Develop an automated algorithm to systematically detect freeway bottlenecks, and quantify and visualize their impactsDevelop an automated algorithm to systematically detect freeway bottlenecks, and quantify and visualize their impacts Implement this tool in PORTAL, our continuously-updated transportation data archiveImplement this tool in PORTAL, our continuously-updated transportation data archive Introduction Validation Visualization

4 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data PORTAL database Loop Detector Data 20 s count, lane occupancy, speed from 500 detectors (1.2 mi spacing) Incident Data 140,000 since 1999 Weather Data VMS Data 19 VMS since 1999 Data Archive Days Since July 2004 About 300 GB 4.2 Million Detector Intervals Bus Data 1 year stop level data 140,000,000 rows Introduction Validation Visualization

5 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Presentation objectives Brief review of data and methods Lessons learned from validating our algorithm and evaluating proposed improvements Showcase our visualization tools Brief review of data and methods Lessons learned from validating our algorithm and evaluating proposed improvements Showcase our visualization tools Introduction Validation Visualization

6 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Reading a contour plot Interstate bridge MP 308 I-405 MP 304 I-405 MP 300 OR-217 MP 292 I-205 MP 288 I-84 MP 302 Introduction Validation Visualization

7 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Reading a contour plot Interstate bridge MP 308 I-405 MP 304 I-405 MP 300 OR-217 MP 292 I-205 MP 288 I-84 MP 302 Introduction Validation Visualization

8 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Reading a contour plot Interstate bridge MP 308 I-405 MP 304 I-405 MP 300 OR-217 MP 292 I-205 MP 288 I-84 MP 302 Introduction Validation Visualization

9 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Contour plots in real time ? Interstate bridge MP 308 I-405 MP 304 I-405 MP 300 OR-217 MP 292 I-205 MP 288 I-84 MP 302 Introduction Validation Visualization

10 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Mockup of desired tool Interstate bridge MP 308 I-405 MP 304 I-405 MP 300 OR-217 MP 292 I-205 MP 288 I-84 MP 302 Introduction Validation Visualization

11 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Data I-5 Northbound corridor has best loop detector coverage: 20 detectors over 22 miles, giving 1.15 mi average detector spacingI-5 Northbound corridor has best loop detector coverage: 20 detectors over 22 miles, giving 1.15 mi average detector spacing Chose 24 representative mid-week days for testing and validationChose 24 representative mid-week days for testing and validation Averaged data across all 3 lanes, removed bad detectors, and imputed missing valuesAveraged data across all 3 lanes, removed bad detectors, and imputed missing values I-5 Northbound corridor has best loop detector coverage: 20 detectors over 22 miles, giving 1.15 mi average detector spacingI-5 Northbound corridor has best loop detector coverage: 20 detectors over 22 miles, giving 1.15 mi average detector spacing Chose 24 representative mid-week days for testing and validationChose 24 representative mid-week days for testing and validation Averaged data across all 3 lanes, removed bad detectors, and imputed missing valuesAveraged data across all 3 lanes, removed bad detectors, and imputed missing values MP 308 MP 284 Introduction Validation Visualization

12 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Our starting point Based on a California field experimentBased on a California field experiment Three parameters: Using 5-minute aggregated data, declare active bottleneck between two detectors in a given time period if:Three parameters: Using 5-minute aggregated data, declare active bottleneck between two detectors in a given time period if: Speed difference across bottleneck is > 20 mph, andSpeed difference across bottleneck is > 20 mph, and Upstream speed is < 40 mphUpstream speed is < 40 mph Next consecutive detectors < 40 mph upstream of active bottleneck are considered congestedNext consecutive detectors < 40 mph upstream of active bottleneck are considered congested Based on a California field experimentBased on a California field experiment Three parameters: Using 5-minute aggregated data, declare active bottleneck between two detectors in a given time period if:Three parameters: Using 5-minute aggregated data, declare active bottleneck between two detectors in a given time period if: Speed difference across bottleneck is > 20 mph, andSpeed difference across bottleneck is > 20 mph, and Upstream speed is < 40 mphUpstream speed is < 40 mph Next consecutive detectors < 40 mph upstream of active bottleneck are considered congestedNext consecutive detectors < 40 mph upstream of active bottleneck are considered congested Introduction Validation Visualization

13 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data False Alarm Rate Max Upstream Speed Min Speed Differential Parameter optimization example Detection Rate Max Upstream Speed Min Speed Differential Try to maximize Detection Rate: # bottlenecks detected total # of bottlenecks Try to maximize Detection Rate: # bottlenecks detected total # of bottlenecks …while you minimize False Alarm Rate: # of false detections total # of detections …while you minimize False Alarm Rate: # of false detections total # of detections Introduction Validation Visualization

14 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Alternate performance metrics Our algorithm matches ground truth well if we have high Detection Rate and low False Alarm Rate, but this is contradictory. Compromise by seeking high values of…Our algorithm matches ground truth well if we have high Detection Rate and low False Alarm Rate, but this is contradictory. Compromise by seeking high values of… Accuracy: # of correct decisions / total # of decisionsAccuracy: # of correct decisions / total # of decisions “Sum Score”: DetectionRate + (1 – FalseAlarmRate)“Sum Score”: DetectionRate + (1 – FalseAlarmRate) “Product Score”: DetectionRate * (1 – FalseAlarmRate)“Product Score”: DetectionRate * (1 – FalseAlarmRate) Our algorithm matches ground truth well if we have high Detection Rate and low False Alarm Rate, but this is contradictory. Compromise by seeking high values of…Our algorithm matches ground truth well if we have high Detection Rate and low False Alarm Rate, but this is contradictory. Compromise by seeking high values of… Accuracy: # of correct decisions / total # of decisionsAccuracy: # of correct decisions / total # of decisions “Sum Score”: DetectionRate + (1 – FalseAlarmRate)“Sum Score”: DetectionRate + (1 – FalseAlarmRate) “Product Score”: DetectionRate * (1 – FalseAlarmRate)“Product Score”: DetectionRate * (1 – FalseAlarmRate) Introduction Validation Visualization

15 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Bottleneck detection results All three new metrics agree on optimal settings for our chosen Portland freeway corridor:All three new metrics agree on optimal settings for our chosen Portland freeway corridor: Using 3-minute aggregated data, declare a bottleneck when speed difference is > 15 mph and upstream speed is 15 mph and upstream speed is < 35 mph Method appears validated! On average:Method appears validated! On average: Successfully finds 82% of bottlenecksSuccessfully finds 82% of bottlenecks Only 16% of detections are false alarmsOnly 16% of detections are false alarms All three new metrics agree on optimal settings for our chosen Portland freeway corridor:All three new metrics agree on optimal settings for our chosen Portland freeway corridor: Using 3-minute aggregated data, declare a bottleneck when speed difference is > 15 mph and upstream speed is 15 mph and upstream speed is < 35 mph Method appears validated! On average:Method appears validated! On average: Successfully finds 82% of bottlenecksSuccessfully finds 82% of bottlenecks Only 16% of detections are false alarmsOnly 16% of detections are false alarms Introduction Validation Visualization

16 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Optimized results Introduction Validation Visualization

17 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Lane-by-lane analysis Thus far, have pooled data from all lanesThus far, have pooled data from all lanes Proposed improvement: analyze each lane separately and have lanes “vote” on whether bottleneck is activeProposed improvement: analyze each lane separately and have lanes “vote” on whether bottleneck is active …but no consistent improvement in performance metrics; not worth tripling the processing time…but no consistent improvement in performance metrics; not worth tripling the processing time Thus far, have pooled data from all lanesThus far, have pooled data from all lanes Proposed improvement: analyze each lane separately and have lanes “vote” on whether bottleneck is activeProposed improvement: analyze each lane separately and have lanes “vote” on whether bottleneck is active …but no consistent improvement in performance metrics; not worth tripling the processing time…but no consistent improvement in performance metrics; not worth tripling the processing time Introduction Validation Visualization

18 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Fundamental diagrams Fundamental diagrams can suggest how occupancy or volume data may be used to improve algorithmFundamental diagrams can suggest how occupancy or volume data may be used to improve algorithm Introduction Validation Visualization

19 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Congestion tracking Interstate bridge MP 308 I-405 MP 304 I-405 MP 300 OR-217 MP 292 I-205 MP 288 I-84 MP 302 Introduction Validation Visualization

20 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Congestion tracking Interstate bridge MP 308 I-405 MP 304 I-405 MP 300 OR-217 MP 292 I-205 MP 288 I-84 MP 302 Introduction Validation Visualization

21 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Queue propagation speeds Interstate bridge MP 308 I-405 MP 304 I-405 MP 300 OR-217 MP 292 I-205 MP 288 I-84 MP mph 14.1 mph 7.66 mph Introduction Validation Visualization

22 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Delay (in vehicle-hrs) Interstate bridge MP 308 I-405 MP 304 I-405 MP 300 OR-217 MP 292 I-205 MP 288 I-84 MP veh-hrs 1569 veh-hrs veh-hrs Introduction Validation Visualization

23 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Congestion tracking: worst 90% Interstate bridge MP 308 I-405 MP 304 I-405 MP 300 OR-217 MP 292 I-205 MP 288 I-84 MP 302 Introduction Validation Visualization

24 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Congestion tracking: 75% Interstate bridge MP 308 I-405 MP 304 I-405 MP 300 OR-217 MP 292 I-205 MP 288 I-84 MP 302 Introduction Validation Visualization

25 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Congestion tracking: 50% Interstate bridge MP 308 I-405 MP 304 I-405 MP 300 OR-217 MP 292 I-205 MP 288 I-84 MP 302 Introduction Validation Visualization

26 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Congestion tracking: rarest 10% Interstate bridge MP 308 I-405 MP 304 I-405 MP 300 OR-217 MP 292 I-205 MP 288 I-84 MP 302 Introduction Validation Visualization

27 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Congestion tracking: 1000 days Introduction Validation Visualization Analyzed data from 1000 weekdays (Jan. 1, 2005 to Nov. 13, 2008)Analyzed data from 1000 weekdays (Jan. 1, 2005 to Nov. 13, 2008) Movie of percentile plots give a sense of entire distribution of bottleneck frequencyMovie of percentile plots give a sense of entire distribution of bottleneck frequency Analyzed data from 1000 weekdays (Jan. 1, 2005 to Nov. 13, 2008)Analyzed data from 1000 weekdays (Jan. 1, 2005 to Nov. 13, 2008) Movie of percentile plots give a sense of entire distribution of bottleneck frequencyMovie of percentile plots give a sense of entire distribution of bottleneck frequency

28 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Next steps Implement into PORTAL 2.0Implement into PORTAL 2.0 Rank recurrent congestion on Portland’s freeways (by vehicle-hours of delay caused)Rank recurrent congestion on Portland’s freeways (by vehicle-hours of delay caused) Improve detection algorithm: incorporate occupancy/volume data, weather conditions, incident data, etc.Improve detection algorithm: incorporate occupancy/volume data, weather conditions, incident data, etc. Implement into PORTAL 2.0Implement into PORTAL 2.0 Rank recurrent congestion on Portland’s freeways (by vehicle-hours of delay caused)Rank recurrent congestion on Portland’s freeways (by vehicle-hours of delay caused) Improve detection algorithm: incorporate occupancy/volume data, weather conditions, incident data, etc.Improve detection algorithm: incorporate occupancy/volume data, weather conditions, incident data, etc.

29 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data AcknowledgmentsAcknowledgments Oregon Department of TransportationOregon Department of Transportation Federal Highway AdministrationFederal Highway Administration TriMetTriMet The City of Portland, ORThe City of Portland, OR National Science FoundationNational Science Foundation CONACYT (Mexico)CONACYT (Mexico) TransPort ITS CommitteeTransPort ITS Committee Oregon Department of TransportationOregon Department of Transportation Federal Highway AdministrationFederal Highway Administration TriMetTriMet The City of Portland, ORThe City of Portland, OR National Science FoundationNational Science Foundation CONACYT (Mexico)CONACYT (Mexico) TransPort ITS CommitteeTransPort ITS Committee Visit PORTAL Online:

30 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Thank You!

31 Techniques for Validating an Automatic Bottleneck Detection Tool Using Archived Freeway Sensor Data Optimized results DetectionRate values for the 24 days are approx. normal: Mean.82, Stdev.12, 95% CI of mean (.78,.87), 95% prediction interval: (.55, 1) FalseAlarmRate values are also approx. normal: Mean.16, Stdev.11, 95% CI of mean (.12,.21), 95% prediction interval: (0,.43) DetectionRate values for the 24 days are approx. normal: Mean.82, Stdev.12, 95% CI of mean (.78,.87), 95% prediction interval: (.55, 1) FalseAlarmRate values are also approx. normal: Mean.16, Stdev.11, 95% CI of mean (.12,.21), 95% prediction interval: (0,.43)