Travel Time Technology Evaluation Comparison of Multiple Data Sources Preliminary Results Jonathan Riehl Wisconsin Traffic Operations and Safety Laboratory Department of Civil and Environmental Engineering University of Wisconsin-Madison Elizabeth Schneider Bureau of Traffic Operations Wisconsin Department of Transportation June 20, 2017

Outline Background Study Overview Data Availability and Reliability Travel Time Comparisons Conclusions

Why Travel Times are Important Quality of Service for Drivers Accurate Current Useful – Provide Alternatives Monitor Roadway Traffic Conditions Overall Performance Performance Management and Planning Short Term – Red Flags

Why Travel Times are Important Interstates years ago, but did have to fill in some non-remaining areas November 2016 – Routes of Significance non-interstate limited access, work with local agencies Federal Rule 23 CFR 511 (SAFETEA-LU Section 1201) – Travel Time Requirements Mandates travel time accuracy to within 85% of the actual travel time delivered to the traveling public within 10 minutes of the initial speed measurement with an overall travel time availability of 90 percent

Travel Time Technologies Can calculate travel times using two detectors Must be close together to be accurate The old floating car method is the oldest probe data method Point Sensors Loop Detectors Microwave Detectors Probe Data Systems Original Floating Car Method* Video and License Plate Readers Radar Bluetooth WiFi Technology High-Frequency GPS Data CV Technology 3rd Party Travel Times *Not to be confused with current floating car methods

Public Facing Data

T3E Study Overview – Project Objectives Use of real-time travel times and historic average travel times Which methods work best for a specific real-time situation? Rural vs. urban Freeway vs. arterial Peak periods vs. daytime vs. nighttime Temporary vs. permanent Overall costs and benefits Acquiring data Maintaining data Processing data Integrating data Data quality

WisDOT Travel Times Timeline NPMRDS not intended for real-time Mid 2000s: Loops and microwave devices – speeds to travel times 2008: Evaluation of INRIX and AirSage travel time data 2013: National Performance Management Research Data Set (NPMRDS) released using data from HERE 2014: Bluetooth – arterials in SE Region and freeways in SW Region 2015-16: Integration and use of TomTom probe data

T3E Study Overview – Travel Time Technologies in this Study TomTom Bluetooth NPMRDS Microwave / Loop Automatic Traffic Recorders (for count validation)

T3E Study Overview – Study Area and Periods Corridor Corridor Start/End Location Route Type Data Types US 12/18 I-39/90 to WIS 73 East of Madison Rural Arterial TomTom, NPMRDS, Bluetooth US 14 M (Madison) US 12/18 to County MM Fitchburg Rural/ Urban Freeway TomTom, NPMRDS, Bluetooth, ATR County M US 18/151 to County MM Fitchburg/ Verona Rural Minor Arterial TomTom, NPMRDS US 14 J (Janesville) I-39/90 to WIS 140 East of Janesville Rural/ Urban Arterial WIS 73 I-39/90 to WIS 106 Albion TomTom, NPMRDS, Microwave E Washington (US 151) Blair St to Portage Rd Madison Urban Arterial I-39/90 IL Border to I-94 Dane/ Rock Rural Freeway TomTom, NPMRDS, Bluetooth, ATR, Microwave/Loop US 12 I-39/90 to Parmenter St South of Madison Urban Freeway TomTom, NPMRDS, Bluetooth, ATR, Microwave, Loop Time Periods: AM Rush, AM Peak, PM Rush, PM Peak, Weekday Daytime, Weekend Daytime, Nighttime

The Data Reliability Problem Availability Accuracy Latency Consistency Missing Data Incorrect Data Sampling Rate Vehicle Reidentification

Data Availability Measures Total vehicle count Number of vehicles counted/matched by detectors Total vehicle percentage Percentage of vehicles counted out of total on route Observation percentage Percentage of segment-intervals that have at least one vehicle detected on route Travel time availability percentage Percentage of time intervals that have calculable travel times

Data Availability – Example Travel Time Route Segment 1 Segment 2 Segment 3 Segment 4 Time Period – 7-8am, 15 minute intervals Average ATR count for time period = 500 Detector Count Segment 1 Segment 2 Segment 3 Segment 4 7:00-7:14 2 4 3 7:15-7:29 6 7:30-7:44 1 7:45-7:59 Full Time Period 7 15 Total Vehicle Count  7.5 veh/seg/hr Total Vehicle Percentage  1.5 % Observation Percentage  11/16 = 68.8% Travel Time Availability Percentage*  75% *WisDOT requires 2/3 of segments in the route to have data available for travel time to be calculable

Data Availability – Urban Principal Arterial US 151 (East Washington – Blair to Hagan) Possibly add Observation Percentage and TT Reliability Percentage Should be available moving forward ATMS = Loop or Microwave Total Vehicle Counts Available by Hour (NB | SB) Time Period TomTom Bluetooth NPMRDS ATMS ATR AM Rush (07:00-09:00 M-F) 22.9 | 36.3 17.9 | 31.0 Unknown N/A 1206 | 2457 AM Peak (07:30-08:30 M-F) 25.4 | 39.1 18.4 | 32.8 1219 | 2713 PM Rush (15:00-18:00 M-F) 47.8 | 42.3 33.4 | 25.7 2709 | 1571 PM Peak (16:30-17:30 M-F) 49.9 | 43.8 33.3 | 25.5 2979 | 1579 Weekday Daytime (09:00-15:00 M-F) 33.0 | 37.9 23.3 | 24.8 1493 | 1424 Weekend Daytime (07:00-19:00 S-U) 26.4 | 29.0 15.6 | 17.9 1095 | 1074 Nighttime (22:00-04:00 M-U) 13.5 | 14.7 6.9 | 6.3 352 | 284 About 1-4% Units are in average number of vehicles per segment/detector per hour Values averaged for month of July 2016

Data Availability – Rural Freeway I-39/90 (IL Border to I-94) Total Vehicle Counts Available by Hour (NB | SB) Time Period TomTom Bluetooth NPMRDS ATMS ATR AM Rush (07:00-09:00 M-F) 82.2 | 61.4 163.7 | 138.5 Unknown 2634 | 2049 2777 | 2219 AM Peak (07:30-08:30 M-F) 83.4 | 61.8 164.9 | 136.8 2811 | 2036 2973 | 2351 PM Rush (15:00-18:00 M-F) 124.1 | 120.6 190.6 | 209.6 3012 | 2349 3094 | 3261 PM Peak (16:30-17:30 M-F) 124.1 | 122.1 185.5 | 209.5 3090 | 2912 3197 | 3490 Weekday Daytime (09:00-15:00 M-F) 125.5 | 111.3 190.8 | 199 2646 | 3081 2695 | 2527 Weekend Daytime (07:00-19:00 S-U) 126.3 | 112.4 139.9 | 139.6 2577 | 2328 2393 | 2431 Nighttime (22:00-04:00 M-U) 44.4 | 39.9 60.4 | 55.6 543 | 477 478 | 479 About 2-10% Units are in average number of vehicles per hour per segment or detector Values averaged for month of July 2016

Speed Comparison – AM Rush NPMRDS tends to be faster on a rural principal arterial Large discrepancy in speeds for Wis 73 (short route) S. Min. Art. R. Freeway U. Freeway R. Prpl. Art. S. Prpl. Art. Sub. Fwy. U. Prpl. Art. R. Min. Art.

Speed Comparison – PM Rush 1. ATMS faster on freeways, possibly in general S. Min. Art. R. Freeway U. Freeway R. Prpl. Art. S. Prpl. Art. Sub. Fwy. U. Prpl. Art. R. Min. Art.

Speed Comparison – Weekday Daytime 1. Speeds on urban routes faster during weekday daytime than rush on all sources, rurals are similar or in some cases slower due to trucks S. Min. Art. R. Freeway U. Freeway R. Prpl. Art. S. Prpl. Art. Sub. Fwy. U. Prpl. Art. R. Min. Art.

Speed Comparison – Nighttime 1. Nighttime speeds still look good even though fewer matches 2. Nighttime speeds generally lower due to trucks S. Min. Art. R. Freeway U. Freeway R. Prpl. Art. S. Prpl. Art. Sub. Fwy. U. Prpl. Art. R. Min. Art.

Speed Comparisons – Details Bluetooth versus NPMRDS Tend to follow similar average speeds NPMRDS values much more variable

Speed Comparisons – Details TomTom versus NPMRDS Note the time sets AM Rush PM Rush Weekend Daytime Nighttime Weekday Daytime Early Morning Evening Post Rush

Speed Comparisons – Details TomTom versus NPMRDS Check for missing observations AM Rush PM Rush Weekend Daytime Nighttime Weekday Daytime Early Morning Evening Post Rush

Speed Comparisons – Details ATMS versus Bluetooth US 12 (Beltline) ATMS = microwave/loop

Speed Comparisons – Details ATMS versus Bluetooth Lack of Bluetooth observations Could be due to placement of sensors WIS 73 Insert diagram showing segment this represents

Statistical Analysis of Travel Times Mean absolute error (MAE) – Magnitude of Differences Root mean square error (RMSE) – Highlights Large Differences Correlation coefficient (Corr) – Linear Relationship Why did you choose and what did they mean. MAE= 1 t i=1 t abs( Travel Time i A − Travel Time i B RMSE= 1 𝑡 𝑖=1 𝑡 Travel Time t A − Travel Time t B 2 𝝆=( 1 𝜎 𝐴 ∗ 𝜎 𝐵 ∗𝑇 ) 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐴 − 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐴 )( 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐵 − 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐵

Statistical Analysis of Travel Times Theil’s inequality coefficient (U) – Alignment of time-series data Bias proportion (UM) – Extent of systematic error Variance proportion (US) – How similarly varying the sources are Covariance proportion (UC) – Extent of un unsystematic error 𝑈= 1 𝑇 𝑖=1 𝑇 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐴 − 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐵 2 1 𝑇 𝑖=1 𝑇 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐴 2 + 1 𝑇 𝑖=1 𝑇 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐵 2 𝑈 𝑀 = 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐴 − 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐵 2 ( 1 𝑇 ) 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐴 − 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐵 2 𝑈 𝑠 = 𝜎 𝐴 − 𝜎 𝐵 2 ( 1 𝑇 ) 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐴 − 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐵 2 𝑈 𝑐 = 2(1−𝝆 ) 𝜎 𝐴 𝜎 𝐵 ( 1 𝑇 ) 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐴 − 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐵 2

Statistical Analysis of Travel Times 3-5 Conclusions highlighted Units: Mae, RMSE in MPH, r is unitless, so are U values

Cost Comparisons – New Deployments For limited deployments, costs are similar For large deployments, third-party probe data plans are significantly cheaper Deployments at small scale very expensive With increasing number of miles, all detection methods see reduction in cost per mile

Cost Comparisons TomTom NPMRDS* Bluetooth Microwave Loop Rural Freeway   TomTom NPMRDS* Bluetooth Microwave Loop Rural Freeway 10 mi. 13.7 15.1 17.3 32.1 25.2 100 mi. 2.1 1.5 9.6 24.4 17.5 1000 mi. 0.6 0.2 8.8 23.7 16.7 Urban Freeway 26.0 55.7 38.3 18.3 48.0 30.6 47.2 29.9 Rural Arterial 14.3 24.1 18.4 6.6 16.4 10.7 5.8 15.6 10.0 Urban Arterial Net present cost estimates in thousands of dollars per mile, total for both directions

General Findings Bluetooth and 3rd Party Probe Data offer a fraction of the matches – especially for complete routes Data availability does not necessarily affect quality / reliability Just because there is a low sample rate (1% for instance), travel times are not necessarily inaccurate All technologies provide reasonable accurate travel times for most routes Short routes seem to have large error in travel times between technologies

General Findings Bluetooth reports some very high times if outliers not removed TomTom source pool weighted toward passenger devices/vehicles more than fleet vehicles ATMS speeds higher than either Bluetooth or NPMRDS In part be attributable to the technical distinction between time mean speeds (such as from spot speeds from point detectors) and space mean speeds (from probe data such as Bluetooth and NPMRDS), the latter always being slower by definition

What technology should you use? Many other things go into this other than accuracy No clear answer across the board, all are reasonably accurate Answer varies based on need (cost, accuracy, reliability, etc.) We now have more tools to answer these questions Examples Temporary travel times for small construction project Real-time travel times vs. historic average travel times Statewide travel time deployment

Thank You! WisDOT Bureau of Traffic Operations TomTom Drakewell Ltd. Acknowledgments WisDOT Bureau of Traffic Operations TomTom Drakewell Ltd. Traffic & Parking Control Co., Inc. Contacts Jon Riehl, TOPS Lab jonathan.riehl@wisc.edu Elizabeth Schneider, WisDOT Project Manager elizabeth1.schneider@dot.wi.gov Thank You!