1. Travel Time Technology Evaluation Comparison of Multiple Data Sources
Preliminary Results
Jonathan Riehl
Wisconsin Traffic Operations and Safety Laboratory
Department of Civil and Environmental Engineering
University of Wisconsin-Madison
Elizabeth Schneider
Bureau of Traffic Operations
Wisconsin Department of Transportation
June 20, 2017

2. Outline Background Study Overview Data Availability and Reliability
Travel Time Comparisons
Conclusions

3. Why Travel Times are Important
Quality of Service for Drivers
Accurate
Current
Useful – Provide Alternatives
Monitor Roadway Traffic Conditions
Overall Performance
Performance Management and Planning
Short Term – Red Flags

4. Why Travel Times are Important
Interstates years ago, but did have to fill in some non-remaining areas
November 2016 – Routes of Significance non-interstate limited access, work with local agencies
Federal Rule 23 CFR 511 (SAFETEA-LU Section 1201) – Travel Time Requirements
Mandates travel time accuracy to
within 85% of the actual travel time delivered to the traveling public
within 10 minutes of the initial speed measurement
with an overall travel time availability of 90 percent

5. Travel Time Technologies
Can calculate travel times using two detectors
Must be close together to be accurate
The old floating car method is the oldest probe data method
Point Sensors
Loop Detectors
Microwave Detectors
Probe Data Systems
Original Floating Car Method*
Video and License Plate Readers
Radar
Bluetooth
WiFi Technology
High-Frequency GPS Data
CV Technology
3rd Party Travel Times
*Not to be confused with current floating car methods

6. Public Facing Data

7. T3E Study Overview – Project Objectives
Use of real-time travel times and historic average travel times
Which methods work best for a specific real-time situation?
Rural vs. urban
Freeway vs. arterial
Peak periods vs. daytime vs. nighttime
Temporary vs. permanent
Overall costs and benefits
Acquiring data
Maintaining data
Processing data
Integrating data
Data quality

8. WisDOT Travel Times Timeline
NPMRDS not intended for real-time
Mid 2000s: Loops and microwave devices – speeds to travel times
2008: Evaluation of INRIX and AirSage travel time data
2013: National Performance Management Research Data Set (NPMRDS) released using data from HERE
2014: Bluetooth – arterials in SE Region and freeways in SW Region
: Integration and use of TomTom probe data

9. T3E Study Overview – Travel Time Technologies in this Study
TomTom
Bluetooth
NPMRDS
Microwave / Loop
Automatic Traffic Recorders (for count validation)

10. T3E Study Overview – Study Area and Periods
Corridor
Corridor Start/End
Location
Route Type
Data Types
US 12/18
I-39/90 to WIS 73
East of Madison
Rural Arterial
TomTom, NPMRDS, Bluetooth
US 14 M (Madison)
US 12/18 to County MM
Fitchburg
Rural/ Urban Freeway
TomTom, NPMRDS, Bluetooth, ATR
County M
US 18/151 to County MM
Fitchburg/ Verona
Rural Minor Arterial
TomTom, NPMRDS
US 14 J (Janesville)
I-39/90 to WIS 140
East of Janesville
Rural/ Urban Arterial
WIS 73
I-39/90 to WIS 106
Albion
TomTom, NPMRDS, Microwave
E Washington (US 151)
Blair St to Portage Rd
Madison
Urban Arterial
I-39/90
IL Border to I-94
Dane/
Rock
Rural Freeway
TomTom, NPMRDS, Bluetooth, ATR, Microwave/Loop
US 12
I-39/90 to Parmenter St
South of Madison
Urban Freeway
TomTom, NPMRDS, Bluetooth, ATR, Microwave, Loop
Time Periods: AM Rush, AM Peak, PM Rush, PM Peak, Weekday Daytime, Weekend Daytime, Nighttime

11. The Data Reliability Problem
Availability
Accuracy
Latency
Consistency
Missing Data
Incorrect Data
Sampling Rate
Vehicle Reidentification

12. Data Availability Measures
Total vehicle count
Number of vehicles counted/matched by detectors
Total vehicle percentage
Percentage of vehicles counted out of total on route
Observation percentage
Percentage of segment-intervals that have at least one vehicle detected on route
Travel time availability percentage
Percentage of time intervals that have calculable travel times

13. Data Availability – Example
Travel Time Route
Segment 1
Segment 2
Segment 3
Segment 4
Time Period – 7-8am, 15 minute intervals
Average ATR count for time period = 500
Detector Count
Segment 1
Segment 2
Segment 3
Segment 4
7:00-7:14 2 4 3
7:15-7:29 6
7:30-7:44 1
7:45-7:59
Full Time Period 7 15
Total Vehicle Count  7.5 veh/seg/hr
Total Vehicle Percentage  1.5 %
Observation Percentage  11/16 = 68.8%
Travel Time Availability Percentage*  75%
*WisDOT requires 2/3 of segments in the route to have data available for travel time to be calculable

14. Data Availability – Urban Principal Arterial US 151 (East Washington – Blair to Hagan)
Possibly add Observation Percentage and TT Reliability Percentage
Should be available moving forward
ATMS =
Loop or Microwave
Total Vehicle Counts Available by Hour (NB | SB)
Time Period
TomTom
Bluetooth
NPMRDS
ATMS
ATR
AM Rush (07:00-09:00 M-F)
22.9 | 36.3
17.9 | 31.0
Unknown
N/A
1206 | 2457
AM Peak (07:30-08:30 M-F)
25.4 | 39.1
18.4 | 32.8
1219 | 2713
PM Rush (15:00-18:00 M-F)
47.8 | 42.3
33.4 | 25.7
2709 | 1571
PM Peak (16:30-17:30 M-F)
49.9 | 43.8
33.3 | 25.5
2979 | 1579
Weekday Daytime (09:00-15:00 M-F)
33.0 | 37.9
23.3 | 24.8
1493 | 1424
Weekend Daytime (07:00-19:00 S-U)
26.4 | 29.0
15.6 | 17.9
1095 | 1074
Nighttime (22:00-04:00 M-U)
13.5 | 14.7
6.9 | 6.3
352 | 284
About 1-4%
Units are in average number of vehicles per segment/detector per hour
Values averaged for month of July 2016

15. Data Availability – Rural Freeway I-39/90 (IL Border to I-94)
Total Vehicle Counts Available by Hour (NB | SB)
Time Period
TomTom
Bluetooth
NPMRDS
ATMS
ATR
AM Rush (07:00-09:00 M-F)
82.2 | 61.4
163.7 | 138.5
Unknown
2634 | 2049
2777 | 2219
AM Peak (07:30-08:30 M-F)
83.4 | 61.8
164.9 | 136.8
2811 | 2036
2973 | 2351
PM Rush (15:00-18:00 M-F)
124.1 | 120.6
190.6 | 209.6
3012 | 2349
3094 | 3261
PM Peak (16:30-17:30 M-F)
124.1 | 122.1
185.5 | 209.5
3090 | 2912
3197 | 3490
Weekday Daytime (09:00-15:00 M-F)
125.5 | 111.3
190.8 | 199
2646 | 3081
2695 | 2527
Weekend Daytime (07:00-19:00 S-U)
126.3 | 112.4
139.9 | 139.6
2577 | 2328
2393 | 2431
Nighttime (22:00-04:00 M-U)
44.4 | 39.9
60.4 | 55.6
543 | 477
478 | 479
About 2-10%
Units are in average number of vehicles per hour per segment or detector
Values averaged for month of July 2016

16. Speed Comparison – AM Rush
NPMRDS tends to be faster on a rural principal arterial
Large discrepancy in speeds for Wis 73 (short route)
S. Min. Art.
R. Freeway
U. Freeway
R. Prpl. Art.
S. Prpl. Art.
Sub. Fwy.
U. Prpl. Art.
R. Min. Art.

17. Speed Comparison – PM Rush
1. ATMS faster on freeways, possibly in general
S. Min. Art.
R. Freeway
U. Freeway
R. Prpl. Art.
S. Prpl. Art.
Sub. Fwy.
U. Prpl. Art.
R. Min. Art.

18. Speed Comparison – Weekday Daytime
1. Speeds on urban routes faster during weekday daytime than rush on all sources, rurals are similar or in some cases slower due to trucks
S. Min. Art.
R. Freeway
U. Freeway
R. Prpl. Art.
S. Prpl. Art.
Sub. Fwy.
U. Prpl. Art.
R. Min. Art.

19. Speed Comparison – Nighttime
1. Nighttime speeds still look good even though fewer matches
2. Nighttime speeds generally lower due to trucks
S. Min. Art.
R. Freeway
U. Freeway
R. Prpl. Art.
S. Prpl. Art.
Sub. Fwy.
U. Prpl. Art.
R. Min. Art.

20. Speed Comparisons – Details Bluetooth versus NPMRDS
Tend to follow similar average speeds
NPMRDS values much more variable

21. Speed Comparisons – Details TomTom versus NPMRDS
Note the time sets
AM Rush
PM Rush
Weekend Daytime
Nighttime
Weekday Daytime
Early Morning
Evening Post Rush

22. Speed Comparisons – Details TomTom versus NPMRDS
Check for missing observations
AM Rush
PM Rush
Weekend Daytime
Nighttime
Weekday Daytime
Early Morning
Evening Post Rush

23. Speed Comparisons – Details ATMS versus Bluetooth
US 12 (Beltline)
ATMS =
microwave/loop

24. Speed Comparisons – Details ATMS versus Bluetooth
Lack of Bluetooth observations
Could be due to placement of sensors
WIS 73
Insert diagram showing segment this represents

25. Statistical Analysis of Travel Times
Mean absolute error (MAE) – Magnitude of Differences
Root mean square error (RMSE) – Highlights Large Differences
Correlation coefficient (Corr) – Linear Relationship
Why did you choose and what did they mean.
MAE= 1 t i=1 t abs( Travel Time i A − Travel Time i B
RMSE= 1 𝑡 𝑖=1 𝑡 Travel Time t A − Travel Time t B 2
𝝆=( 1 𝜎 𝐴 ∗ 𝜎 𝐵 ∗𝑇 ) 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐴 − 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐴 )( 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐵 − 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐵

26. Statistical Analysis of Travel Times
Theil’s inequality coefficient (U) – Alignment of time-series data
Bias proportion (UM) – Extent of systematic error
Variance proportion (US) – How similarly varying the sources are
Covariance proportion (UC) – Extent of un unsystematic error
𝑈= 1 𝑇 𝑖=1 𝑇 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐴 − 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐵 𝑇 𝑖=1 𝑇 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐴 𝑇 𝑖=1 𝑇 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐵 2
𝑈 𝑀 = 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐴 − 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐵 ( 1 𝑇 ) 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐴 − 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐵 2
𝑈 𝑠 = 𝜎 𝐴 − 𝜎 𝐵 2 ( 1 𝑇 ) 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐴 − 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐵 2
𝑈 𝑐 = 2(1−𝝆 ) 𝜎 𝐴 𝜎 𝐵 ( 1 𝑇 ) 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐴 − 𝑇𝑟𝑎𝑣𝑒𝑙 𝑇𝑖𝑚𝑒 𝑡 𝐵 2

27. Statistical Analysis of Travel Times
3-5 Conclusions highlighted
Units: Mae, RMSE in MPH, r is unitless, so are U values

28. Cost Comparisons – New Deployments
For limited deployments, costs are similar
For large deployments, third-party probe data plans are significantly cheaper
Deployments at small scale very expensive
With increasing number of miles, all detection methods see reduction in cost per mile

29. Cost Comparisons TomTom NPMRDS* Bluetooth Microwave Loop Rural Freeway
TomTom
NPMRDS*
Bluetooth
Microwave
Loop
Rural Freeway
10 mi.
13.7
15.1
17.3
32.1
25.2
100 mi.
2.1
1.5
9.6
24.4
17.5
1000 mi.
0.6
0.2
8.8
23.7
16.7
Urban Freeway
26.0
55.7
38.3
18.3
48.0
30.6
47.2
29.9
Rural Arterial
14.3
24.1
18.4
6.6
16.4
10.7
5.8
15.6
10.0
Urban Arterial
Net present cost estimates in thousands of dollars per mile, total for both directions

30. General Findings
Bluetooth and 3rd Party Probe Data offer a fraction of the matches – especially for complete routes
Data availability does not necessarily affect quality / reliability
Just because there is a low sample rate (1% for instance), travel times are not necessarily inaccurate
All technologies provide reasonable accurate travel times for most routes
Short routes seem to have large error in travel times between technologies

31. General Findings
Bluetooth reports some very high times if outliers not removed
TomTom source pool weighted toward passenger devices/vehicles more than fleet vehicles
ATMS speeds higher than either Bluetooth or NPMRDS
In part be attributable to the technical distinction between time mean speeds (such as from spot speeds from point detectors) and space mean speeds (from probe data such as Bluetooth and NPMRDS), the latter always being slower by definition

32. What technology should you use?
Many other things go into this other than accuracy
No clear answer across the board, all are reasonably accurate
Answer varies based on need (cost, accuracy, reliability, etc.)
We now have more tools to answer these questions
Examples
Temporary travel times for small construction project
Real-time travel times vs. historic average travel times
Statewide travel time deployment

33. Thank You! WisDOT Bureau of Traffic Operations TomTom Drakewell Ltd.
Acknowledgments
WisDOT Bureau of Traffic Operations
TomTom
Drakewell Ltd.
Traffic & Parking Control Co., Inc.
Contacts
Jon Riehl, TOPS Lab
Elizabeth Schneider, WisDOT Project Manager
Thank You!