1. University of Maryland, College Park
Evaluation of Travel Information and Prediction System (TIPS) for Work Zone Operations
University of Maryland, College Park

2. TIPS overview Traffic Information and Prediction System
The messages displayed on these three PCMS are the range of travel time from each spot to Spot 5 (I-695 Gore) (e.g., “12 TO 16 MIN TO I-695”).

3. Criteria of TIPS Evaluation
System Accuracy and Reliability
Data collection for evaluation: Testing vehicle method
TIPS Accuracy : comparing the actual measured travel times to those displayed from each PCMS.
TIPS Reliability : consistency between the travel time information reported on the website and those displayed on each PCMS.

4. System Accuracy Evaluation on 11/14
PCMS #1
PCMS #2
PCMS #3 LB UB
1750 vph
2400 vph
2200 vph
4250 vph
1650 vph
2650 vph
# of incorrect 9* 8 7
# of correct 6* 27
System accuracy (%)
40.00
77.14
79.41
Volume difference from Sensor data
(Note: “No message” data on PCMS #1 were excluded from the sample size)
The accuracy drops as PCMS #1 is far away from I-695.

5. System Accuracy Evaluation on 11/14
Frequency distribution of predicted travel time errors
Deviation of predicted travel time error (sec)
Frequency
PCMS #1
PCMS #2
PCMS #3
<= -480 1
<= -240 2
<=
<= 240 3
<= 480
>
Total 9 8 7
Statistical analysis of predicted travel time errors
PCMS #1
PCMS #2
PCMS #3
Average (seconds)
149.11
38.75
206.29
Standard deviation
415.05
308.11
181.70

6. System Accuracy Evaluation on 11/19
PCMS #1
PCMS #2
PCMS #3 LB UB
1900 vph
5000 vph
2100 vph
4100 vph
1100 vph
2400 vph
# of incorrect 8 5 4
# of correct 14 15
System accuracy (%)
57.89
73.68
78.95
Volume difference from Sensor data
The volume difference from the sensor data at PCMS #1 is greater than those at PCMS #2 and #3
The accuracy drops as PCMS #1 is far away from I-695 and the volume difference increases.

7. System Accuracy Evaluation on 11/19
Frequency distribution of predicted travel time errors
Deviation of predicted travel time error (sec)
Frequency
PCMS #1
PCMS #2
PCMS #3
<= -480 1
<= -240
<= 4 2
<= 240
<= 480
>
Total 8 5
Statistical analysis of predicted travel time errors
PCMS #1
PCMS #2
PCMS #3
Average (seconds)
-21.00
85.40
87.25
Standard deviation
315.37
268.95
401.38

8. System Accuracy Evaluation on 11/2421
PCMS #1
PCMS #2
PCMS #3 LB UB
1550 vph
2450 vph
1200
vph
2400
1000
2300
# of incorrect 2
# of correct 19
System accuracy (%)
100.00
90.48
Volume difference from Sensor data
The LB and UB of volumes are low and their differences are small, which mean that traffic conditions are uncongested and stable.
The system can achieve a high accuracy under the stable traffic flow patterns.

9. System Accuracy Evaluation on 11/24
Frequency distribution of predicted travel time errors
Deviation of predicted travel time error (sec)
Frequency
PCMS #1
PCMS #2
PCMS #3
<= -480
<= -240
<=
<= 240 2
<= 480
>
Total
Statistical analysis of predicted travel time errors
PCMS #1
PCMS #2
PCMS #3
Average (seconds)
N/A
148.00
Standard deviation
16.97

10. System Accuracy Evaluation on 11/25
PCMS #1
PCMS #2
PCMS #3 LB UB
600 vph
3750 vph
1000 vph
4300 vph
1500 vph
2750 vph
# of incorrect 29 12 6
# of correct 19 25
System accuracy (%)
6.45
61.29
80.65
Volume difference from Sensor data
The performance of such a system clearly varies with the range of flow rate variation.
PCMS #1 experiences the largest range of flow variation, and yields the poorest results.

11. System Accuracy Evaluation on 11/25
Frequency distribution of predicted travel time errors
Deviation of predicted travel time error (sec)
Frequency
PCMS #1
PCMS #2
PCMS #3
<= -480 12
<= -240 4
<= 1
<= 240 5
<= 480 2
> 10
Total 29 6
Statistical analysis of predicted travel time errors
PCMS #1
PCMS #2
PCMS #3
Average (seconds)
103.92
157.50
Standard deviation
792.84
224.49
168.95

12. System Accuracy Evaluation on 12/0410
PCMS #1
PCMS #2
PCMS #3 LB UB
1200 vph
3000 vph
1100
vph
4400 vph
1500 vph
2700 vph
# of incorrect 3 1
# of correct 13 12
System accuracy (%)
76.92
100.00
92.31
Volume difference from Sensor data UB
4400 vph
The accuracy drops as PCMS #1 is far away from I-695 and the volume difference increases.

13. System Accuracy Evaluation on 12/04
Frequency distribution of predicted travel time errors
Deviation of predicted travel time error (sec)
Frequency
PCMS #1
PCMS #2
PCMS #3
<= -480
<= -240
<= 1
<= 240
<= 480
>
Total 2
Statistical analysis of predicted travel time errors
PCMS #1
PCMS #2
PCMS #3
Average (seconds)
-70.00
N/A
151.00
Standard deviation
190.13

14. System Accuracy Evaluation
Conclusions
The accuracy is often dropped for those PCMS far away from Spot 5 (I-695 Gore), or experiencing a wide range of the flow rate variation.
The system doesn’t provide reliable travel time information during congested peak hours, especially for a short peak or a transition period between off-peak and peak hours.

15. System Reliability Evaluation on 11/14 and 11/19
PCMS #1
PCMS #2
PCMS #3
# of Inconsistence 4 1
# of Consistence 9 33 30
System reliability (%)
69.23
97.06
96.77
(Note: “No message” from the PCMS is excluded from the sample size.)
11/19
PCMS #1
PCMS #2
PCMS #3
# of Inconsistence 1
# of Consistence 19 18
System reliability (%)
100.00
94.74

16. System Reliability Evaluation on 11/24 and 11/25
PCMS #1
PCMS #2
PCMS #3
# of Inconsistence
# of Consistence 21
System reliability (%)
100.00
11/25
PCMS #1
PCMS #2
PCMS #3
# of Inconsistence 24 1
# of Consistence 9 33 30
System reliability (%)
27.27
97.06
96.77
(Note: “Blank” from the website is included from the sample size.)

17. System Reliability Evaluation on 12/04
PCMS #1
PCMS #2
PCMS #3
# of Inconsistence 7
# of Consistence 6 13
System reliability (%)
46.15
100.00
(Note: “Blank” from the website is included from the sample size.)

18. System Reliability Evaluation
Conclusions
The system reliability also show patterns similar to the system accuracy.
That is, the reliability of a PCMS decreases with its distance to Spot 5 (I-695 Gore).