Danger Prediction by Case-Based Approach on Expressways C. Y. Fang, P. Y. Wu, S. L. Chang, and S. W. Chen National Taiwan Normal University Department of Computer Science and Information Engineering

Outline Introduction System Flowchart and Database IEEE ITSC 2008 Outline Introduction System Flowchart and Database Weighted Relational Map The Matching Algorithm Experimental Results Conclusions and Future Work

Introduction Driving risk reduction approach IEEE ITSC 2008 Introduction Driving risk reduction approach Passive approach To reduce the degree of injury in case of an accident Examples: seat belts and airbags Active approach To prevent accidents in advance Example: driver assistance system The dangerous driving event prediction system To predict dangerous driving events Based on the weighted relational map the driving factors for the host vehicle the driving factors for nearby vehicles the driving factors for the roadway conditions

System Flowchart Driving Factors IEEE ITSC 2008 Driving Factors Relational Map for Driving Event Construction Relational Map of Dangerous Case Database Map C Map D Relational Map Matching Degree of Danger Generation Dangerous Case Insertion yes Map C no Accident Occurred? no Dangerous? The current driving factors can be obtained from different detection system with various kinds of sensors. Using this driving factors, the system can construct the relational map for driving events. Called map C. This current relational map (map C) can be used to find a most similar map from the dangerous relational maps stored in the database. Called Map D. Compared with map D, the system can compute the degree of danger of map C. yes Warning Output System Flowchart

Dividing Database into Sub-Databases IEEE ITSC 2008 Dividing Database into Sub-Databases To speed the matching process Dangerous case database is divided into four sub-databases based on road conditions. Dangerous Case Database In this paper the database will be partitioned into sub-database according to road conditions, because different road conditions will generate different dangerous driving events. For example, a vehicle changes its lane on an ordinary road is usually safe, but this kind of vehicle behavior is dangerous in a tunnel section. Interchange Section Sub-Database Tunnel Section Sub-Database Ordinary Section Sub-Database Tollbooth Section Sub-Database

Dividing Sub-Database into Classes IEEE ITSC 2008 Dividing Sub-Database into Classes Each sub-database is divided into six classes based on weather conditions. Interchange Section Sub-Database So these kinds of partition will speed the matching process. Cloudy Class Hazy Class Snowy Class Sunny Class Misty Class Rainy Class

Outline Introduction System Flowchart and Database IEEE ITSC 2008 Outline Introduction System Flowchart and Database Weighted Relational Map The Matching Algorithm Experimental Results Conclusions and Future Work

Driving Factors Lateral distance Longitudinal n3 n1 n2 IEEE ITSC 2008 Driving Factors Lateral distance Longitudinal n3 n1 n2 Host vehicle Input data for nearby vehicles Lateral distance Longitudinal distance Relative lateral speed Relative longitudinal speed The driving factors for nearby vehicles are: (1) the left-front vehicle and the host vehicle are close (2) the preceding vehicle and the host vehicle are close (3) the right-front vehicle and the host vehicle are close (4) the left vehicle and the host vehicle are close (5) the right vehicle and the host vehicle are close (6) the left-rear vehicle and the host vehicle are close (7) the following vehicle and the host vehicle are close (8) the right-rear vehicle and the host vehicle are close Many kinds of input data can be obtained from different detection systems with various sensors. The input data can be divided into three classes: for nearby vehicles, for host vehicle and for the roadways. For example, if we obtain the … and … between the host vehicle and left nearby vehicle, Then we can check the driving factor number 4 occur or not. In our paper we define eight driving factors for nearby vehicle list here.

Driving Factors Input data for host vehicle IEEE ITSC 2008 Driving Factors Input data for host vehicle Lateral distance to left/right obstacle Turning angle of front wheel Turn signal on/off Speed of host vehicle Driver’s level of alertness The driving factors for host vehicle are: (9) the host vehicle turns left (10) the host vehicle turns right (11) the host vehicle speeds up (12) the host vehicle slows down (13) driver’s level of alertness increases (14) driver’s level of alertness decreases (15) the host vehicle turns on the turn signal (16) the host vehicle maintains constant speed (17) the host vehicle goes straight Similar to the driving factors for nearby vehicle, we define nine driving factors for host vehicle in this paper. the driving factors for host vehicle can be obtained from the input data for host vehicle. For example, speed of host vehicle can be used to check the occurrence of driving factors number 11, 12, and 16. This three driving factors are all relative to the speed of host vehicle. In this paper we define nine driving factors for host vehicle. List here.

Driving Factors Driving factors for roadway: IEEE ITSC 2008 Driving Factors Driving factors for roadway: (18) smooth and straight roadway (19) smooth and curved left roadway (20) smooth and curved right roadway (21) downgrade and straight roadway (22) downgrade and curved right roadway (23) downgrade and curved left roadway (24) upgrade and straight roadway (25) upgrade and curved left roadway (26) upgrade and curved right roadway Similar to the above examples, here list the driving factors for the roadway. In this paper, we totally list 26 driving factors.

A Relational Map Each node represents one driving factor. IEEE ITSC 2008 A Relational Map Each node represents one driving factor. Node 18 : smooth and straight roadway Node 16 : the host vehicle maintains constant speed Node 1 : the left-front vehicle and the host vehicle are close Node 11 : the host vehicle speeds up Node 9 : the host vehicle turns left Two requirements to generate new nodes Fixed sampling interval Any driving factors occurring between samples 18 1 11 16 9 T Here we introduce how to construct a relational map. Here is an example of a relational map. If each node represents on driving factor introduce above, then a relational map can be constructed by linking the driving factors based on time order. Every fixed sampling interval, the system will general new nodes. And if any driving factors occurring between samples, the system will general new nodes.

Weighted Relational Map IEEE ITSC 2008 Weighted Relational Map 18 1 11 16 9 0.5 0.7 0.9 0.4 0.8 T Node value Node number Link weight Relational map Weighted Relational map We need to extra define two kind of weights in the weighted relational map. One is the node value, indicate the importance of the driving factor. The other is the link weight between adjacent nodes, indicate the degree of relationship between two adjacent nodes.

The Node Value The node value (the importance of node) IEEE ITSC 2008 The Node Value The node value (the importance of node) Initialized with a constant Increased or decreased based on the relationships with the previous, present and following nodes Examples of increasing node values The left-front vehicle and the host vehicle are close at time t -1, and the host vehicle turns left at time t. The host vehicle speeds up at time t -1, and slows down at time t. Examples of decreasing node values The host vehicle turns on the turn signal at time t -1, and turns left at time t. The host vehicle turns on the turn signal at time t -1, and turns right at time t.

The Weight Between Adjacent Nodes IEEE ITSC 2008 The Weight Between Adjacent Nodes The link weight: : value of driving factor at : value of driving factor at　 α : a constant ∆t : the time between successive driving factors The weight is large if and are very different. Example: vehicle changes its speed The weight will be large if delta t is small and the difference between l t and l t-1 is large. For example, when a vehicle change its speed between two successive driving factors, If the speed change is large, then the weight is high. The larger the speed change, the higher the weight is. 改變量越大 改變的時間越短 值會越大 說出此式子的優點…. 煞車 方向燈

Outline Introduction System Flowchart and Database IEEE ITSC 2008 Outline Introduction System Flowchart and Database Weighted Relational Map The Matching Algorithm Experimental Results Conclusions and Future Work

Defining the Driving Factor Sets for Each Node IEEE ITSC 2008 Defining the Driving Factor Sets for Each Node Three driving factor sets for : previous, present and following sets : set for node at time 18 1 11 16 9 0.5 0.7 0.9 0.4 0.8 T Weighted Relational map Before discussing the matching algorithm, we first define three sets of node ni. For example, here is a weighted relational map. Let the node ni be the node 16. Then the previous set of node 16 is the set including one node, node 18. Since there is no other node on the same layer with node 16, the present set is empty. Finally the following set of node 16 is the set including node 1 and node 11.

Relationship Between Adjacent Nodes IEEE ITSC 2008 Relationship Between Adjacent Nodes Node value This equation defines the relationship between two adjacent nodes. For example, the relationship value will be large if …,…, and ,,, these three values are all large. Link weight T

Relationship Between the Nodes on Same Layer IEEE ITSC 2008 Relationship Between the Nodes on Same Layer Node value This equation defines the relationship between the nodes on the same layer. We find the minimum node value of these nodes as the degree of the relationship between the nodes on same layer. T

Table from Weighted Relational Map IEEE ITSC 2008 Table from Weighted Relational Map Set null 0.9/16 0.5/18 0.318/1,0.182/11 0.4/11 0.9/1 0.4/16 0.4/1 0.444/1,0.356/11 0.4/9,0.4/16 end 0.9/9 0.9 0.9 Now we can transform the weighted relational map into a table like this one. For example, the first column of three row indicates the node ni,t is node 16. The second column indicates the previous set of node 16 includes a node, node 18. And the number 0.5 is the degree of relationship between node 16 and node 18. The three column indicates the present set of the same layer. It is empty here. The forth column indicates the following set of node 16 is nodes 1 and 11. And the number 0.318 is the degree of relationship between node 16 and node 1. 0.5 0.7 1 0.5 0.9 0.8 0.7 9 0.7 Weighted Relational map 18 16 0.4 1 0.9 0.5 11 0.8 0.7 16 T

IEEE ITSC 2008 Matching Algorithm Map C is the current weighted relational map formed in real time, and is the driving factor in C. Map D is the dangerous weighted relational map in the database, is the driving factor in D. : the similarity between two maps N : the number of driving factors in C : the similarity between two driving factors. The matching algorithm is used to match the current map C and one dangerous case of the database, map D. The matching algorithm calculates the similarity between map C and map D to measure the matching degree of these two maps.

Matching Algorithm where | | : scalar cardinality : fuzzy intersection IEEE ITSC 2008 Matching Algorithm where | | : scalar cardinality : fuzzy intersection : fuzzy union The similarity function consists of the summation of f functions. And the f function consists of the scalar cardinalities of the fuzzy intersection and fuzzy union.

Matching Algorithm and are constants. IEEE ITSC 2008 Matching Algorithm 　　 : fuzzy driving factor set for node at time in the map C. 　　 : fuzzy driving factor set for node at time in the map D. T( ) is the weighting function. Here shows the relative equations of fuzzy intersection. You can see other equations in the paper. and are constants.

Outline Introduction System Flowchart and Database IEEE ITSC 2008 Outline Introduction System Flowchart and Database Weighted Relational Map The Matching Algorithm Experimental Results Conclusions and Future Work

IEEE ITSC 2008 Experimental Results 0.9 0.5 0.5 0.833 0.182 11 1 11 0.2 0.5 0.9 0.833 0.182 Map C 18 1 0.957 0.6 0.5 0.444 9 14 21 0.6 0.5 0.9 0.5 0.5 Sim(C,D)=1 0.9 0.5 0.5 0.833 0.182 11 1 11 0.2 0.5 0.9 There is no doubt, if the map C is equal to map D then the similarity is 1. 0.833 0.182 18 1 0.957 0.6 0.5 Map D 0.444 9 14 21 0.6 0.5 0.9 0.5 0.5 T 2.0 1.5 1.0 0.5 0.0

Example (1) 18 1 11 9 0.5 0.9 0.957 0.2 0.444 Sim(C3,D) =0.66 Map C3 IEEE ITSC 2008 18 1 11 9 0.5 0.9 0.957 0.2 0.444 Sim(C3,D) =0.66 Map C3 14 0.833 0.6 Sim (C4,D) =0.74 Map C4 Example (1) Sim(C1,D) =0.48 18 0.5 1 0.9 0.957 Sim(C2,D) =0.65 Map C2 Map C1 18 1 14 11 9 21 0.0 0.5 1.0 1.5 2.0 T 0.9 0.957 0.2 0.444 0.833 0.6 0.182 Map D The first example shows when the map C is growing as time goes on, The similarity between map C and map D will increase too.

IEEE ITSC 2008 Example (1) 0.9 0.5 0.5 0.833 0.182 11 1 11 0.2 0.5 0.9 0.833 0.182 Map C5 18 1 0.957 0.6 0.5 0.444 9 14 21 0.6 0.5 0.9 0.5 0.5 Sim(C5,D)=1 18 1 14 11 9 21 0.0 0.5 1.0 1.5 2.0 T 0.9 0.957 0.2 0.444 0.833 0.6 0.182 Map D

IEEE ITSC 2008 Example (2) Map D1 18 1 11 9 0.5 0.9 0.957 0.2 0.444 0.9 0.5 Sim(C,D1)= 0.508 0.833 11 1 0.5 0.9 0.2 0.833 18 1 Map C 0.957 0.6 0.444 9 14 0.6 0.9 0.5 Sim(C,D2)= 0.743 Map D2 18 1 14 11 9 21 0.5 0.9 0.957 0.2 0.444 0.833 0.6 0.182 This example shows the case that there are two similar maps stored in the database. One map is a submap of another. For example, the map D1 is a submap of map D2. Moreover, the map D1 is the submap of map C, and the map C is the sebmap of map D2. Since the current map map C dose not occur the dangerous case represented by map D1, then the similarity between map D1 and map C is smaller than that between map D2 and C. T 1.5 1.0 0.5 0.0

Example (3) T Sim(C,D2)= 0.938 Map D2 Map C 18 1 14 11 9 21 0.5 0.9 IEEE ITSC 2008 Example (3) T Sim(C,D2)= 0.938 Map D2 Map C 18 1 14 11 9 21 0.5 0.9 0.348 0.25 0.222 0.842 0.2 0.4 0.211 0.118 Map D1 0.316 Sim(C,D1)= 0.967 This example shows the case that the nodes are the same but the link weights are different. From this example, we know that the link weights of map C is more similar to those of maps D1. Thus the similarity between map D1 and C is larger than that between map D2 and C. 2.0 1.5 1.0 0.5 0.0

IEEE ITSC 2008 Example (4) T 18 1 14 11 9 21 0.5 0.9 0.957 0.2 0.444 0.833 0.6 0.182 0.0 1.0 1.5 2.0 22 3 20 10 0.4 0.125 Sim(C,D2)= 0.081 Map C Hit with right-front vehicle Map D2 Hit with left-front vehicle 6 12 Map D1 Hit with left-rear vehicle Sim(C,D1)= 0 When the current map C is very different to the maps stored in the dangerous case database. Then each similarity is very low.

Conclusions and Future Work IEEE ITSC 2008 Conclusions and Future Work The proposed system Predicting dangerous driving events based on the weighted relational map which is constructed by the driving factors Using fuzzy matching algorithm to get the similarity between two weighted relational maps Future Work Improving the method to experimental threshold of level of danger Hoping test vehicles could equip with the prototype system in the future

Thank you for your attention! IEEE ITSC 2008 Thank you for your attention!