1 Li Li [WSC17] Institute of Integrated Sensor Systems Department of Electrical and Computer Engineering Multi-Sensor Soft-Computing System for Driver Drowsiness Detection Li Li, Klaudius Werber, Carlos F. Calvillo, Khac Dong Dinh, Ander Guarde and Andreas König 10-Dec-2012  Introduction  Driving Scene Modeling and Hardware Setup  Software Components and Algorithms  Experimental Results  Conclusion and Future Work

2 Li Li [WSC17]  Major factor in 20 percent of all accidents in the United States in 2006  The second most frequent cause of serious truck accidents on German highways  Major damage caused by drowsy truck or bus drivers Introduction Enhance active safety with advanced driver assistance

3 Li Li [WSC17] Hardware Setup DeCaDrive System  Multi-sensing interfaces Depth camera Steering angle sensor Pulse rate sensor … …  PC-based soft-computing subsystem  PC-based driving simulator

4 Li Li [WSC17]  SoA depth camera +Extention of 2D image with distance +Wide field of view +Relatively low computational cost +Robust to lighting variations (active sensing) +Non-intrusive and non-obstructive (eye-safe NIR light source) Hardware Setup PMD CamCube Microsoft Kinect SoftKinetic DepthSense

5 Li Li [WSC17]  Steering angle sensor Steering behavior of driver Correlation with driver status and driver intention  Pulse rate sensor Heart health and fitness Time domain analysis Frequency domain analysis Hardware Setup embedded

6 Li Li [WSC17] Software Components and Algorithms  Overview of the data processing flow

7 Li Li [WSC17] Feature Computation  Features being computed from multiple sensor measurements

8 Li Li [WSC17] Experimental Results  Test subjects Five male test subjects 22 to 25 years old (mean: 23.6, std:1.1) All have driver‘s license for at least 4 years No alcohol drinking before test  Experiments One hour driving simulation for each test subject 588-minute driving sequence recorded Ground truth: not drowsy, a little drowsy, deep drowsy Through self-rated score and response time

9 Li Li [WSC17] Experimental Results  Examples of different sensor features blink frequency low steering percentage mean pulse rate

10 Li Li [WSC17] Experimental Results Depth image Eye pupil and corners  Screenshot of online processing of various sensor data

11 Li Li [WSC17]  Results of ANN based classifier with two training algorithms Experimental Results

12 Li Li [WSC17]  Confusion matrix of LM 80 hidden neurons 10-fold cross-validation  Confusion matrix of SCG 40 hidden neurons 10-fold cross-validation Experimental Results

13 Li Li [WSC17]  Drowsiness level classification accuracy depending on selected features Experimental Results

14 Li Li [WSC17] Conclusion and Future Work  Contribution Emerging framework for driver status monitoring and intention detection with multi-sensor soft-computing system Classification of three different drowsiness levels with up to 98.9% accuracy based on data sets of five test subjects.  Future work Validation with more statistics and with data from real vehicles Variance compensation by adaptive learning Optimization of feature selection with sophisticated heuristics Utilization of other advanced classification techniques, e.g., SVM Integration of more embedded sensors with wireless technology

15 Li Li [WSC17] Thank you!