1. Multi-Sensor Soft-Computing System for Driver Drowsiness Detection
Computer-Aided Design of Analog and Mixed-Signal ICs using Cadence DFW II icfb
Institute of Integrated Sensor Systems Department of Electrical and Computer Engineering
Multi-Sensor Soft-Computing System
for Driver Drowsiness Detection
Li Li, Klaudius Werber,
Carlos F. Calvillo, Khac Dong Dinh,
Ander Guarde and Andreas König
10-Dec-2012
Introduction
Driving Scene Modeling and Hardware Setup
Software Components and Algorithms
Experimental Results
Conclusion and Future Work
Andreas König, 2001

2. Computer-Aided Design of Analog and Mixed-Signal ICs using Cadence DFW II icfb
Introduction
Major factor in 20 percent of all accidents in the United States in 2006
The second most frequent cause of serious truck accidents on German highways
Major damage caused by drowsy truck or bus drivers
Enhance active safety with advanced driver assistance
Andreas König, 2001

3. Hardware Setup DeCaDrive System Multi-sensing interfaces
Computer-Aided Design of Analog and Mixed-Signal ICs using Cadence DFW II icfb
Hardware Setup
DeCaDrive System
Multi-sensing interfaces
Depth camera
Steering angle sensor
Pulse rate sensor
… …
PC-based soft-computing subsystem
PC-based driving simulator
Andreas König, 2001

4. Hardware Setup SoA depth camera Extention of 2D image with distance
Computer-Aided Design of Analog and Mixed-Signal ICs using Cadence DFW II icfb
Hardware Setup
SoA depth camera
Extention of 2D image with distance
Wide field of view
Relatively low computational cost
Robust to lighting variations (active sensing)
Non-intrusive and non-obstructive (eye-safe NIR light source)
Microsoft Kinect
PMD CamCube
SoftKinetic DepthSense
Make them suitable for driver status monitoring in various lighting conditions, especially in truck driver cabin.
Andreas König, 2001

5. Hardware Setup Pulse rate sensor Steering angle sensor
Computer-Aided Design of Analog and Mixed-Signal ICs using Cadence DFW II icfb
Hardware Setup
Pulse rate sensor
Heart health and fitness
Time domain analysis
Frequency domain analysis
Steering angle sensor
Steering behavior of driver
Correlation with driver status and driver intention
embedded
Andreas König, 2001

6. Software Components and Algorithms
Computer-Aided Design of Analog and Mixed-Signal ICs using Cadence DFW II icfb
Software Components and Algorithms
Overview of the data processing flow
Andreas König, 2001

7. Computer-Aided Design of Analog and Mixed-Signal ICs using Cadence DFW II icfb
Feature Computation
Features being computed from multiple sensor measurements
Andreas König, 2001

8. Experimental Results Test subjects Experiments Five male test subjects
Computer-Aided Design of Analog and Mixed-Signal ICs using Cadence DFW II icfb
Experimental Results
Test subjects
Five male test subjects
22 to 25 years old (mean: 23.6, std:1.1)
All have driver‘s license for at least 4 years
No alcohol drinking before test
Experiments
One hour driving simulation for each test subject
588-minute driving sequence recorded
Ground truth: not drowsy, a little drowsy, deep drowsy
Through self-rated score and response time
Andreas König, 2001

9. Experimental Results Examples of different sensor features
Computer-Aided Design of Analog and Mixed-Signal ICs using Cadence DFW II icfb
Experimental Results
Examples of different sensor features
blink frequency
low steering percentage
mean pulse rate
Andreas König, 2001

10. Computer-Aided Design of Analog and Mixed-Signal ICs using Cadence DFW II icfb
Experimental Results
Screenshot of online processing of various sensor data
Eye pupil and corners
Depth image
Andreas König, 2001

11. Computer-Aided Design of Analog and Mixed-Signal ICs using Cadence DFW II icfb
Experimental Results
Results of ANN based classifier with two training algorithms
Andreas König, 2001

12. Experimental Results Confusion matrix of SCG Confusion matrix of LM
Computer-Aided Design of Analog and Mixed-Signal ICs using Cadence DFW II icfb
Experimental Results
Confusion matrix of SCG
40 hidden neurons
10-fold cross-validation
Confusion matrix of LM
80 hidden neurons
10-fold cross-validation
Andreas König, 2001

13. Computer-Aided Design of Analog and Mixed-Signal ICs using Cadence DFW II icfb
Experimental Results
Drowsiness level classification accuracy depending on selected features
Andreas König, 2001

14. Conclusion and Future Work
Computer-Aided Design of Analog and Mixed-Signal ICs using Cadence DFW II icfb
Conclusion and Future Work
Contribution
Emerging framework for driver status monitoring and intention detection with multi-sensor soft-computing system
Classification of three different drowsiness levels with up to 98.9% accuracy based on data sets of five test subjects.
Future work
Validation with more statistics and with data from real vehicles
Variance compensation by adaptive learning
Optimization of feature selection with sophisticated heuristics
Utilization of other advanced classification techniques, e.g., SVM
Integration of more embedded sensors with wireless technology
In this paper, we present a novel system approach to driver drowsiness detection based on multi-sensor data fusion and soft-computing algorithms. By observing head movement and facial features with depth camera, by monitoring steering behavior and pulse rate of driver the presented system is able to classify three different drowsiness levels with up to 98.9% accuracy based on data sets of five test subjects.
The robustness of the presented approach needs to be validated with more statistics and with data from real vehicles. The variance of drivers, vehicles, road and weather conditions can be compensated by adaptive learning. Feature selection can be optimized with sophisticated heuristics, e.g., genetic algorithm (GA) and particle swarm optimization (PSO). Advanced classification techniques such as support vector machine (SVM) can be incorporated in the system as well for further optimization. In addition, we plan to integrate more miniaturized embedded sensors with wireless technology, e.g., EEG and ECG sensors, to improve the effectiveness and robustness of the system.
Andreas König, 2001

15. Computer-Aided Design of Analog and Mixed-Signal ICs using Cadence DFW II icfb
Thank you!
Andreas König, 2001