Drowsiness Detection System Using Heartbeat Rate in Android-based Handheld Devices Advisor : Dr. Kai-Wei Ke Presenter : D. Jayasakthi Department of Electrical Engineering and Computer Science

Contents Introduction Motivation Objective Methodology Results Conclusion 26/24/2013

Introduction Driver drowsiness is a major cause of traffic crashes. Drowsy driving is a serious issue in our society not only because it affects those who are driving while drowsy, but because it puts all other road users in danger. Therefore, the use of assisting systems that monitor a driver’s level of vigilance is important to prevent road accidents. These systems should then alert the driver in the case of drowsiness or inattention 36/24/2013

Motivation A common activity in most people’s life is driving; therefore, making driving safe is an important issue in everyday life. Even though the driver’s safety is improving in road and vehicle design, the total number of serious crashes is still increasing. Most of these crashes result from impairments of the driver’s attention. 46/24/2013

Motivation Drowsiness detection can be done in various ways based on the results of different researchers. The most accurate technique towards driver fatigue detection is dependent on physiological phenomena like brain waves, heart rate etc. Also different techniques based on the behaviors can be used, which are natural and non-intrusive. These techniques focus on observable visual behaviors from changes in a human’s facial features like eyes, head and face. 56/24/2013

Objective The aim of the thesis is develop a prototype for drowsiness detection system. The application is developed using the android SDK and it will detect the heart beat signals from the i_Mami-HRM2 heart rate monitoring device. ECG signal obtained from the sensor is analyzed in time domain and frequency domain. 66/24/2013

Objective In frequency domain, the power spectral density (PSD) is found. From the PSD the Low Frequency(LF) to High Frequency(HF) ratio is estimated. It is found that the LF/HF ratio decreases as the person becomes sleepy. As a result the drowsiness of a person can be detected from this power ratio. 76/24/2013

How it Works 6/24/20138 Autonomic Nervous System (ANS) activity presents alterations during stress, extreme fatigue and drowsiness. Wakefulness states are characterized by an increase of sympathetic activity and/or a decrease of parasympathetic activity. Extreme relaxation states are characterized by an increase of parasympathetic activity and/or a decrease of sympathetic activity.

How it Works The ANS activity can be measured non-invasively from the Heart Rate Variability (HRV) signal obtained from ECG. Power on low frequency (LF) band ( Hz) is considered as a measure of sympathetic activity. Power on high frequency (HF) band ( Hz) is considered of parasympathetic origin in classical HRV analysis. Balance between sympathetic and parasympathetic systems is measured by the LF/HF ratio. 6/24/20139

Methodology Various methods that has been implemented are: Bluetooth module ECG Measuring Heart beat Heart Rate Variability Various Signal Processing Methods applied to the ECG signals are: Decimation Hamming Window Fast Fourier Transform Calculate the low to high frequency ratio 106/24/2013

I-Mami HRM2 and Android Phone 6/24/ I-Mami HRM2 sensor from Microtime Computer Inc. Garmin Asus A50

Pairing the Sensor with the Mobile First the device discovery is done in order to connect the sensor with the mobile. If a device is discoverable, it will respond to the discovery request by sharing some information, such as the device name and its unique MAC address. Once a connection is made with a remote device for the first time, a pairing request will be automatically presented to the user. The user must enter a 4 digit pin number for the device to be paired. 126/24/2013

136/24/2013 Scan for Bluetooth Devices Pairing Request Sensor has been paired with the mobile

Bluetooth Module A A No Ye s 146/24/2013 Initialize Bluetooth Socket Perform a lookup on the remote device in order to match the UUID UUID - Universally Unique Identifier

1. Main Screen with all modules 2.Bluetooth Module 3. List of paired device 4. Sensor Connected to the mobile 156/24/ Device not connected

Display ECG signals As a result of the electrical stimulation a change in potential of the order of 1mV can be measured during the cardiac cycle. This signal is known as the electrocardiogram (ECG). The ECG detector works mostly by detecting and amplifying the tiny electrical changes on the skin that are caused during each heartbeat. The I-Mami HRM2 heart rate monitoring device is used to fetch the heart rate of a person and it is displayed in the android mobile with the help of programmable application, developed by using android SDK. 166/24/2013

2. Select Sensor from menu 3. Displays the paired devices 4. Displays the ECG signals 176/24/ ECG Module Main Screen

Displaying the Heart Rate The heart rate is the number of heart beats per minute. Normal heart rate of a human being depends on the age. For example, children will have higher heart rates comparing with the adults. This measurement can be done in various ways with respect to time. 60 seconds (no calculation needed) - most accurate 15 seconds (multiply by 4) 10 seconds (multiply by 6) Less than 10 seconds = less precise 186/24/2013

1.Heart Rate Measurement Module 2. Select a device from menu 3. Lists the paired device 4. Displays the heart rate and other values. 196/24/2013

Heart Rate Variability Heart rate variability (HRV), known as the variation of the period between consecutive heartbeats over time. HRV refers to the variations in the beat intervals or correspondingly the instantaneous HR. In time domain analysis, based on beat to beat or NN intervals some variables are analyzed. They are SDNN: Standard Deviation of all normal to normal intervals index. Often calculated over a 24-hour period. SDANN, the standard deviation of the average NN intervals calculated over short periods, usually 5 minutes. SDANN is therefore a measure of changes in heart rate due to cycles longer than 5 minutes. NN50: Number of pairs of adjacent NN intervals differing by more than 50 ms in the entire recording pNN50: The proportion of NN50 divided by total number of NNs. AVNN: Average of all NN intervals. 206/24/2013

1. Heart Rate Variability 2. Select a device from menu 3. Select the sensor 4.Displays the HRV 216/24/2013

Method to Detect Drowsiness Drowsines s Detection Obtain ECG signal from sensor Reduce the sampling rate to 50 Hz Apply Hamming Window Apply FFT Calculate LF/HF ratio Is Ratio Decre asing Person Becomes Drowsy Person is not drowsy No Yes 226/24/2013

Decimation Consider a band-limited discrete-time signal x(m) with a base-band spectrum X(f). The sampling rate can be decreased by a factor of L through discarding of L–1 samples for every L samples of x(m). Decimation by a factor of L can be achieved through a two-stage process of: (a) Low-pass filtering of the zero-inserted signal by a filter with a cutoff frequency of Fs/2L, where Fs is the sampling rate. (b) Discarding of L–1 samples for every L samples The decimation factor is simply the ratio of the input rate to the output rate. It is usually symbolized by "M", so input rate / output rate=M. 236/24/2013

Decimation The sampling frequency of the sensor was 250 Hz which means 250 samples per second. It was very high to process the ECG signals. So the sampling frequency was reduced by 50 Hz which means 250/50 = 5 samples per second. The decimation was done using a low pass filter technique. 246/24/2013

Hamming Window Technique Windowing functions, enhances the ability of an FFT to extract spectral data from signals. Windowing functions act on raw data to reduce the effects of the leakage that occurs during an FFT of the data. There are many window functions available. For an ECG signal the appropriate window function is the Hamming Window. The formula for Hamming window is w(n)=0.54−0.46cos(2πn/N−1). If x(n) is the signal,then we get the windowed signal by multiplying x(n) with the w(n). 256/24/2013

Fast Fourier Transform(FFT) The FFT is a highly elegant and efficient algorithm, which is still one of the most used algorithms in speech processing, communications, frequency estimation, etc Basic radix-2 algorithm is used which requires N to be a power of 2. FFT is applied to the windowed ECG signal. By applying FFT, the power spectrum was found. LF/HF ratio is calculated every 1 minute. If this ratio decreases then the person in becoming drowsy. 266/24/2013

Results- While Awake 276/24/2013

Results – While Asleep 286/24/2013

Conclusion A non-obstructive, real-time, continuous monitoring method for determining the drowsiness of the driver has been described. From the results it is clear that the LF/HF ratio decreases when the person is sleeping. Since ECG is one of the most easy to use physiological signals, a definite relation between drowsiness and HRV may lead to safer driving. By applying FFT, the computational complexity is reduced. 296/24/2013

Reference S. Hu and R. Bowlds, "Pulse wave sensor for non-intrusive driver's drowsiness detection," in Engineering in Medicine and Biology Society, EMBC Annual International Conference of the IEEE, Minneapolis, MN, G. Furman, A. Baharav, C. Cahan and S. Akselrod, "Early detection of falling asleep at the wheel: A Heart Rate Variability approach," Computers in Cardiology, pp , S. Elsenbruch, M. Harnish, and W. C. Orr, “Heart rate variability during waking and sleep in healthy males and females,” Sleep, vol. 22, pp , /24/201330

Thank You 316/24/2013