Advisor : Dr. Kai-Wei Ke Presenter : D. Jayasakthi Wireless and Broadband Networks Lab, Department of Electrical Engineering and Computer Science, National Taipei University of Technology, Taiwan-106.

Contents Introduction Objective Design and Analysis Implementation Experimental Results Conclusions 2

Introduction A common activity in most people’s life is driving; therefore, making driving safe is an important issue in everyday life. Even though the driver’s safety is improving in road and vehicle design, the total number of serious crashes is still increasing. Most of these crashes result from impairments of the driver’s attention. 3

Drowsy driving is a serious issue in our society not only because it affects those who are driving while drowsy, but also it puts all other road users in danger. Therefore, the use of assisting systems that monitor a driver’s level of vigilance is important to prevent road accidents. These systems should then alert the driver in the case of drowsiness or inattention. 4

Drowsiness detection can be done in various ways based on the results of different researchers. The most accurate technique towards driver fatigue detection is dependent on physiological phenomena like brain waves, heart rate etc. Also different techniques based on the behaviors can be used, which are natural and non-intrusive. These techniques focus on observable visual behaviors from changes in a human’s facial features like eyes, head and face. 5

Objective The aim of the thesis is develop a prototype for drowsiness detection system. The application is developed using the android SDK and it will detect the heart beat signals from a heart rate monitoring device. Also using this heart rate monitoring device, the ECG signals are obtained. As a result of the electrical stimulation a change in potential of the order of 1mV can be measured during the cardiac cycle. This signal is known as the Electrocardiogram (ECG). 6

ECG signal obtained from the sensor is analyzed in frequency domain. In frequency domain, the power spectral density (PSD) is found. From the PSD the Low Frequency(LF) to High Frequency(HF) ratio is estimated. It is found that the LF/HF ratio decreases as the person becomes drowsy. As a result the drowsiness of a person can be detected from this power ratio. 7

How it Works Autonomic Nervous System (ANS) activity presents alterations during stress, extreme fatigue and drowsiness. Wakefulness states are characterized by an increase of sympathetic activity and/or a decrease of parasympathetic activity. Extreme relaxation states are characterized by an increase of parasympathetic activity and/or a decrease of sympathetic activity. 8

The ANS activity can be measured non-invasively from the Heart Rate Variability (HRV) signal obtained from ECG. Power on low frequency (LF) band ( Hz) is considered as a measure of sympathetic activity. Power on high frequency (HF) band ( Hz) is considered of parasympathetic origin in classical HRV analysis. Balance between sympathetic and parasympathetic systems is measured by the LF/HF ratio. 9

Design and Analysis Various Signal Processing Methods needed to apply to the ECG signals are: Decimation Hamming Window Fast Fourier Transform Calculate the low to high frequency ratio 10

Decimation Consider a band-limited discrete-time signal x(n) with a base-band spectrum X(f). The sampling rate can be decreased by a factor of M through discarding of M–1 samples for every M samples of x(n). 11 h[k] M x[n] v[n] y[n] Filter Sampling Rate Compressor FsFs F s /M

Decimation by a factor of M can be achieved through a two-stage process of: (a) Low-pass filtering of the zero-inserted signal by a filter with a cutoff frequency of F s /M, where F s is the sampling rate. (b) Discarding of L–1 samples for every L samples The decimation factor is simply the ratio of the input rate to the output rate. It is usually symbolized by "M", so input rate / output rate=M. 12

The sampling frequency of the sensor was 250 Hz which means 250 samples per second. It was very high to process the ECG signals. So the sampling frequency was reduced by 50 Hz. The decimation was done using a low pass filter technique. 13

Hamming Window Technique Windowing functions, enhances the ability of an FFT to extract spectral data from signals. Windowing functions act on raw data to reduce the effects of the leakage that occurs during an FFT of the data. There are many window functions available. For an ECG signal the appropriate window function is the Hamming Window. The formula for Hamming window is w(n)=0.54−0.46cos(2πn/N−1). If x(n) is the signal,then we get the windowed signal by multiplying x(n) with the w(n). 14

Fast Fourier Transform(FFT) The FFT is a highly elegant and efficient algorithm, which is still one of the most used algorithms in speech processing, communications, frequency estimation, etc., Basic radix-2 algorithm is used which requires N to be a power of 2. FFT is applied to the windowed ECG signal. By applying FFT, the power spectrum is found. LF/HF ratio is calculated every 1 minute. If this ratio decreases then the person is becoming drowsy. 15

Implementation Various methods that has been implemented are: Bluetooth Module ECG for Drowsiness Detection Heart Beat Measurement Heart Rate Variability 16

I-Mami HRM2 and Android Phone 17 I-Mami HRM2 sensor from Microtime Computer Inc. Garmin Asus A50

Pairing Sensor with the Mobile First the device discovery is done in order to connect the sensor with the mobile. If a device is discoverable, it will respond to the discovery request by sharing some information, such as the device name and its unique MAC address. Once a connection is made with a remote device for the first time, a pairing request will be automatically presented to the user. The user must enter a 4 digit pin number for the device to be paired. 18

Scan for Bluetooth Devices Pairing Request Sensor has been paired with the mobile 19

Bluetooth Communication 20 Start Bluetooth Module Obtain bluetooth device object Use this object to acquire bluetooth socket Initialize bluetooth socket Perform lookup on remote device in order to match UUID Match UUID? Channels will not be opened Share RFCOMM Chanel Obtain signals from sensor No Yes UUID - Universally Unique Identifier

1. Main Screen with all Modules 2.Bluetooth Module 3. List of Paired Devices 4. Sensor Connected to the Mobile 5. Device Not Connected Bluetooth Communication 21

ECG Signals Electrocardiography is the interpretation of the electrical activity of the heart over a period of time. The ECG detector works mostly by detecting and amplifying the tiny electrical changes on the skin that are caused during each heartbeat. The I-Mami HRM2 heart rate monitoring device is used to fetch the heart rate of a person and it is displayed in the android mobile with the help of programmable application, developed by using android SDK. 22

1. Select Sensor from menu2. Displays paired devices3. Displays ECG Signals ECG Signals 23

Heart Rate Measurement The heart rate is the number of heart beats per minute. Normal heart rate of a human being depends on the age. For example, children will have higher heart rates comparing with the adults. This measurement can be done in various ways with respect to time. 60 seconds (no calculation needed) - most accurate 15 seconds (multiply by 4) 10 seconds (multiply by 6) Less than 10 seconds = less precise 24

1.Heart Rate Measurement Module 2. Select a Device from Menu3. Lists the Paired Device4. Displays the Heart Rate and other values 25

Heart Rate Variability Heart rate variability (HRV), known as the variation of the period between consecutive heartbeats over time. In time domain analysis, based on beat to beat or NN intervals some variables are analyzed. They are SDNN: Standard Deviation of all normal to normal intervals index. Often calculated over a 24- hour period. SDANN, the standard deviation of the average NN intervals calculated over short periods, usually 5 minutes. SDANN is therefore a measure of changes in heart rate due to cycles longer than 5 minutes. NN50: Number of pairs of adjacent NN intervals differing by more than 50 ms in the entire recording pNN50: The proportion of NN50 divided by total number of NNs. AVNN: Average of all NN intervals. 26

1. Select a Device from Menu 2. Select the Sensor 3.Displays the HRV 27

Implementation of Drowsiness Detection Technique 28 Obtain ECG signal from sensor Reduce the sampling rate to 50 Hz Apply Hamming Window Apply FFT Calculate LF/HF ratio Is Ratio Decreasing ? Person is not drowsy Person Becomes Drowsy Collect next 512 samples of data Start Drowsiness Detection Is max_count ? Alert the person No Yes No A A A A

Experimental Setup The testing was performed on two persons, one male and one female. Data was collected when the person’s were awake and asleep. Two hours of data was collected from each of them. The testing was repeated 10 times for both male and female and also when they were awake and asleep. 29

Experimental Results for Male, State: Awake 30 Time Ratio Time Ratio Time Ratio Time Ratio

Time Ratio Time Ratio Time Ratio Time Ratio Experimental Results for Male, State: Awake 31

Experimental Results for Male, State: Asleep Time Ratio Time Ratio Time Ratio Time Ratio

Experimental Results for Male, State: Asleep TimeRatioTimeRatioTimeRatioTimeRatio

Experimental Results for Female, State: Awake Time Ratio Time Ratio Time Ratio Time Ratio

Experimental Results for Female, State: Awake Time Ratio Time Ratio Time Ratio Time Ratio

Experimental Results for Female, State: Asleep Time Ratio Time Ratio Time Ratio Time Ratio

Experimental Results for Female, State: Asleep Time Ratio Time Ratio Time Ratio Time Ratio

Comparison of Power Ratio for Awake and Sleep States 38

Comparison of Power Ratio for Awake and Sleep States

Conclusions In this research work, drowsiness detection has been analyzed based on the ECG signal obtained from the sensor. The application is successfully able to detect the heart beat accurately and it also displays the heart rate variability and the ECG signals in the android devices, respectively. By applying FFT to the obtained ECG signal, the power on the low and high frequency components were measured. Then the LF/HF ratio was calculated for every one minute. 40

From the graph it is clear that the HRV analysis on the two hour heart rate time series showed that LF/HF ratio had a decreasing trend when they were asleep or feeling drowsy. When a decreasing trend is identified below 0.17 and for a max_count value of 3, if the value continuously decreases, then an alarm will be invoked automatically in order to alert the driver. 41

My contributions for the thesis are Display the ECG signal successfully on the android mobile. Apply the signal processing techniques and process the obtained ECG signal. Find the power spectrum and calculate the LF/HF ratio and save in database for analysis purpose. Collect data when awake and asleep from both male and female for 2 hours. Analyze the power ratio values in both the cases. Detect the drowsy state from the collected data. Compare the power ratio values for both male and female. 42

Future Works The efficiency of this system could be further improved by employing the sensors on the seat belt to achieve better accuracy. In addition more data must be collected from human test in order to improve the accuracy of drowsiness judgement. Further with the help of the ECG, we can also analyze the person’s mood on a daily basis. 43

44