Biometric Classification of Heart Sound for Continual User Authentication and Clinical Applications Leigh Anne Clevenger Pace University, DPS ’16 leighanne.clevenger@gmail.com Advisor: Dr. Charles C. Tappert

Heart Sound Biometrics - Mobile Device User Authentication Time = 0; Authenticate? Yes No No Access Time = 0+ to 30 min; Authenticate? Yes No 11/13/2018 L.A. Clevenger No Access – return to fingerprint authentication

Heart Sound Biometrics – Mobile Device Clinical Applications Time = 0; Authenticate? Yes No No Notification Sent Time = 0+ to 30 min; Authenticate? Notify Cardiologist or 911 Yes No 11/13/2018 L.A. Clevenger

Diagram of Heart and Typical Waveform https://www.youtube.com/watch?v=eBIWoLNzngI http://www.heart-valve-surgery.com/heart-surgery-blog/2010/05/30/video-tricuspid-valve-repair/ 11/13/2018 L.A. Clevenger

Phao, 2007: Heart Sound as a Biometric First paper in this research area, published in Pattern Recognition User authentication and identification accuracy rates 96% - 99%, 1000 data samples on their own dataset Welch Allyn Meditron electronic stethoscope, sampling rate of 2 KHz and 16 bits. 0.5 sec windows 11/13/2018 L.A. Clevenger

Spadaccini and Beritelli, 2007-2013: HSCT11 Heart Sound Evaluation Spadaccini and Beritelli provide a 200 person, two recordings each, public database of normal heart sounds (HSCT11), and use the S1 and S2 features for user authentication Biometric Analysis Modality Signal split into 4 second windows S1 and S2 identified, then window endpoints are connected Apply Chirp z-Transform (CZT), allowing high resolution analysis of narrow frequency bands Features extracted by Mel-Frequency Cepstral Coefficients (MFCC). Added to the feature vector First-to-Second Ratio (FSR) as the average power ratio of S1 to S2. Structural and Statistical Classification Structural – distance between input signal and stored template, using cepstral distance and an amplifying factor. Equal Error Rate (EER) of 36.86% Statistically – uses Gaussian Mixture Model – Universal Background Model (GMM-UBM) recognition technique. EER of 13.66% 11/13/2018 L.A. Clevenger

Spadaccini and Beritelli: HSCT11 Heart Sound Evaluation 11/13/2018 L.A. Clevenger

Zhao, 2013: Marginal Spectrum Analysis-based System Zhao et al use their own data, plus the HSCT11 dataset, for user authentication Database – 40 participants, 280 heart sounds. 0.256 sec windows Results Marginal Spectrum Analysis (MSA) – recognition rate of 94% Fourier spectrum (traditional algorithm) – recognition rate of 84% MSA on HSCT11 database – recognition rate of 92% 11/13/2018 L.A. Clevenger

Heart Sound Biometric Analysis Select waveforms from database Preprocess to find S1-S2 pairs; Apply noise reduction if needed Create data frames Generate Feature Vectors for each frame Calculate matching score Train and test the system, and evaluate classifier results 11/13/2018 L.A. Clevenger

Pre-processing to improve authentication accuracy Spadaccini split input signal into time windows S1 and S2 identified, placed in separate time windows, window endpoints are connected Chirp z-Transform (CZT) is applied, allowing high resolution analysis of narrow frequency bands Zhao, Sun, Abo-Zahhad De-noise using selected wavelets: improved Empirical Mode Decomposition (EMD) wavelet algorithm and the Shannon energy envelope algorithm, Discrete Wavelet Transformation (DWT) 11/13/2018 L.A. Clevenger

FFT and Cepstrum Generation 11/13/2018 L.A. Clevenger

Mel Frequency Cepstral Coefficient Generation Mel filterbank: maps frequency to mel scale – linear below 1kHz, log above Cepstrum is spectrum of log of the mel spectrum Result is 12 cepstral coefficients per frame, and 1 value for energy – the sum over time of the power of the samples in the frame Additional data for improved authentication accuracy can be appended to the MFCC feature vector: FSR, wave average, wave average differences, etc 11/13/2018 L.A. Clevenger

Innovative Technologies supporting Heart Sound Biometrics 11/13/2018 L.A. Clevenger

Demo of Heart Sound Input Visualization and audio using Audacity ThinkLabs Digital Stethoscope, HSCT11 Data iStethoscope Pro iPhone app, Heart Challenge A Data 3M Littmann Digital Stethoscope, Medscape Data 11/13/2018 L.A. Clevenger

Research Plans – Next Steps Investigate literature and current regulations for cardiac clinical applications scenarios, to identify opportunities and challenges Continue to create features for heart sound, using clinical features of 1st and 2nd beats, and additional features such as First-to-Second beat ratio. Use the HSCT-11 public heart sound database and scilab scripts. Incorporate methodologies from related research Implement machine learning algorithms for training and testing the system 11/13/2018 L.A. Clevenger

Backup 11/13/2018 L.A. Clevenger

Fundamentals of heart sound Heart function Is nonlinear Is periodic Has time-varying characteristics Heart rate variability (HRV) is non-stationary – it is effected by breathing, variation in blood flow (cardic-hemodynamics), and the environment Has randomness: palpitations - skipped or added beats; arrhythmia – change in beating patterns Typical heart sound measurement facts Range from 0-600 Hz S1 - First heart sound between 50-150 Hz, closure of the tricuspid and mitral valves S2 - Second heart sound between 50-200 Hz, closure of the aortic and pulmonary valves Second peak 250-350 Hz Research for user authentication often ignores 3rd and 4th heart sound as “murmur” In research, sampling frequencies used for measurement vary, some examples are 6400 Hz and 11025 Hz 11/13/2018 L.A. Clevenger

MFCC Generation Widely used for speech and speaker recognition Most often used feature extraction algorithm in heart sound biometric literature Results in 13 element feature vector Generate using *.wav files from Beritelli database and m scripts with scilab (free version of matlab) Scripts from http://www.ee.columbia.edu/~dpwe/resources/matlab/rastamat Resources D. Jurafsky, J. H. Martin, “Speech and Language Processing” http://practicalcryptography.com/miscellaneous/machine-learning/guide-mel-frequency-cepstral-coefficients-mfccs/ 11/13/2018 L.A. Clevenger

Public Heartsound Databases and Digital Stethoscope Sensors #signals recording length (s) sensor used frequency HSCT-11 [16,17] 206 60 ThinkLabs 11025 Hz, 16 bits per sample WelchAllyn [18] 150 30 Not available 16 kHz Heartchallenge A [19] 176 1 to 30 iStethoscope Pro 30 kHz to 44 kHz Heartchallenge B [19] 656 DigiScope 4 kHz Primer [20] 23 8 kHz to 44 kHz Texasheart [21] 22 8 kHz Medscape [22] 7 Littmann Stethoscope 10 kHz Stethoscope sensor Reference ThinkLabs Rhythm Digital Electronic Stethoscope http://www.thinklabs.com/   iStethoscope Pro iOS app https://itunes.apple.com/us/app/istethoscope-pro/id322110006?mt=8 DigiScope http://digiscope.diagnozit.com/ 3M Littmann Electronic Stethoscope Model 3200 http://www.3m.com/3M/en_US/company-us/all-3m-products/~/3M-Littmann-Electronic-Stethoscope-Model-3200?N=5002385+8707795+8707798+8711017+8711096+8711500+8711724+8727094+3293188392&rt=rud 11/13/2018 L.A. Clevenger