1. Wearable Sensors: Creating Research Projects for New Technologies
Leigh Anne Clevenger
Pace University, DPS ’16
Advisor: Dr. Charles C. Tappert

2. Studies and Projects with new technologies – getting started
Temptation is to build a study or project around the new technology
Always start with a strong research problem statement
Impactful research is a response to a real, relevant need
Part of research is to justify the problem statement
Provides motivation for others to get involved
Participation in research
By students
By other faculty
Interest and possibly funding
Industry groups
Administration
Patent example
11/20/2018
L.A. Clevenger

3. Agenda Review of wearable sensor technology
Research Study #1 – Activity Trackers to promote healthy lifestyles
Research Study #2 – Heart Sound biometric for user authentication and clinical applications
Wearable sensor experiences for students
11/20/2018
L.A. Clevenger

4. Smartwatches and their Sensors - July 2015 (1 of 3)
11/20/2018
L.A. Clevenger

5. Smartwatches and their Sensors - July 2015 (2 of 3)
11/20/2018
L.A. Clevenger

6. Smartwatches and their Sensors - July 2016 (3 of 3)
11/20/2018
L.A. Clevenger

7. Innovative Technologies supporting Heart Sound Biometrics
11/20/2018
L.A. Clevenger

8. Public Heartsound Databases and Digital Stethoscope Sensors
#signals
recording length (s)
sensor used
frequency
HSCT-11 [16,17]
206 60
ThinkLabs
11025 Hz, 16 bits per sample
WelchAllyn [18]
150 30
Not available
16 kHz
Heartchallenge A [19]
176
1 to 30
iStethoscope Pro
30 kHz to 44 kHz
Heartchallenge B [19]
656
DigiScope
4 kHz
Primer [20] 23
8 kHz to 44 kHz
Texasheart [21] 22
8 kHz
Medscape [22] 7
Littmann Stethoscope
10 kHz
Stethoscope sensor
Reference
ThinkLabs Rhythm Digital Electronic Stethoscope
iStethoscope Pro iOS app
DigiScope
3M Littmann Electronic Stethoscope Model 3200
11/20/2018
L.A. Clevenger

9. Research Study #1: Health Features of Activity Trackers: Motivation, Goal Achievement, and Usability

10. Research questions What is a wearable device?
Do wearable device-activity trackers improve people’s health and motivation?
Do wearable device-activity trackers have an impact on our habits?
11/20/2018
L.A. Clevenger

11. Study overview
One of the major problems with using mobile technology to form healthy habits is that it lacks a social component separating it from the reality of people’s lives; which leads to problems keeping up with a routine or building healthy habits.
This study examines whether wearable technology with a social aspect will improve efficacy of forming healthy habits.  Participants will be surveyed on potential factors impacting their health goals and daily activity data collected.  Our research focuses on the survey and activity data collected on a group of  users furnished with socially integrated technology.
11/20/2018
L.A. Clevenger

12. What we did - Methodology
We conducted a 6-week user study on the efficacy of using activity trackers to motivate individuals to form healthy habits, meet fitness goals and assess the level of impact the integration of social components have in achieving those desired results. In between each of the questionnaires we will collect tracking data and performance analysis
4 questionnaires
Initial setup phase - Day 1
Solo phase - Week 1-2
Networking phase - Weeks 3-5
Ending phase - Week 6
Post study phase - Weeks 7
11/20/2018
L.A. Clevenger

13. Data collection
Eliminate noise data  validate data with user feedback pivot data
11/20/2018
L.A. Clevenger

14. Data analysis
Pivot Data
11/20/2018
L.A. Clevenger

15. Results and findings – user activity data
2500 user activities were collected, eight users improved at reaching their goals during the team phase compared to the solo phase, with the greatest individual percentage jumping from 43% to 62%.
11/20/2018
L.A. Clevenger

16. Results and findings – user activity data
There was a 7% increase during the team phase, which proves social networking has a positive impact on goal achievement.
11/20/2018
L.A. Clevenger

17. Results and findings – user survey
External Feedback
There were 19 respondents in total. There were quite a few similarities between our closed group of participants and this external group of students.
The purpose of buying tracker is monitoring their activity levels and a desire to reach their ideal weight. Just like our participants, the great majority wear the device as close to 24/7 as possible.
When asked how the tracker has affected their daily routine, they felt it encouraged them to continually look for ways to keep moving.
The majority claimed meeting goals anywhere between three and five days a week, putting them right in line with our closed group rate of 42%.
11/20/2018
L.A. Clevenger

18. Conclusion
Step count was most important for measuring healthy habit formation when compared to other recorded metrics.
‘All’ rated the accuracy of their device based on confidence at 70% or above for the week 3 Survey; the ‘majority’ of participants said on the Final Survey 80%. Thus, the survey participants feel confident fitness trackers can be useful and accurate to meet their objective. Overall, the user study participants agreed that using the activity tracker helped make them more active.
Calorie counting is still in debate based on both external and internal feedback
The device itself was not a primary motivator but only a facilitator; however, all agreed that it was becoming a more important factor in their daily lives
11/20/2018
L.A. Clevenger

19. Future outlook
Based on the user study conduced we are looking to future possibilities
Develop and propose a standard framework to facilitate comparisons between people using different activity trackers. Until reporting standards are adopted by all activity tracker software, utility programs to transpose data to a common format could be developed.
A social networking and challenge site which allows users of any activity tracker to easily register and track challenges with users on different activity trackers.
11/20/2018
L.A. Clevenger

20. Research Study #2: Biometric Classification of Heart Sound for Continual User Authentication and Clinical Applications

21. Research Problem Statement
Passive monitoring of sensor data as the smartphone or smartwatch is being used, which can provide continual authentication, is not currently available on most mobile devices. This monitoring can maintain confidence that the device owner is the current user without inconveniencing users by asking them to re-authenticate, for example with password, swipe, or fingerprint [4][7][9]
Biometrics used for passive monitoring do not currently include heart sound, which is an interesting choice because it is constantly available, hard to obtain from another person, and has been shown to be reasonably unique between individuals.
Clinical cardiology applications currently do not take advantage of the algorithms of heart sound authentication, for example to indicate a change in the patient’s heart sound on an in-home wearable mobile device app.
11/20/2018
L.A. Clevenger

22. Expected Contribution to the Field
Development of a new biometric analysis process, or “modality”.
Using heart sound input from a known public database, extend the current research on that data in areas of feature extraction and classifier choice to compare and potentially improve user authentication accuracy over selected modalities used in the literature.
Apply the new modality to a private database which has not been run through biometric analysis, compare quality and consistency of results to the original tests, and use this database to drive improvements to the modality if significant new features are present.
Demonstrate feasibility of applying the new modality for heart sound biometrics to passive and continual screening for clinical applications. Using the heart sound biometric for a cardiac patient allows passive monitoring of sensor data for screening for a change in heart sound without requiring the user to remember to perform a specific action.
Advancing the field of biometric data analysis for continual and passive user authentication by analyzing the benefits of heart sound as a way of providing added security and convenience to users.
11/20/2018
L.A. Clevenger

23. Heart Sound Biometrics - Mobile Device User Authentication
Time = 0; Authenticate?
Yes No
No Access
Time = 0+ to 30 min; Authenticate?
Yes No
11/20/2018
L.A. Clevenger
No Access – return to fingerprint authentication

24. Heart Sound Biometrics – Mobile Device Clinical Applications
Time = 0; Authenticate?
Yes No
No Notification Sent
Time = 0+ to 30 min; Authenticate?
Notify Cardiologist or 911
Yes No
11/20/2018
L.A. Clevenger

25. Diagram of Heart and Typical Waveform
11/20/2018
L.A. Clevenger

26. Heart Sound Biometric Analysis
Select waveforms from database
Preprocess to find S1-S2 pairs; Apply noise reduction if needed
Create data frames
Generate Feature Vectors for each frame
Calculate matching score
Train and test the system, and evaluate classifier results
11/20/2018
L.A. Clevenger

27. Demo of Heart Sound Input
Visualization and audio using Audacity
ThinkLabs Digital Stethoscope, HSCT11 Data
iStethoscope Pro iPhone app, Heart Challenge A Data
3M Littmann Digital Stethoscope, Medscape Data
11/20/2018
L.A. Clevenger

28. Spadaccini and Beritelli: HSCT11 Heart Sound Evaluation
11/20/2018
L.A. Clevenger

29. Wearable Sensor Experiences for Students Sensor Dashboard and Trigger

30. Wearable Sensor Communication
Sensor capabilities
Near Field Communication (NFC)
Bluetooth Low Energy (BLE) and iBeacons for Geofencing
Sensor data can be read using apps available from the Google Playstore or Apple AppStore – including Sensor Dashboard or Trigger - or using a free, open source Software Development Kit for Android or iOS
11/20/2018
L.A. Clevenger

31. Smartwatch app combination of “Sensor Dashboard”
Smartwatch running Android Wear Operating System
Android smartphone, Sensor Dashboard app
Start Sensor Dashboard on phone, then it takes 2-3 min to find the sensors on the wearable. Keep wearable close to the phone. Starts recording the data on the phone in real time.
Sensors include light, accelerometer, gyroscope (seems to not work well), gravity, magnetic field, step, heartrate, linear acceleration, rotation vector.
See a change in phone display by exercising each type of sensor.
Capable of downloading data as CSV, to be read into a spreadsheet
Need to understand what is reported for each sensor
11/20/2018
L.A. Clevenger

32. Write and scan NFC Tags using Trigger – SimpleSleep (1 of 3)
Any Android smartphone
Example which turns off ringer and turns down display brightness for use at night
Head on into Trigger, navigate to the My Tasks section and click the “+ New Task” button in the top right to get started.
First, we setup the Triggers for our actions. Go ahead and tap the “+” button in the top right.
Tap NFC to continue.
Tap Next.
Add restrictions. I always recommend playing with options, so feel free to apply any restrictions you deem fit for your needs. For our purposes, we do not want any restrictions, so simply tap Done to continue.
You can also add multiple Triggers, so that the NFC Tag only works if your device is connected to a specific WiFi network, for example. Here again, let’s keep it simple and not add additional Triggers.
11/20/2018
L.A. Clevenger

33. Write and scan NFC Tags using Trigger – SimpleSleep (2 of 3)
At the bottom, rename your Task Uniquely and concisely. I will call mine “SimpleSleep.”
Tap the “+” button in the top right corner to add your Tasks.
From the long list of available categories, tap Sound & Volume.
Tap Ring volume and Notification volume.
You can either now tap Sound & Volume again to close the category, or just leave it and move on.
Tap Display.
Tap Brightness.
You’ve now selected which settings to configure, tap Next to set values for them.
Adjust Ring Volume and Notification Volume to your desired value, I will choose 0.
Adjust your Display Brightness setting, once again, I will choose 1. Please be aware of how our device brightness values operate, some devices have a completely black screen at value 0.
Tap Add to Task.
Review your Tasks, add more if you so choose, then hit Next to continue.
11/20/2018
L.A. Clevenger

34. Write and scan NFC Tags using Trigger – SimpleSleep (3 of 3)
Trigger will now show you the Switch Tasks section. As it states, if you create Tasks here then your NFC Tag will toggle between firing your above Tasks and the ones you assign here. This is great if you want to use the same NFC Tag to toggle on and off certain settings. We’re going to skip this option today.
At this point, take a deep breath, and find your NFC tag. Take the case off your smartphone (best the first time - you can experiment later to see if it works leaving the case on)
Tap Done.
Take your NFC Tag and place it against the back of your device. You may need to move the NFC tag around on the back of the phone to find the place where the phone can successfully write to it.
You will hear a confirmation sound that your NFC Tag has been written to.
Feel free to write as many tags as you wish by tapping them one at a time against your device.
By default tags are re-writable, so if you don’t like the behavior you can modify the trigger program and re-scan the tag.
Tap Done to complete.
There you have it. You have successfully written to your own NFC Tag so that it fires a custom task on your device
Try putting the tag in the same spot, to see if the changes to ringer and brightness occur.
11/20/2018
L.A. Clevenger

35. Write and scan NFC Tags using Trigger – run Pandora
Follow the instructions for SimpleSleep, with these changes:
On “Add one or more actions” screen
Tap Applications & Shortcuts
Open Application
Speak Text
Tap Next
On “Configure Actions” screen
Select the application to open (Pandora)
Type in the text to speak (Have a good run)
The app lists what application will be launched, and what text spoken
Continue following the SimpleSleep instructions to write a new tag
11/20/2018
L.A. Clevenger

36. Geo-fencing and iBeacons
Geofencing is a feature in a software program that uses the global positioning system (GPS) or radio frequency identification (RFID) to define geographical boundaries.
Requires iPhone app programming
11/20/2018
L.A. Clevenger

37. References for Smartwatches and Smartphones to get you started – more added every day
Galaxy S5 (has a lot of sensors, and open source android software development kit)
11/20/2018
L.A. Clevenger

38. Contributors Spring 2016 Pace University Master’s Students
Anthony Martini George Angulo David Brogan Jue Wang
11/20/2018
L.A. Clevenger

39. Copyright for Material Reuse
Copyright© 2016 Leigh Anne Clevenger and Charles Tappert Pace University. Please properly acknowledge the source for any reuse of the materials as below.
Leigh Anne Clevenger and Charles Tappert, 2016 GenCyber Cybersecurity Workshop, Pace University
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation. A copy of the license is available at
11/20/2018
L.A. Clevenger