William Fadel, Ph.D. August 1, 2018 Classification of Walking and Stair Climbing Based on Raw Accelerometry Data William Fadel, Ph.D. August 1, 2018

Outline of talk How to measure physical activity (PA)? available devices what do the data look like? how are the data currently reported? Feature extraction methods Classification using tree-based methods How to identify types of PA collected in a free-living environment? (subject-level classifier) How to identify types of PA collected in a free-living environment? (population-level classifier) Brief punchline of paper 1, and then focus on paper 2 and 3. Reduce first 10

How to measure physical activity? Popular approach is to use acceleration measurements Actigraphy is a non-invasive monitoring of human rest/activity cycles. A small actigraph unit, accelerometer, is worn by an individual to measure motor activity. Acquired acceleration values form 3-dimensional time series that reflect history of subjects’ real-life activities

What do these data look like? 24 hours  60 minutes  60 seconds

How are the data reported? 24 hours of data  9000*60min*24hours = 12,960,000 = 12.96 million observations 504 Activity Count

Can we trust our step-count? On average most devices perform quite well, but we would like to do better! 14 participants (trial x 2) = 28 observations From: Accuracy of Smartphone Applications and Wearable Devices for Tracking Physical Activity Data JAMA. 2015;313(6):625-626. doi:10.1001/jama.2014.17841

Can we trust our step-count? “Fitbit trackers have a finely tuned algorithm for step counting. The algorithm is designed to look for motion patterns that are most indicative of people walking. The algorithm determines whether a motion's size is large enough by setting a threshold...Other factors can create enough acceleration to meet our threshold and cause some over counting of steps, such as riding on a bumpy road. Equally, it's possible for the algorithm to undercount (not meet the required acceleration threshold). Examples here include walking on a very soft surface such as a plush carpet.” On average most devices perform quite well, but we would like to do better! From: https://help.fitbit.com/articles/en_US/Help_article/1143 (accessed 11/30/2016)

What your activity tracker sees https://www.nytimes.com/interactive/projects/well/2014/03/accelerometers.html Use the blog rather than the video!

Research goals Extract features that quantify the important aspects of walking. Use extracted features to build an interpretable classification model. Study the properties of the classification model. Only focus on papers 2 and 3

IU walking and driving study Motivation: Walking and driving are the most common activities among the general population Lack of available data to validate activity recognition methodology Study Purpose: Identify patterns of walking, stair climbing, and driving from raw accelerometry data Collect data that simulates a free-living environment

IU walking and driving study Participants: We enrolled 32 healthy adults. 19 females and 13 males Age: Mean = 39 years (SD = 8.9 years) Range: 23 – 54 years Data collection: 4 Actigraph GT3X+ accelerometers left wrist, left hip, left ankle, right ankle Data collected at sampling frequency of 100 Hz Participants asked to clap 3 times between types of activity (gold standard) Internally mark the raw data with 3 consecutive spikes

Walking on level ground 𝑣𝑚 𝑡 = 𝑥 𝑡 2 +𝑦 𝑡 2 +𝑧 𝑡 2

Down/up/down stairs

Goal #1 – feature extraction Let’s look at a short interval for each activity type Vector Magnitude, 𝑣𝑚(𝑡)) Tri-axial view, 𝑥 𝑡 , 𝑦 𝑡 , 𝑧 𝑡 Short-time FFT (STFT)

Walking Descending Stairs Ascending Stairs

Feature extraction Divide the signal into windows of a given length Within each window, extract the features of the signal to describe the activity being performed. FFT features DWT features Time domain features

Four FFT features f1 – dominant frequency (cadence) 𝒓𝒂𝒕𝒊𝒐.𝑽𝑴 – ratio of all shaded regions to the total area under spectrum p1 – squared magnitude (power) at f1 p1_TP – p1 divided by area under total power spectrum (0.3-12.5Hz)

Two DWT features 𝑫𝑾𝑻_𝑽𝑴𝟐= 𝑗=𝛼 𝛽 𝑑 𝑗 2 /𝑉 𝑀 2 𝑫𝑾𝑻_𝑽𝑴𝟐= 𝑗=𝛼 𝛽 𝑑 𝑗 2 /𝑉 𝑀 2 where the 𝑑 𝑗 are the detail coefficients from level 𝑗 of the decomposition and 𝑉 𝑀 2 = 𝑡=1 𝑇 𝑣𝑚 𝑡 2 𝑫𝑾𝑻_𝑻𝑷= 𝑗=𝛼 𝛽 𝑑 𝑗 2 / 𝑗=1 𝐽 𝑑 𝑗 2

Time domain features 𝑀𝑒𝑎𝑛 𝑉𝑀 = 1 𝑇 𝑡=1 𝑇 𝑣𝑚(𝑡) 𝑀𝑒𝑎𝑛 𝑉𝑀 = 1 𝑇 𝑡=1 𝑇 𝑣𝑚(𝑡) 𝑆𝐷 𝑉𝑀 = 1 𝑇−1 𝑡=1 𝑇 𝑣𝑚(𝑡)−𝑀𝑒𝑎𝑛 𝑉𝑀 2 𝑉𝑀𝐶= 1 𝑇 𝑡=1 𝑇 𝑣𝑚(𝑡)−𝑀𝑒𝑎𝑛(𝑉𝑀) 𝐶𝑜𝑟𝑟 𝑋𝑌 , Corr(XZ), and Corr(YZ) 𝐴𝑐𝑡𝐼𝑛𝑡= 𝑠 𝑥 + 𝑠 𝑦 + 𝑠 𝑧 3 Define notation!

Goal #2 – build a classifier (CART) From: http://www.stat.wisc.edu/~loh/treeprogs/guide/wires11.pdf (accessed 12/2/2016)

Classification model (subject level) Data structure: Response: Activity = {walking, descending stairs, ascending stairs} Predictors: 13 extracted features Classification tree for each subject (n = 32): Train algorithm on a subset of the data Test algorithm on the remaining data. Evaluate classifier using 100 iterations of random cross validation to obtain an estimate of classification accuracy and variability.

Classification accuracy

Feature Importance (10.24s) Condense these slides. Illustrate what is consistent and what is different across scenarios.

Classification model (pop. level) Goals: Build population based classifiers based on features extracted previously. Compare results between normalized and non-normalized features Difficulties: Large subject to subject variability in walking characteristics How to normalize the data? walking is the most prevalent periodic activity normalize everything to walking 𝑧 ∗ = 𝑥−𝑚𝑒𝑑𝑖𝑎𝑛 𝑥 𝑀𝐴𝐷 𝑥 where 𝑀𝐴𝐷 𝑥 =1.4826∗𝑚𝑒𝑑𝑖𝑎𝑛 𝑥−𝑚𝑒𝑑𝑖𝑎𝑛 𝑥

Why do we normalize?

Classification accuracy This will be a 3x3 panel plot of the normalized vs non-normalized results for the 3 activities!

Trees HIP 10.24s ratio.VM < -0.47 SD(VM) >= -0.75 p1_TP >= -1 = Descending stairs = Walking = Ascending Stairs ratio.VM < -0.47 SD(VM) >= -0.75 p1_TP >= -1 Mean(VM) < 4.1

Conclusions We can differentiate between walking on level ground and stair climbing with good accuracy. Best classification accuracy was achieved with accelero-meters worn at the ankles and features extracted using larger window sizes for both subject- and group-levels. We lose some accuracy when building a group-level classifier compared to subject-level. Normalization of features has a greater impact for sensors worn at the hip and wrist versus the ankles.

Acknowledgements Indiana University Jaroslaw Harezlak (Biostatistics, research advisor) Xiaochun Li (Biostatistics, committee member) Constantin Yiannoutsos (Biostatistics, committee member) Andrea Chomistek (Epidemiology, committee member) Steven Albertson (undergraduate intern) Johns Hopkins School of Public Health Jacek Urbanek (Biostatistics, collaborator)

Thank you!