Department of Computer and Electrical Engineering A Study of Time-based Features and Regularity of Manipulation to Improve the Detection of Eating Activity Periods During Free-Living MS Defense Exam Jose Luis Reyes Dr. Adam Hoover (chair) Dr. Eric Muth Dr. Richard Groff April 24, 2014

Outline Motivation and Background Design and Methods Results Conclusion

Obesity Common – 34% of U.S. population are obese [ Centers for Disease Control and Prevention ] Serious – 5 th leading risk for global deaths [WHO, 2014] – Heart disease, stroke, type 2 diabetes, and certain types of cancer [Centers for Disease Control and Prevention] Costly – In 2008, annual medical cost was $147 billion in the U.S. [Centers for Disease Control and Prevention] – In 2008, medical cost was $1,429 higher than of those of normal weight. [Centers for Disease Control and Prevention]

Obesity treatments Dietary changes Exercise and activity Behavior changes Weight-loss medication Weight-loss surgery Limit energy intake (EI)* Balancing EI and EE (energy expenditure)

Monitoring EI Most widely used tools Food diary 24-hour recall Food frequency questionnaire Technology-based tools Camera [Martin et al., 2009] Wearable sensors [Amft et al., 2008]

Bite Counter Watch-like device Wrist motion tracking Accelerometer and gyroscope

Previous work Goal: Detection of eating activity periods Based on accelerometer ( AccX, AccY, AccZ ) and gyroscope ( Yaw, Pitch, Roll ) readings Data segmentation Classification of eating activity (EA) and non-eating activity (non-EA) periods based on features Overall accuracy obtained was 81%

Novelty Previous work considered only sensor-based features We consider the time component Time since last eating activity Cumulative eating time Periodicity of manipulation over time Regularity of manipulation

Design and methods Overview of algorithm Data collection New features Regularity of manipulation Time since last EA Cumulative eating time Evaluation metrics

Overview of algorithm ( Dong et al., 2013 ) Data smoothing - Gaussian kernel

Overview of algorithm Sum of acceleration,

Overview of algorithm Data segmentation Peak detection Sum of acceleration Hysteresis threshold

Overview of algorithm Features Manipulation Linear acceleration Wrist roll motion Regularity of roll

Overview of algorithm Naive Bayes Classifier Assign most probable class, c i in C Given features f 1,f 2, …, f N Feature probability

Data collection Collected using iPhone 4 Programmable, large amount of memory, accelerometer and gyroscope Recorded at 15Hz 2 sets of data Set 1: 20 recordings Set 2: 23 recordings A total of 449 hours of data Data training 5 minute non-EA segments Full segments for EA

Current work Motivation: improve previous accuracy of 81% Introduction of 3 new features: Regularity of manipulation Time since last EA Cumulative eating time

Features Feature 1, regularity of manipulation Regularity of peaks around (deg/s)/G Peaks every 10 – 30 seconds? EA manipulation segmentNon-EA manipulation segment

Regularity of manipulation Smooth manipulation data ( N = 225, R = 37.6 ) Compute FFT Compute: Units: (deg/s 3 )/G

Regularity of manipulation Calculate for each segment in data Distribution statistics can be used for Bayes classifier 29>> Distributions (set 1)

Regularity of manipulation Distributions (set 2) 34>>

Features Feature 2, time since last eating activity Time component After a person eats, very unlikely to eat again immediately Probability starts increasing as time passes

Time since last EA Let t last = end time of last segments classified as EA Let t = middle of time of unknown segment currently being classified Then,

Time since last EA Bayes classifier requires probability distributions for both EA and non-EA It is possible to calculate time between meals Nonsensical for opposite class Time since last non-EA? 1 – p(f|EA)

Time since last EA Compute cumulative distribution function (CDF) of time since last EA. p(f|EA) = CDF, p(f|nonEA) = 1 - CDF CDF for time since last EA (set 2)

Features Feature 3, cumulative eating time Time component People spend a certain amount of time eating and drinking in a day(Around 1.1 hrs. according to Dept. of Labor Statistics )

Cumulative eating time At time t, cumulative eating time: Distribution of times involving non events are nonsensical Compute CDF for each recording and average in each data set

Cumulative eating time CDF for cumulative eating time (set 2)

Cumulative eating time p(f|EA) = σ 2 cdf, μ cdf from average CDF p(f|nonEA) = 1 – p(f|EA)

Evaluation metrics Overall accuracy EA accuracy Non-EA accuracy

Results Previous work Statistics Accuracy

Results Regularity of manipulation Statistics Accuracy

Regularity of manipulation (Results) Standard deviation relatively large for EA distribution ( <<18 ) <<18 Set 1’s EA distribution non Gaussian FFT not completely discriminating between EAs and non-EAs

Regularity of manipulation (Results) Smoothed manipulation segment from EA distribution (right tail) Smoothed manipulation segment from non-EA distribution (left tail)

Regularity of manipulation (Results) Smoothed manipulation segment from EA distribution (middle) Smoothed manipulation segment from non-EA distribution (middle) <<20

Regularity of manipulation (Results) Original data for segment in middle of EA distribution Original data for segment in middle of non-EA distribution

Results Time since last EA Statistics Accuracy Set 1Set 2

Time since last EA (Results) Original 4 features Original 4 features + time since last EA

Time since last EA (Results) Original Including time since last EA FPs are strong inhibitors for immediately subsequent data

Results Cumulative eating time Statistics Accuracy Set 1 Set 2

Cumulative eating time (Results) Original 4 featuresOriginal 4 features + cumulative eating time

Cumulative eating time (Results) Original Including cumulative eating time FPs are strong inhibitors for immediately subsequent data

Conclusion FFT not discriminating between EAs and non-EAs completely Time-based features act as clocks Future work Explore regularity of manipulation using non-sinusoidal transform Explore off-line analysis using time-based features so the optimal daily solution can be found (HMMs)

Questions? Thank you!