1. Accelerometer wear and non-wear Classification using an Ensemble of Unsupervised Predictors
Madalina Fiterau
Computer Science Department, Mobilize Center
Jennifer Hicks
Scott Delp
Bioengineering Department, Mobilize Center
Manisha Desai
Thomas Robinson
Jorge Armando Banda
Farish Haydel
Kristopher I Kapphahn
Manoj Kumar Sharma
Hyatt Errol Moore IV
Stanford University, School of Medicine
PHYSICAL ACTIVITY MONITORING
ENSEMBLE FOR WEAR DETECTION
RESULTS
Important for understanding wide array of health problems
Obesity, diabetes, heart disease
Cancers, osteoporosis, depression
Typically performed with wrist and/or hip worn accelerometers
Challenges
The device is not worn all the time
Non-wear looks similar to sleep and sedentary behavior
No ground truth on wear time
Meaningful analysis depends on unsupervised weartime detection
Stanford GOALS dataset[1]
260 subjects, 7-11 year old chidren affected by obesity
Triaxial ActiGraph GT3X+ (40Hz), collected 24-hours for 7 days
Physical, physiological, behavioral and psychosocial measures
States of activity and sleep are not known in this data set
Objective: determine the best algorithms to use to discern non-wear/wear particularly for the Stanford GOALS data
EXPERT PREDICTIONS
ACCURACY ESTIMATION
Each algorithm will be used as a base classifier in the ensemble
The accuracy of each method will be estimated in an unsupervised way
Prediction IS weighted according to experts’ estimated accuracy
Different parametrizations of the same algorithm can be included in the ensemble as separate experts
Key idea: Estimate agreement over sets of experts,
write equations/inequalities linking consistency to accuracy, solve system of equations to obtain accuracy estimates
Algorithm
Mean (||Estimated - Actual Error||) over subjects
Sleep Study
Simulated Data
Choi
0.0514
0.0107
0.0502
0.0104
NhanesX
0.0094
0.0469
0.0100
NhanesY
0.0527
0.0120
0.0479
0.0125
NahnesZ
0.0494
0.0131
0.0466
NhanesS
0.0498
0.0110
0.0532
0.0136
PadacoX
0.0116
0.0508
0.0126
PadacoY
0.0478
0.0114
0.0492
0.0109
PadacoZ
0.0489
0.0497
0.0128
PVecMag
0.0485
0.0121
0.0545
GMM
0.0512
0.0102
0.0519
METHOD OF MOMENTS FOR ERROR ESTIMATION
TOP PERFORMING EXPERTS
Agreement between experts i and j
E{i} is the error event for expert i
A{i,j} is the agreement between i and j
a{i,j} = P(E{i}∩E{j}) + P(Ē{i}∩Ē{j})
a{i,j} = 1 – e{i} – e{j} +2e{i,j}
Agreement between experts in a set A
eA is the probability that all functions in A are wrong
aA= P(∩i∈AE{i}) + P(∩i∈AĒ {i})
aA= eA ∑I⊆A (-1)|I| eI
Objective: minimize dependence between error rates
c(e) = ∑A:|A|≥2 (eA - Πi∈Aei)2 subject to eA ≤ mini∈A eA\i
Percentage of correctly identified top performing experts
Sleep Study
Simulated Data
Top1
67.74
66.67
Top2
90.32 90
Top3
70.97 70
Top4
100
Top5
WEARTIME ACCURACY
Accuracy averaged over subjects
Sleep Study
Simulated Data
Vote
0.3938
0.1548
0.3830
0.1452
Best estimated
0.7464
0.2855
0.7386
0.2871
Best (oracle)
WEARTIME DETECTION ALGORITHMS
NHANES[2]
Based on sequences of 0 counts
Applied to each axis separately
R ‘accelerometry’
CHOI[3]
Uses vector magnitude
R ‘Physical Activity’
PADACO[4]
Uses smoothing
CONCLUSIONS
EVALUATION DATA
There is no universally best weartime detection algorithm.
We can reasonably estimate the accuracy of each individual expert based on agreement using the method of moments.
We are able to select the best classifier for each subject.
The original data do not have labels
Data from a sleep study is used in the evaluation
Ground truth is available
31 subjects, 12 hours of monitoring
Each algorithm performs better in certain cases
We estimate expert accuracy for each subject separately
Simulated data based on Stanford GOALS, with added periods of non-wear sampled from known non-wear in Sleep Data
References:
[1] Robinson et al. Family, Community and Clinic Collaboration to Treat Overweight and Obese Children: Stanford GOALS. Contemporary clinical trials
[2]
[3] Choi, Leena, et al. "Validation of accelerometer wear and nonwear time classification algorithm." Medicine and science in sports and exercise, 2011
[4]