1. Power-Efficient and Adaptive Duty-Cycling of Smartphone Activity Recognition Systems
Presented by: Vijay Srinivasan (Samsung Research)
Collaborators:
Thomas Phan (Samsung Research)
Kiran Rachuri, Theus Hossmann, Cecilia Mascolo (University of Cambridge)

2. Smartphone activity recognition
Sensors include microphone, accelerometer, GPS, etc.
Walking
Driving
Biking
Idle
Running
Current generation
on-device sensors
Sensor sampling
Signal processing
Classification model
Classification engine
Feature extraction
What is sar?
What sensors are used?
Real-time, on-device activity recognition

3. Applications Physical activity monitoring Automatic Driving UI
Two example applications
Explain them
Automatic Driving UI

4. Interesting wellness visualizations
Instead of a boring pie-chart, researchers have come up with more interesting visualizations.
For exam
Ubifit from Intel, 2008
BeWell from Dartmouth, 2011
Social motivation in the future

5. Battery drain is a problem
Continuous activity recognition should not reduce the Smartphone's battery life
Aim to keep battery drain per day due to activity recognition to less than 20%
Avoid heavy use of power-hungry sensors
GPS, sound, etc.

6. Battery drain is a problem
Using accelerometers is power-efficient.
What percentage of your phone’s battery charge is drained per day due to always-on activity recognition? *
Keeping the smartphone CPU awake to collect and process data: 72.5% !
Total battery drain per day: 88.4% !
Data acquisition: 13.9%
Data processing: 2.0%
* Android on Galaxy SII

7. Reducing battery drain
Do not keep the activity recognition system running all the time.
Less time spent in power-hungry ON state
Example of duty-cycling
User activities:
Idle
Walking
Define wake time here
Idle
Idle
Walking
Walking
ON power: ~ 250 mW
Power
Activity recognition ON
OFF power: ~ 9 mW
Activity recognition OFF
Time

8. Two key questions
How much sensor data is needed in the ON state to accurately infer the user’s current activity?
Sampling duration
How long can the OFF state be maintained before we miss the user’s next activity transition?
Sleep duration
User activities:
Idle
Walking
Idle
Idle
Walking
Walking
Sleep duration
Power
Activity recognition ON
Activity recognition OFF
Time
Sampling duration

9. Goals
Reduce sampling duration and accurately infer user’s current activity
Increase sleep duration as much as possible while missing as few activity transitions as possible
In other words, reduce the wake time of the activity recognition system
Wake time is the proportion of all time when system is ON
User activities:
Idle
Walking
Idle
Idle
Walking
Walking
Sleep duration
Power
Activity recognition ON
Activity recognition OFF
Time
Sampling duration

10. Adaptive duty-cycling
Adaptive sleep durations
Longer sleep durations for missable (Idle) event stream
Shorter sleep durations for unmissable (Walking, Running, etc.) event stream
User activities:
Idle
Walking
Idle
Idle
Walking
Walking
Power
Activity recognition ON
Activity recognition OFF
Time

11. Adaptive duty-cycling
Adaptive sampling durations
Shorter sampling durations for idle events
Longer sampling durations for active events
User activities:
Idle
Walking
Idle
Idle
Walking
Walking
Power
Activity recognition ON
Activity recognition OFF
Time

12. Adaptive duty-cycling
Leverage user’s natural idle periods to save power
Adaptive duty-cycling
Achieves 90% of the accuracy of an always-on system
Reduces wake time by 96.7%
Reduces power consumption by 81.9%
User activities:
Idle
Walking
Idle
Idle
Walking
Walking
Power
Activity recognition ON
Activity recognition OFF
Time

13. Outline Related work Our approach Experimental Setup Evaluation
Activity Recognition
Adaptive Duty-Cycling
Experimental Setup
Evaluation
Example applications
Future directions

14. Related work on activity recognition
Early work on mobile sensor-based activity recognition used specialized on-body sensors
Bao 2004, Lester 2006, Tapia 2007, Consolvo 2008
Smartphones are fast becoming the platform of choice for mobile activity recognition research
Miluzzo 2008 (CenceMe), Lu 2010 (Jigsaw), Reddy 2010
We treat battery use as a first-class challenge for accelerometer-based activity recognition

15. Related work on power-efficient smartphone activity recognition
First work to our knowledge to systematically address the key bottleneck for accelerometer-based activity recognition
Approach
Idea
Reduces power from data acquisition? (13.9% battery) per day)
Reduces power from data processing?
(2% battery per day)
Reduces power from keeping smartphone awake? (72.5% battery per day)
Triggered sensing (Lu et al, Sensys 2010)
Trigger high power sensor such as GPS based on low power accelerometer
Yes No
Activity-adaptive approach (Yan et al, ISWC 2012)
Vary sampling rate and set of features used (time-domain vs. frequency domain) to save power
Admission control (Rachuri et al, Ubicomp 2010; Lu et al, Sensys 2010)
Use heuristics to determine if sensor data window should be sent to classifier

16. Outline Related work Our approach Experimental Setup Evaluation
Activity Recognition
Adaptive Duty-Cycling
Experimental Setup
Evaluation
Example applications
Future directions

17. Overview of activity recognition
Supervised classification with C4.5 decision trees
Offline training phase to create decision tree (stored as a JSON file)
Real-time classification uses decision tree from JSON file

18. Feature extraction Derive three orientation independent data streams
For each stream:
Segment into fixed-size time windows
Compute time and frequency domain features for each window

19. Feature extraction Time-domain features Frequency-domain features
Mean, standard deviation, entropy, etc.
Frequency-domain features
Energy, highest magnitude frequency, etc.

20. Evaluation on scripted activities
Scripted activity data collection from 17 test subjects
Over 6 hours of data
Research lab members, late-20s to 50s
Activities: walking (on a at surface), walking up stairs, walking down stairs, running, bicycling, driving, standing, and sitting.

21. Evaluation on scripted activities
Perform 10-fold cross-validation to evaluate recognition accuracy
Per window recognition accuracy is 98.4% (for 4-second windows)
More rigorous field evaluation in the future

22. Outline Related work Our approach Experimental Setup Evaluation
Activity Recognition
Adaptive Duty-Cycling
Experimental Setup
Evaluation
Example applications
Future directions

23. Adaptive sleep scheduling algorithm
Maintain a probability of sensing
On each activity output by classifier:
If interesting (say walking), set
If missable (say idle), set
Ramp-up and ramp-down factors
User activities:
Idle
Walking
Idle
Idle
Walking
Walking
Power
Activity recognition ON
Activity recognition OFF
Time

24. Adaptive sleep scheduling algorithm
Initialize sleep interval
Repeatedly increment by step interval until a uniform random number from is greater than the probability of sensing
Key parameters
Ramp-up and ramp-down factors
Step interval
Minimum sleep interval

25. Adaptive sampling durations with a two-tier classifier
Tier I
Tier II
Two key insights
Performing accurate idle vs. non-idle classification requires a smaller wake time compared to full activity classification
Leverage user’s natural idle periods to save power
User activities:
Idle
Walking
Idle
Idle
Walking
Walking
Power
Activity recognition ON
Activity recognition OFF
Time

26. Adaptive sampling durations with a two-tier classifier
Train two decision tree classifiers offline with two different sampling windows
Tier I Idle classifier
Use idle window size of seconds
Recognizes only idle vs. non-idle (transform training labels to idle or non-idle)
Tier II activity classifier
Use active window size of 4-8 seconds
Recognizes entire set of physical activities such as walking, driving, idle, biking etc.
Tier I
Tier II
Two-Tier classifier

27. Smartphone implementation
Acquire wake lock to ensure that activity recognition is not interrupted
Sample tri-axial accelerometer data at 32 Hz
Perform feature extraction and two-tier activity classification
Pass activity result to adaptive sleep scheduling algorithm to compute sleep duration T
Set an alarm T seconds into the future
Release wake lock to allow smartphone to enter low power sleep mode
And repeat!
System implemented on both Android and Tizen
Set alarm
and release wakelock
Alarm triggered
Acquire wakelock
Power
Activity recognition ON
Phone in sleep mode
Time

28. Outline Related work Our approach Experimental Setup Evaluation
Activity Recognition
Adaptive Duty-Cycling
Experimental Setup
Evaluation
Example applications
Future directions

29. Naturalistic data collection
Goal: Collect long-term naturalistic smartphone accelerometer data to evaluate adaptive duty-cycling
Continuous accelerometer data at 65 Hz
~ 35 MB per day after compression
No ground truth collected
Challenging to obtain long-term, continuous ground truth

30. Evaluation metric for accuracy
Use always-on activity recognition output as ground truth
Interpret accuracy as a percentage of always-on classifier accuracy.
Four coarse-grained activities: walking, running, idle, and driving.
Metric: Aggregate per class accuracy
For each class (say driving), compute time slice accuracy:
Percentage of seconds that are correctly classified to the target class (driving) across all users
Aggregate per class accuracy is the average time slice accuracy across all classes.

31. Evaluation metrics for power
Wake time: Proportion of time when activity recognition system is ON
Platform independent measure
Empirical power consumption in mW
Platform dependent
May vary greatly for the same wake time measure depending on sleep and wakeup overhead

32. Schemes implemented for analysis
Four variants implemented for analysis:
Fixed sleep duration with fixed sample duration
Adaptive sleep duration with fixed sample duration
Fixed sleep duration with adaptive sample duration
Adaptive sleep duration with adaptive sample duration

33. Parameter exploration in each scheme
Goal: Given an accuracy lower bound, compute the lowest wake time or power consumption achievable by each scheme over the parameter space
For example, the best power consumption achievable by scheme X to achieve an accuracy greater than 90% is 200 mW

34. Outline Related work Our approach Experimental Setup Evaluation
Activity Recognition
Adaptive Duty-Cycling
Experimental Setup
Evaluation
Example applications
Future directions

35. Wake Time vs. Accuracy
Adaptive duty-cycling performs best with a wake time of less than 4% for a 90% accuracy bound
Adaptive sample durations outperform adaptive sleep scheduling

36. Wake time and power consumption
Significant additional overhead for each wakeup and sleep
Ramp-up costs during wakeup
Lag time during sleep
Ramp-up cost

37. Wake time and power consumption
When comparing duty-cycling schemes, better wake time does not imply better power consumption
Schemes that wake up fewer times are better able to handle the wakeup and sleep overhead
Ramp-up cost

38. Power consumption evaluation
Goal: For each duty-cycling scheme, evaluate best power consumption achievable given an accuracy lower bound
Compute power consumption by extrapolating empirical measurements of sleep and wakeup overhead
Simple linear model to capture the relationship between the energy of each wakeup and the duration of the wakeup

39. Power consumption vs. Accuracy
Adaptive duty-cycling achieves a power consumption of ~45 mW at 90% accuracy
Reduces always-on power (253 mW) by 81.9%!
Adaptive sleep scheduling outperforms adaptive sample durations, by waking up fewer times and reducing the wakeup/sleep overhead

40. Conclusions
Adaptive duty-cycling schemes leverage the user’s natural idle periods to save power
Achieves 90% of the accuracy of an always-on system
Reduces wake time by 96.7%
Reduces power consumption by 81.9%
Battery drain for activity recognition is greatly reduced
99.5% battery drain per day for always-on system
17.7% battery drain per day for adaptive duty-cycling

41. Outline Related work Our approach Experimental Setup Evaluation
Activity Recognition
Adaptive Duty-Cycling
Experimental Setup
Evaluation
Example applications
Future directions

42. Log visualization Visualization application on Tizen
Without duty-cycling, battery does not last a single day

43. Narrative generation We log:
User activities
User places on major activity transitions (reverse geocoding)
Use natural language generation techniques to create an activity narrative
Intelligent GPS triggering

44. Narrative generation Example narrative: Potential applications:
On December 9, I took a 2 hour, 23 minute round trip from my home. I started the day at 10:31 AM by driving for 10 miles from my home to Samsung R&D Center where I stayed for 31 minutes. I then went 1 mile from there to Subway Sandwiches on First Street where I remained for 28 minutes. After that, I drove to Valley Fair Mall where I hung out for 37 minutes. I finally drove for 12 miles to my home in San Jose.
Potential applications:
Activity summarization for the blind
Multimedia captions for descriptive story-telling

45. Outline Related work Our approach Experimental Setup Evaluation
Activity Recognition
Adaptive Duty-Cycling
Experimental Setup
Evaluation
Example applications
Future directions

46. Future directions
Exploit relationships between sensor types and activity predicates to save power
E.g.) At home -> Not driving, At work -> Not sleeping
Low power cores dedicated for sensor data acquisition and processing
Lower overhead with each wakeup/sleep
Potentially execute Tier I on low-power core and Tier II if necessary on higher power core

47. Thanks Questions?

48. Backup slides

49. Spurious classification pruning