A two-tier classifier for duty-cycling smartphone activity recognition systems Vijay Srinivasan Thomas Phan ACM PhoneSense, November 6, 2012, Toronto, Canada

Background Walking Driving Biking Idle Running Current generation on-device accelerometers Sensor sampling Signal processing Classification model Classification engine Feature extraction Real-time, on-device activity recognition Smartphone activity recognition using accelerometers has many exciting use cases Fitness and health tracking Context driven UI Smartphone activity recognition is the process of transforming raw sensor data to user understandable activities such as walking, driving, biking etc using a combination of signal processing and machine learning techniques. In particular, smartphone activity recognition using accelerometers has many exciting use cases. It can be used for tracking and visualizing the user’s health and fitness level It can be used to trigger a change in the UI based on the user’s current activity; for example, driving mode UI can be automatically activated if we detect that the user is driving.

Problem statement Activity recognition in commodity Samsung smartphones is power hungry Bulk of the power consumption comes from keeping the smartphone awake to perform activity recognition Problem: How to reduce the amount of time the smartphone stays awake (wake time) to respond to a query for current user activity? Power consumption breakdown of our accelerometer-based activity recognition system Power

Solution overview: Two-Tier classification approach Two-Tier classifier Tier I Tier II Two key insights Performing accurate idle vs. non-idle classification requires a smaller wake time compared to full activity classification Leverage user’s natural idle periods to save power Example operation Walking activity inferred using Tier II Power consumption of 230 mW Idle activity inferred using Tier I Power Activity recognition ON Time Power

Contributions First approach to dynamically change the wake time of the activity recognition system to reduce power Reduces wake time and power consumption of activity recognition Wake time reduction of 70.1% and 93.0% compared to fixed wake time scheme for 2 test subjects At 91% accuracy lower bound Higher reduction in power under high accuracy or low latency requirements. Enables applications such as: Context-driven UI Context-driven reminder applications

Prior art Existing power reduction research in mobile sensing is not very useful for accelerometer-based activity recognition since Wake lock is the main bottleneck consuming ~200 mW Sensor sampling and classification only consume ~30-40 mW Approach Idea Reduces power from sensor sampling? Reduces power from activity classification? Reduces power from keeping smartphone awake? Triggered sensing (Lu et al, Sensys 2010) Trigger high power sensor such as GPS based on low power accelerometer Yes No Activity-adaptive approach (Yan et al, ISWC 2012) Vary sampling rate and set of features used (time-domain vs. frequency domain) to save power Admission control (Rachuri et al, Ubicomp 2010; Lu et al, Sensys 2010) Use heuristics to determine if sensor data window should be sent to classifier

Prior art Adaptive sensing approach (Rachuri et al, MobiCom 2011) Does address the high power consumption of keeping smartphone awake Adapts the sleep period between wake times dynamically to save power Is being used in combination with the two-tier approach Is not very useful in reducing power when there are low latency requirements of less than 30 seconds

Detail 1/4: Duty-cycling overview Acquire wake lock to ensure that activity recognition is not interrupted Sample tri-axial accelerometer data at 32 Hz Perform feature extraction and two-tier activity classification Set an alarm for when the smartphone should wake up next Release wake lock to allow smartphone to enter low power sleep mode And repeat! Set alarm and release wakelock Alarm triggered Acquire wakelock Power Activity recognition ON Phone in sleep mode Time

Detail 2/4: Feature extraction Transform tri-axial accelerometer data into three orientation-independent axes Segment accelerometer time series into finite sampling windows of duration T seconds Transform each sampling window to a feature vector for use by classifiers Time-domain features: Mean, variance etc Frequency-domain features: Highest magnitude frequency etc. Data acquisition from accelerometer Tri-axis normalization Sliding-window partitioning FFT computation Feature extraction Features Classification

Detail 3/4: Offline classifier training Use labeled accelerometer data to build a decision tree classifier model using the C4.5 algorithm Train two classifiers offline with two different sampling windows Tier I Idle classifier Use idle window size of 0.5-1 seconds Recognizes only idle vs. non-idle (transform training labels to idle or non-idle) Tier II activity classifier Use active window size of 4-8 seconds Recognizes entire set of physical activities such as walking, driving, idle, biking etc.

Detail 4/4: Online activity classification Use Tier I idle classifier with 1 second sample window to determine if user is idle If idle, go to sleep immediately to save power If non-idle, extend wake time to 4-8 seconds and use Tier II activity classifier to identify physical activity Tier I Tier II

Two additional solutions implemented for comparison State of the art single-Tier scheme Example: Always use fixed wake time of 4 seconds Confidence-based multi-tier scheme Obtain confidence from decision tree node accuracy Example: Continue sampling till confidence > 90% or maximum window size is reached

Empirical evaluation Power vs. accuracy tradeoff evaluation Three schemes evaluated on labeled accelerometer data Scheme I: Single-Tier scheme Scheme II: Confidence-based multi-Tier Scheme Scheme III: Two-Tier scheme Use naturalistic labeled activity data collected from two subjects

Evaluation metrics Wake time percentage Classification accuracy Percentage of time that the two-tier classifier keeps the smartphone awake Example: 30% wake time percentage means phone is awake 30% of the time due to the activity recognition system alone Classification accuracy Percentage of time slices correctly classified by the activity recognition system Assumes interpolation when sleep periods are introduced Example: 90% classification accuracy means that the two-tier classifier is correct about the user’s activities 90% of the time

Parameter settings for each scheme Multiple parameter values possible for each of the three schemes Perform a brute force exploration of the parameter space to determine the best instance of each scheme

Experiment #1 Compute the lowest wake time percentage (over all possible parameter values) for each scheme given a lower bound on classification accuracy Observation: Significant reduction in wake time percentages for two-tier approach at high classification accuracy lower bounds Subject I results Subject II results

Experiment #2 Compute the lowest wake time percentage (over all possible parameter values) for each scheme given two constraints Lower bound on classification accuracy (shown along Y-axis) Upper bound on sleep period in seconds (shown on X-axis) Observation: About 50% drop in wake time percentage (compared to best of either scheme I and II) for latency bounds less than 60 seconds

Effect of lag time and ramp-up on energy consumption - Tier I Energy consumption of Tier I wakeup spike on galaxy S2with idle window of 1 second: 148.7 uAh Tier I classification Sleep mode Sleep mode Lag time

Effect of lag time and ramp-up on energy consumption - Tier II Energy consumption of Tier II wakeup spike on Galaxy S2with active window of 4 seconds: 330.8 uAh Tier II classification Sleep mode Sleep mode Lag time

Conclusions and future work First approach to dynamically change the wake time of the activity recognition system to reduce power Reduces wake time and power consumption of activity recognition Wake time reduction of 70.1% and 93.0% compared to fixed wake time scheme for 2 test subjects At 91% accuracy lower bound Higher reduction in power under high accuracy or low latency requirements. Enables applications such as: Context-driven UI Context-driven reminder applications Ongoing work Integrate two-tier wakeup with aggressive sleep periods Incorporate empirical energy models into evaluation Explore multi-tier confidence based approach with probabilistic classifier