PeopleTones: a system for the detection and notification of buddy proximity on mobile phones Kevin A. Li Timothy Sohn Steven Huang William G. Griswold University of California, San Diego

phones offer tons of computational power

messaging You are driving by Safeway. Reminder: Buy steak. context-aware reminders

You are driving by Safeway. Reminder: Buy steak. context-aware reminders

the information deluge Mobile phones offer a proactive computing opportunity for context sensing poor sensing could lead to annoying information flood proactive notification + poor sensing = disaster

two states: nearby and far away (> 2 city blocks) when a buddy becomes near, play their cue runs on commodity hardware (Windows Smartphone) PeopleTones

“It was so cool to see who was home by the time I got home. I could tell if Jenny was home when I passed by University. So if we were going to go eat or something I could ask her. Oh she’s home, so let’s call her and see if she wants to eat.”

“One time at the library, I wanted to eat with someone and so I went outside to call someone. The phone vibrated. I just called the person to meet up.”

privacy-friendly proximity detection algorithm technique for reducing sensor noise and power consumption method for generating a language of understandable vibrotactile cues exploratory study of buddy proximity cues contributions

proximity detection

location sensing on iphone

[LaMarca et al. 2005]

requirements not annoying -> when the system detects a buddy is near, they had better be near since it’s nice to know information, the phone doesn’t have to detect nearby all the time -> if a buddy is nearby, it’s ok if the system doesn’t detect it all the time

initial data collection used a GSM phone to record the cell towers it sees every 5 minutes 3 GSM phones, kept 1 stationary try at a variety of distances ( miles)

initial measurements a and b are the sets of cell towers seen by each phone

initial measurements a and b are the sets of cell towers seen by each phone

evaluating proximity algorithm

Could our overlap-ratio algorithm detect proximity accurately enough to support nice-to-know information?

the dataset used the dataset collected by from wardriving seattle 1 [Chen et. al., 2006]

coverage Suburb Downtown

precision 100% precision => every report was valid recall 100% recall => every near incident was detected what metric do we use?

how do we use this dataset? we care about when two phones are: near each other far from each other

how do we extract the relevant data? why not pull out each set of data by different distance thresholds? turns out mobile phone tower readings fluctuate over time we know phones are in the same car so extract pairs sampled within different time intervals (work on wording)

nearby Extract pairs of readings taken within 90s 569,264 pairs from Suburb 379,285 pairs from Downtown

nearby precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

nearby precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

nearby precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

nearby precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

100% precision => every report was valid 100% recall => every near incident was detected nearby recall

far apart Used the entire dataset 55,181,015 pairs from Suburb 36,769,390 pairs from Downtown

far apart precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

far apart precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

far apart precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

far apart precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

far apart recall 100% precision => every report was valid 100% recall => every near incident was detected

reducing sensor noise

potential approaches wait for 2 consecutive-same-readings – Too many false positives wait for 3 consecutive-same-readings – Too much delay

2-bit-filter

filter evaluation f or noise filtering, interested in transitions from near to far and vice-versa extract readings at 30s intervals try three algorithms on this new subset, baseline is single report

filter evaluation FilterFalse Positive Reduction 1-same (baseline)0% 2-same-filter53.8% 3-same-filter80.9% 2-bit-filter84.9%

you can always increase precision by throwing more power at it What about power consumption?

minimizing power

adaptive sampling rate sampling once every 20s kills the phone in less than a day increasing sampling rate to once per 90s helps but introduces a worst-case delay of 270s sample at 90s when in steady state, 20s when transitioning

mapping music to vibrations

problem we want to convert music to vibrations… …but the phone’s vibrator only turns on and off

pulse width modulation electric motors and light dimmers do this to save power in the case of vibrotactile motors this also decreases the rate at which it spins -> allows us to create different vibration levels

capturing the essence of music

high level approach just using beat doesn’t always work mapping lyrics doesn’t work well

remove noise isolate 6.6kHz to 17.6kHz components using 8 th order Butterworth Filter use amplitude threshold, to keep only components greater than the average

take running sum take running sum of absolute value, generate 1 value every 20ms this keeps length consistent

exaggerate features compose output from previous step with power function: Ax n,x is sample, A and n are constants, 10<=A<15, 1<=n<=2

Beethoven’s 5 th Symphony matching vibration sequence Michael Jackson – Smooth Criminal matching vibration sequence samples

so far…

Would the techniques we used for proximity detection, sensor noise filtering and vibrotactile cues work in the wild? Can peripheral cues be deployed on mobile phones despite poor sensors and actuators? field study

only two states, nearby and far away when a buddy is near, play their song if phone is in vibrate mode, play a matching vibrotactile sequence PeopleTones

participants 3 groups of friends, 2 weeks

what did you do?

user response to the cue

could you tell who it was?

designing peripheral cues for the wild higher comprehension rate when users select their own cues obtrusiveness of music cues was not a concern mapping music to vibration was most successful for people who knew the songs well semantic association is key to learnability

lessons proximity algorithm worked well in the wild cues were not socially obtrusive emphasizing elimination of false positives is effective in combination with 2-bit counter dwelling/lingering leads to successful recall

mobile phones for context awareness despite crappy sensors and actuators, mobile phones can be used for context awareness careful system level design allows the exploration of context-aware applications

PeopleTones: a system for the detection and notification of buddy proximity on mobile phones Kevin A. Li Timothy Sohn Steven Huang William G. Griswold University of California, San Diego

EXTRA SLIDES

generating vibrations

measuring vibrations

zzz z z z vibrotactile signals

You are driving by Safeway. Reminder: Buy steak. The slopes on Beaver Run have opened! unobtrusive notifications

same location Extract pairs of readings taken within 5s 28,625 pairs from Suburb 19,087 pairs from Downtown GPS confirmed 99.9% within 100m of each other

same location recall 100% precision => every report was valid 100% recall => every near incident was detected