PeopleTones: a system for the detection and notification of buddy proximity on mobile phones Kevin A. Li Timothy Sohn Steven Huang William G. Griswold University of California, San Diego

phones offer tons of computational power

messaging You are driving by Safeway. Reminder: Buy steak. context-aware reminders

You are driving by Safeway. Reminder: Buy steak. context-aware reminders

the information deluge Mobile phones offer a proactive computing opportunity for context sensing poor sensing could lead to annoying information flood proactive notification + poor sensing = disaster

two states: nearby and far away (> 2 city blocks) when a buddy becomes near, play their cue runs on commodity hardware (Windows Smartphone) PeopleTones

“It was so cool to see who was home by the time I got home. I could tell if Jenny was home when I passed by University. So if we were going to go eat or something I could ask her. Oh she’s home, so let’s call her and see if she wants to eat.”

“One time at the library, I wanted to eat with someone and so I went outside to call someone. The phone vibrated. I just called the person to meet up.”

privacy-friendly proximity detection algorithm technique for reducing sensor noise and power consumption method for generating a language of understandable vibrotactile cues contributions

proximity detection

location sensing on iphone

[LaMarca et al. 2005]

requirements not annoying -> when the system detects a buddy is near, they had better be near since it’s nice to know information, the phone doesn’t have to detect nearby all the time -> if a buddy is nearby, it’s ok if we don’t detect it all the time

initial data collection used a GSM phone to record the cell towers it sees every 5 minutes 3 GSM phones, kept 1 stationary try at a variety of distances

initial measurements a and b are the sets of cell towers seen by each phone

initial measurements a and b are the sets of cell towers seen by each phone

evaluating proximity algorithm

Could our overlap-ratio algorithm be used to detect proximity in a decently accurate way?

the dataset used the dataset collected by from wardriving seattle 1 [Chen et. al., 2006]

coverage Suburb Downtown

precision 100% precision => every report was valid recall 100% recall => every near incident was detected what metric do we use?

how do we use this dataset? we care about when two phones are: near each other far from each other

how do we extract the relevant data? why not pull out each set of data by different distance thresholds? turns out mobile phone tower readings fluctuate over time we know phones are in the same car so extract pairs sampled within different time intervals (work on wording)

nearby Extract pairs of readings taken within 90s 569,264 pairs from Suburb 379,285 pairs from Downtown

nearby precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

nearby precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

nearby precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

nearby precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

100% precision => every report was valid 100% recall => every near incident was detected nearby recall

far apart precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

far apart precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

far apart precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

far apart precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

far apart recall 100% precision => every report was valid 100% recall => every near incident was detected

reducing sensor noise

potential approaches wait for 2 consecutive-same-readings – Too many false positives wait for 3 consecutive-same-readings – Too much delay

2-bit-filter

filter evaluation For noise filtering, interested in transitions from near to far and vice-versa Extract readings at 30s intervals Try three algorithms on this new subset, baseline is single report

filter evaluation FilterFalse Positive Reduction 1-same (baseline)0% 2-same-filter53.8% 3-same-filter80.9% 2-bit-filter84.9%

Can always increase precision by throwing more power at it What about power consumption?

minimizing power

We can always increase accuracy by sampling faster What about power?

adaptive sampling rate sampling once every 20s kills the phone in less than a day increasing sampling rate to once per 90s helps but introduces a worst-case delay of 270s sample at 90s when in steady state, 20s when transitioning

mapping music to vibrations

problem we want to convert music to vibrations… …but the phone’s vibrator only turns on and off

pulse width modulation electric motors and light dimmers do this to save power in the case of vibrotactile motors this also decreases the rate at which it spins -> allows us to create different vibration levels

capturing the essence of music

high level approach just using beat doesn’t always work mapping lyrics doesn’t work well

remove noise isolate 6.6kHz to 17.6kHz components using 8 th order Butterworth Filter use amplitude threshold, to keep only components greater than the average

take running sum take running sum of absolute value, generate 1 value every 20ms this keeps length consistent

exaggerate features compose output from previous step with power function: Ax n,x is sample, A and n are constants, 10<=A<15, 1<=n<=2

Beethoven’s 5 th Symphony matching vibration sequence Michael Jackson – Smooth Criminal matching vibration sequence samples

so far…

what’s the research question Field study

only two states, nearby and far away when a buddy is near, play their song if phone is in vibrate mode, play a matching vibrotactile sequence PeopleTones

participants 3 groups of friends, 2 weeks

what did you do?

user response to the cue

Could you tell who it was?

lessons cues in the wild should be music higher comprehension rate when users select their own cues obtrusiveness of music cues was not a concern mapping music to vibration was most successful for people who knew the songs well semantic association is key Proximity Power Lessons shouldn’t be a bout peopletones. Lessons should be about what we learned about our algorithms onproximity, power, all that stuff

Next steps

High level idea What’s the big takeaway? We have crappy sensors and actuators in mobile phones. With some careful development, we can take advantage of their ubiquity and turn them into context-aware computing devices that can give us nice to know information without bugging us. This information can be learned, if mapped from somethin we already have semantic associations with, such as music

EXTRA SLIDES

generating vibrations

measuring vibrations

zzz z z z vibrotactile signals

You are driving by Safeway. Reminder: Buy steak. The slopes on Beaver Run have opened! unobtrusive notifications

same location Extract pairs of readings taken within 5s 28,625 pairs from Suburb 19,087 pairs from Downtown GPS confirmed 99.9% within 100m of each other

same location recall 100% precision => every report was valid 100% recall => every near incident was detected