2. PeopleTones: a system for the detection and notification of buddy proximity on mobile phones Kevin A. Li Timothy Sohn Steven Huang William G. Griswold University of California, San Diego

3. ubiquitous ample computational power a few sensors a few actuators  proactive context-awareness

4. messaging You are driving by Safeway. Reminder: Buy steak. location-based reminders

5. The slopes on Beaver Run have opened! unobtrusive notifications Bzzzzt!

6. crappy sensors crappy actuators cheap sensors could lead to many false notifications cheap actuators could lead to misunderstood cues proactive notification + commodity hardware  flood of meaningless notifications

7. two proximity states: far and near (< 2 city blocks) when a buddy becomes near, play her sound or vibration cue runs on commodity hardware (Windows Smartphone) PeopleTones

8. “It was so cool to see who was home by the time I got home. I could tell if Jenny was home when I passed by University. So if we were going to go eat or something I could ask her. Oh she’s home, so let’s call her and see if she wants to eat.”

9. focus on application goals to avoid over- engineered, impractical solution proximity is easier than location acceptable if notifications missed or not understood approach

10. privacy-friendly proximity detection algorithm technique for reducing sensor noise without sapping power method for generating a language of understandable vibrotactile cues exploratory study of buddy proximity cues contributions

11. proximity detection

12. [LaMarca et al. 2005] build on location? GPS doesn’t work indoors, in urban canyons tower-based systems must keep database current require wardriving how about tower overlap? NearMe [Krumm 2004] iPhone

13. initial data collection used a GSM phone to record the cell towers it saw every 5 minutes 3 GSM phones, kept 1 stationary gather data at a variety of distances (0 - 1.2 miles)

14. initial measurements a and b are the sets of cell towers seen by each phone

15. initial measurements a and b are the sets of cell towers seen by each phone

16. evaluating proximity algorithm can our overlap-ratio algorithm detect proximity accurately enough to support nice-to-know information?

17. requirements cannot be annoying  when the system detects a buddy is near, they should really be near OK to not detect every time  if a buddy is nearby and stationary, we’ll have multiple chances

18. the dataset used the dataset collected by wardriving seattle [Chen et. al., 2006]

19. coverage Suburb Downtown

20. precision 100% precision  every report is valid recall 100% recall  every near incident is detected metric: precision and recall

21. how do we extract the relevant data? only care about when two phones are near or far from each other why not pull out each set of data by different distance thresholds? turns out mobile phone tower readings fluctuate over time (e.g., due to load balancing) we can crosscut the dataset to approximate precision and recall for different scenarios

22. nearby extract pairs of readings taken within 90s 569,264 pairs from Suburb 379,285 pairs from Downtown calculate precision and recall for different ratio threshold values

23. nearby precision 100% precision  every report was valid 100% recall  every near incident was detected suburbdowntown

24. nearby precision suburbdowntown 100% precision  every report was valid 100% recall  every near incident was detected

25. nearby precision suburbdowntown 100% precision  every report was valid 100% recall  every near incident was detected

26. nearby recall 100% precision  every report was valid 100% recall  every near incident was detected

27. reducing sensor noise

28. initial approaches wait for 2 consecutive-same-readings – Too many false positives wait for 3 consecutive-same-readings – Too much delay

29. 2-bit-filter (“eventually 3 more”)

30. filter evaluation f or noise filtering, interested in transitions from far to near and vice-versa extract seattle wardrive readings at 30s intervals try three algorithms on this subset, baseline is single report

31. filter evaluation FilterFalse Positive Reduction 1-same (baseline)0% 2-same-filter53.8% 3-same-filter80.9% 2-bit-filter84.9%

32. you can always increase precision by throwing more power at it what about power consumption?

33. adaptive sampling rate sampling once every 20s kills the phone in less than a day increasing sampling rate to once per 90s helps but introduces a worst-case delay of 270s sample at 90s when in steady state, 20s when transitioning

34. buddy cues

35. mapping music to vibrations

36. problem we want to convert music to vibrations… …but the phone’s vibrator only turns on and off …at single frequency, single amplitude

37. pulse width modulation electric motors do this to save power in the case of vibrotactile motors this also decreases its rotational frequency  perceived as different vibration levels can produce 10 levels of 20ms pulses

38. capturing the essence of music

39. overview of approach just using beat doesn’t always work mapping lyrics doesn’t work well basic idea: convey the current energy level of the music

40. remove noise isolate 6.6kHz to 17.6kHz components using 8 th order Butterworth Filter use amplitude threshold, to keep only components greater than the average

41. take running sum take running sum of absolute value, generate 1 value every 20ms this keeps length consistent

42. exaggerate features compose output from previous step with power function: Ax n,x is sample, A and n are constants, 10<=A<15, 1<=n<=2

43. Beethoven’s 5 th Symphony matching vibration sequence Michael Jackson – Smooth Criminal matching vibration sequence examples (requires imagination)

44. so far…

45. would the techniques we used for proximity detection, sensor noise filtering and vibrotactile cues work in the wild? can peripheral cues be deployed on mobile phones despite poor sensors and actuators? (what experiences can such a system enable?) field study

46. two proximity states, far and near (< 2 blocks) when a buddy is near, play their song if phone is in vibrate mode, play a matching vibrotactile sequence PeopleTones

47. participants 3 groups of friends, 2 weeks

48. could you tell who it was?

49. could you tell who it was?

50. user response to the cue

51. designing peripheral cues for the wild higher comprehension rate when users select their own cues obtrusiveness of music cues was not a concern mapping music to vibration was most successful for people who knew the songs well semantic association is key to learnability

52. user experience

53. “One time at the library, I wanted to eat with someone and so I went outside to call someone. The phone vibrated. I just called the person to meet up.”

54. lessons cues were sensible, but not socially obtrusive proximity algorithm worked well in the wild emphasizing elimination of false positives was effective in combination with 2-bit counter dwelling/lingering led to successful recall

55. whole can be greater than its parts despite crappy sensors and actuators, mobile phones can achieve adequate context awareness and notification with careful system-level design these can be brought together into a useful proactive context-aware application like PeopleTones

56. PeopleTones: a system for the detection and notification of buddy proximity on mobile phones Kevin A. Li Timothy Sohn Steven Huang William G. Griswold University of California, San Diego

57. EXTRA SLIDES

58. measuring vibrations

59. zzz z z z vibrotactile signals

60. far apart unfortunately, there are few points in dataset that are both far apart and proximate in time expected atypical results used the entire dataset 55,181,015 pairs from Suburb 36,769,390 pairs from Downtown

61. far apart precision suburbdowntown 100% precision  every report was valid 100% recall  every near incident was detected

62. far apart precision suburbdowntown 100% precision  every report was valid 100% recall  every near incident was detected

63. far apart precision suburbdowntown 100% precision  every report was valid 100% recall  every near incident was detected

64. far apart recall 100% precision  every report was valid 100% recall  every near incident was detected

65. “It was so cool to see who was home by the time I got home. I could tell if Jenny was home when I passed by University. So if we were going to go eat or something I could ask her. Oh she’s home, so let’s call her and see if she wants to eat.”

66. “One time at the library, I wanted to eat with someone and so I went outside to call someone. The phone vibrated. I just called the person to meet up.”

67. generating vibrations

68. same location Extract pairs of readings taken within 5s 28,625 pairs from Suburb 19,087 pairs from Downtown GPS confirmed 99.9% within 100m of each other

69. same location recall 100% precision => every report was valid 100% recall => every near incident was detected

70. You are driving by Safeway. Reminder: Buy steak. eyes-free notifications

71. [LaMarca et al. 2005] build on location? GPS doesn’t work indoors, in urban canyons tower-based systems must keep database current tower adaptation (e.g., load balancing)

72. location sensing on iphone

73. nearby precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

74. nearby precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

75. far apart precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

76. far apart precision 100% precision => every report was valid 100% recall => every near incident was detected suburbdowntown

77. “It was so cool to see who was home by the time I got home. I could tell if Jenny was home when I passed by University. So if we were going to go eat or something I could ask her. Oh she’s home, so let’s call her and see if she wants to eat.”

78. 2-bit-filter