1. 1 Desiging a Virtual Information Telescope using Mobile Phones and Social Participation

2. 2 Virtual Information Telescope

3. 3 Context Next generation mobile phones will have large number of sensors Cameras, microphones, accelerometers, GPS, compasses, health monitors, …

4. 4 Context Each phone may be viewed as a micro lens Exposing a micro view of the physical world to the Internet

5. 5 Context With 3 billion active phones in the world today (the fastest growing comuting platform …) Our Vision is …

6. 6 A Virtual Information Telescope Cloud

7. 7 Telescope Virtual Telescope Cloud Visualization Service Web Service People Physical Space Phones

8. 8 Content Creation Virtual Telescope Cloud Visualization Service Web Service People Physical Space Phones

9. 9 Content Retrieval Virtual Telescope Cloud Visualization Service Web Service Phones Physical Space People

10. 10 MiroBlog Prototype Nokia N95 phones  Symbian platform  Carbide C++ code

11. 11 Prototype

12. 12 Case Studies Micro-Blog phones distributed to volunteers  12 volunteers 4 phones in 3 rounds 3 weeks  Not great UI Basic training for users  Exit interview revealed useful observations

13. 13 From Exit Interview 1.“Fun activity” for free time Needs much “cooler GUI” 2.Privacy control vital, don’t care about incentives “more interesting to reply to questions … interested in knowing who is asking …” 3.Voice is personal, text is impersonal “Easier to correct text … audio blogs easier but …” 4.Logs show most blogs between 5:00 to 9:00pm Probably better for battery usage as well

14. 14 People Virtual Information Telescope Apps Telescope: Rich framework for applications and services

15. 15 Free WiFi? 15

16. 16  Dean’s Office Café  Post-its in the air

17. 17

18. 18 James Duke: Wanted to donate to Princeton to rename as Duke Univ. Duke Trinity College John playing frisbee Tag View

19. 19 Virtual Information Telescope Location Energy Privacy... Apps Research People Incentives Spam Scalability Distillation HCI Mining

20. 20 Problem I Energy Efficient Localization (EnLoc)

21. 21 To GPS or not to GPS GPS is popular localization scheme  Good error characteristics ~ 10m Apps naturally assume GPS  Shockingly, first Micro-Blog demo lasted < 10 hours

22. 22 Cost of Localization Performed extensive measurements  GPS consumes 400 mW, AGPS marginally better  Idle power consumption 55 mW

23. 23 Alternate Localization WiFi fingerprinting, GSM triangulation  Place Lab, SkyHook … Improved energy savings  WiFi 20 hours  GSM 40 hours At the cost of accuracy  40m +  200m +

24. 24 40 Tradeoff Summary: 20 Research Question: Can we achieve the best of both worlds 200

25. 25 Given energy budget, E, Trace T, and location reading costs, e gps, e wifi, e gsm : Schedule location readings to minimize avg. error Given energy budget, E, Trace T, and location reading costs, e gps, e wifi, e gsm : Schedule location readings to minimize avg. error Formulation L(t 0 )L(t 1 ) L(t 2 ) L(t 3 ) L(t 4 ) L(t 6 ) L(t 7 ) Error t0t0 t1t1 t2t2 t3t3 t4t4 t5t5 t6t6 L(t 5 ) t7t7 Accuracy gain from GPS Accuracy gain from WiFi GPS WiFi

26. 26 Dynamic Program Minimize the area under the curve  By cutting the curve at appropriate points  Number of (GPS + WiFi + GSM) cuts must cost < budget

27. 27 Offline optimal offers lower bound on error Online algorithm necessary Online optimal difficult Need to design heuristics

28. 28 Our Approach Do not invest energy if you can predict (even partially)

29. 29 Predictive Heuristics Prediction opportunities exist  Human users are not in brownian motion (exploit inertia)  Exploit habitual mobility patterns  Population distribution can be leveraged Prediction also incorporated into Dynamic Program  Optimal computed on a given predictor Error t0t0 t1t1 t2t2 t3t3 t4t4 t5t5 t6t6 t6t6 Prediction generates different error curve

30. 30 Mobility Profiling Build logical mobility tree per-user  Each link an uncertainty point (UP)  Sample location only when uncertain  Location predictable between UPs Exploit acclerometers  Predict traffic turns  Periodically localize to reset errors Home Gym Road crossing Road crossing Library Grocery Office 8:00 8:15 8:30 12:00 8:05 12:05 3:30 5:30 6:00

31. 31 Humans may deviate from mobility profile Predict based on population statistics Population Statistics Goodwin & Green U-TurnStraightRightLeft E on Green00.8810.0390.078 W on Green000.5960.403 N on Goodwin 00.6400.3590 S on Goodwin 00.51300.486

32. 32 Buy Accuracy with Energy Comparison of optimal with simple interpolation  GPS clearly not the right choice

33. 33 Thoughts Localization cannot be taken for granted  Critical tradeoff between energy and accuracy Substantial room for saving energy  While sustaining reasonably good accuracy However, physical localization  May not be the way to go …  Several motivations to pursue symbolic localization

34. 34 Questions?

35. 35 Problem 2 Symbolic localization (SurroundSense)

36. 36 Symbolic Localization Services may not care about physical location  Symbolic location often sufficient  E.g., coffee shop, movie, park, in-car … Physical to Symbolic conversion  Lookup location name based on GPS coordinate  However, risky RadioShackStarbucks GPS Error range

37. 37 Its possible to localize phones by sensing the ambience such as sound, light, color, movement, orientation… Hypothesis

38. 38 Develop multi-modal fingerprint  Using ambient sound/light/color/movement etc. Starbucks SurroundSense Server Wall RadioShack SurroundSense

39. 39 Each individual sensor not discriminating enough Together, they are quite unique  Use Support Vector Machines to identify uniqueness Fingerprint Database Classification Algorithm (SVM) Location SurroundSense

40. 40 B A C D E GSM provides macro location (mall) SurroundSense refines to Starbucks Should Ambiences be Unique Worldwide?

41. 41 Economics forces nearby businesses to be different Not profitable to have 5 chinese restaurants with same lighting, music, color, layout, etc. SurroundSense exploits this ambience diversity Why will it work? The Intuition:

42. 42 Fingerprints Sound: Color:

43. 43 Fingerprints Light: Movement:

44. 44 + + Ambience Fingerprinting Test Fingerprint Sound Compass Color/Light RF/Acc. Logical Location Fingerprint Filtering & Matching Fingerprint Database = = Candidate Fingerprints Macro Location

45. 45 Full System on Nokia N95 Experimented on 58 stores  10 different clusters  Different parts of Duke campus and in Durham city

46. 46 Full System on Nokia N95 Some classifications were incorrect  But we wanted to know how much incorrect?  We plotted Top-K accuracy  Top-3 accuracy proved to be 100% for all stores

47. 47 Issues and Opportunity Cameras may be inside pockets  Now, we detect when its taken out  Activate cameras, and take pictures  Future phones will be flexible (wrist watch) - see Nokia Morph Electroic compasses can fingerprint layout  Tables and shelves laid out in different orientations  Users forced to orient in those ways

48. 48 Ambience can be a great clue about location Ambient Sound, light, color, movement … None of the individual sensors good enough Combined they may be unique Uniqueness facilitated by economic incentive Businesses benefit if they are mutually diverse in ambience Ambience diversity helps SurroundSense Summary

49. 49 Conclusion The Virtual Information Telescope  A generalization of mobile, location based, social computing Just developing apps  Not enough Many challenges  Energy  Localization  Privacy  Incentives, data distillation … Internet

50. 50 Conclusion Project Micro-Blog  Addressing the challenges systematically  Building a fully functional system with applications  The project snapshot as of today, includes: Micro-Blog: Overall system and application EnLoc: Energy Efficient Localization SurroundSense: Context aware localization CacheCloak: Location privacy via mobility prediction Micro-Blog: Overall system and application EnLoc: Energy Efficient Localization SurroundSense: Context aware localization CacheCloak: Location privacy via mobility prediction

51. 51 PhonePoint Pens Using phone accelerometers  To write short messages in the air

52. 52 Please stay tuned for more at http://synrg.ee.duke.edu Thank You

53. 53 Several research challenges and opportunities 1.Energy-efficient localization 2.Symbolic localization through ambience sensing 3.Location privacy 4.Incentives 5.Spam 6.Information distillation 7.User Inerfacing … Our Research

54. 54 Disclaimer All of our projects are ongoing, hence not fully mature Today’s talk more about the problems than about solutions

55. 55 Today’s Talk Information Telescope Information Telescope Vision System and Applications System and Applications Challenges/Opporunities Ongoing, Future Work Ongoing, Future Work 1.EnLoc 2.SurroundSense 3.CacheCloak

56. 56 One instantiation of this vision through a system called Micro-Blog - Content sharing - Content querying - Content floating

57. 57 Content Sharing Virtual Telescope Cellular, WiFi Cellular, WiFi Visualization Service Web Service People Physical Space Phones

58. 58 Content Querying Virtual Telescope Cellular, WiFi Cellular, WiFi Visualization Service Web Service Phones People Physical Space Some queries participatory Is beach parking available? Some queries participatory Is beach parking available? Others are not Is there WiFi at the beach café? Others are not Is there WiFi at the beach café?

59. 59 Content Floating [on physical space] superb sushi Safe@ Nite? Safe@ Nite?

60. 60 If designed carefully, a variety of applications may emerge on Micro-Blog

61. 61  Free WiFi? 61

62. 62 Applications Tourism  View multimedia blogs … query for specifics Micro Reporters  News service with feeds from individuals On-the-fly Ride Sharing  Ride givers advertize intension w/ space-time sticky notes  Respond to sticky notes once you arrive there Virtual order on physical disorder  Land in a new place, and get step by step information RSS Feeds on Location  Inform me when a live band is playing at the mall

63. 63 Micro-Blog Beta live at http://synrg.ee.duke.edu/microblog.html

64. 64 If designed carefully, the telescope may enable a variety of applications 1. Location based RSS feeds 2. Post-its in the air 3. Tag View

65. 65 Thoughts Micro-Blog: Rich space for applications and services But what are the research challenges here?

66. 66 Virtual Information Telescope Location Energy Privacy Apps Research People Distillation... Incentives Spam Scalability HCI Mining