1. 1 Desiging a Virtual Information Telescope using Mobile Phones and Social Participation Romit Roy Choudhury Asst. Prof. (Duke University)

2. 2 Virtual Information Telescope

3. 3 Context Next generation mobile phones will have large number of sensors Cameras, microphones, accelerometers, GPS, compasses, health monitors, …

4. 4 Context Each phone may be viewed as a micro lens Exposing a micro view of the physical world to the Internet

5. 5 Context With 3 billion active phones in the world today (the fastest growing comuting platform …) Our Vision is …

6. 6 Internet A Virtual Information Telescope

7. 7 One instantiation of this vision through a system called Micro-Blog - Content sharing - Content querying - Content floating

8. 8 Content Sharing Virtual Telescope Cellular, WiFi Cellular, WiFi Visualization Service Web Service People Physical Space Phones

9. 9 Content Querying Virtual Telescope Cellular, WiFi Cellular, WiFi Visualization Service Web Service Phones People Physical Space Some queries participatory Is beach parking available? Some queries participatory Is beach parking available? Others are not Is there WiFi at the beach café? Others are not Is there WiFi at the beach café?

10. 10 Content Floating [on physical space] superb sushi Safe@ Nite? Safe@ Nite?

11. 11 If designed carefully, a variety of applications may emerge on Micro-Blog

12. 12 Applications Tourism  View multimedia blogs … query for specifics Micro Reporters  News service with feeds from individuals On-the-fly Ride Sharing  Ride givers advertize intension w/ space-time sticky notes  Respond to sticky notes once you arrive there Virtual order on physical disorder  Land in a new place, and get step by step information RSS Feeds on Location  Inform me when a live band is playing at the mall

13. 13 MiroBlog Prototype Nokia N95 phones  Symbian platform  Carbide C++ code

14. 14 Micro-Blog Beta live at http://synrg.ee.duke.edu/microblog.html

15. 15 Prototype

16. 16 Case Studies Micro-Blog phones distributed to volunteers  12 volunteers 4 phones in 3 rounds 3 weeks  Not great UI Basic training for users  Exit interview revealed useful observations

17. 17 From Exit Interview 1.“Fun activity” for free time Needs much “cooler GUI” 2.Privacy control vital, don’t care about incentives “more interesting to reply to questions … interested in knowing who is asking …” 3.Voice is personal, text is impersonal “Easier to correct text … audio blogs easier but …” 4.Logs show most blogs between 5:00 to 9:00pm Probably better for battery usage as well

18. 18 Thoughts Micro-Blog: Rich space for applications and services But where exactly is the research here ???!!**

19. 19 Problem I Energy Efficient Localization (EnLoc)

20. 20 To GPS or not to GPS GPS is popular localization scheme  Good error characteristics ~ 10m Apps naturally assume GPS  Shockingly, first Micro-Blog demo lasted < 10 hours

21. 21 Cost of Localization Performed extensive measurements  GPS consumes 400 mW, AGPS marginally better  Idle power consumption 55 mW

22. 22 Alternate Localization WiFi fingerprinting, GSM triangulation  Place Lab, SkyHook … Improved energy savings  WiFi 20 hours  GSM 40 hours At the cost of accuracy  40m +  200m +

23. 23 40 Tradeoff Summary: 20 Research Question: Can we achieve the best of both worlds 200

24. 24 Given energy budget, E, Trace T, and location reading costs, e gps, e wifi, e gsm : Schedule location readings to minimize avg. error Given energy budget, E, Trace T, and location reading costs, e gps, e wifi, e gsm : Schedule location readings to minimize avg. error Formulation L(t 0 )L(t 1 ) L(t 2 ) L(t 3 ) L(t 4 ) L(t 6 ) L(t 7 ) Error t0t0 t1t1 t2t2 t3t3 t4t4 t5t5 t6t6 L(t 5 ) t7t7 Accuracy gain from GPS Accuracy gain from WiFi GPS WiFi

25. 25 Dynamic Program Minimize the area under the curve  By cutting the curve at appropriate points  Number of (GPS + WiFi + GSM) cuts must cost < budget

26. 26 Offline optimal offers lower bound on error Online algorithm necessary Online optimal difficult Need to design heuristics

27. 27 Our Approach Do not invest energy if you can predict (even partially)

28. 28 Predictive Heuristics Prediction opportunities exist  Human users are not in brownian motion (exploit inertia)  Exploit habitual mobility patterns  Population distribution can be leveraged Prediction also incorporated into Dynamic Program  Optimal computed on a given predictor Error t0t0 t1t1 t2t2 t3t3 t4t4 t5t5 t6t6 t6t6 Prediction generates different error curve

29. 29 Mobility Profiling Build logical mobility tree per-user  Each link an uncertainty point (UP)  Sample location only when uncertain  Location predictable between UPs Exploit acclerometers  Predict traffic turns  Periodically localize to reset errors Home Gym Road crossing Road crossing Library Grocery Office 8:00 8:15 8:30 12:00 8:05 12:05 3:30 5:30 6:00

30. 30 Humans may deviate from mobility profile Predict based on population statistics Population Statistics Goodwin & Green U-TurnStraightRightLeft E on Green00.8810.0390.078 W on Green000.5960.403 N on Goodwin 00.6400.3590 S on Goodwin 00.51300.486

31. 31 Buy Accuracy with Energy Comparison of optimal with simple interpolation  GPS clearly not the right choice

32. 32 Thoughts Localization cannot be taken for granted  Critical tradeoff between energy and accuracy Substantial room for saving energy  While sustaining reasonably good accuracy However, physical localization  May not be the way to go …  Several motivations to pursue symbolic localization

33. 33 Questions?

34. 34 Problem 2 Symbolic localization (SurroundSense)

35. 35 Symbolic Localization Services may not care about physical location  Symbolic location often sufficient  E.g., coffee shop, movie, park, in-car … Physical to Symbolic conversion  Lookup location name based on GPS coordinate  However, risky RadioShackStarbucks GPS Error range

36. 36 Its possible to localize phones by sensing the ambience such as sound, light, color, movement, orientation… Hypothesis

37. 37 Develop multi-modal fingerprint  Using ambient sound/light/color/movement etc. Starbucks SurroundSense Server Wall RadioShack SurroundSense

38. 38 Each individual sensor not discriminating enough Together, they are quite unique  Use Support Vector Machines to identify uniqueness Fingerprint Database Classification Algorithm (SVM) Location SurroundSense

39. 39 B A C D E GSM provides macro location (mall) SurroundSense refines to Starbucks Should Ambiences be Unique Worldwide?

40. 40 Economics forces nearby businesses to be different Not profitable to have 5 chinese restaurants with same lighting, music, color, layout, etc. SurroundSense exploits this ambience diversity Why will it work? The Intuition:

41. 41 Fingerprints Sound: Color:

42. 42 Fingerprints Light: Movement:

43. 43 + + Ambience Fingerprinting Test Fingerprint Sound Compass Color/Light RF/Acc. Logical Location Fingerprint Filtering & Matching Fingerprint Database = = Candidate Fingerprints Macro Location

44. 44 Full System on Nokia N95 Experimented on 58 stores  10 different clusters  Different parts of Duke campus and in Durham city

45. 45 Full System on Nokia N95 Some classifications were incorrect  But we wanted to know how much incorrect?  We plotted Top-K accuracy  Top-3 accuracy proved to be 100% for all stores

46. 46 Issues and Opportunity Cameras may be inside pockets  Now, we detect when its taken out  Activate cameras, and take pictures  Future phones will be flexible (wrist watch) - see Nokia Morph Electroic compasses can fingerprint layout  Tables and shelves laid out in different orientations  Users forced to orient in those ways

47. 47 Ambience can be a great clue about location Ambient Sound, light, color, movement … None of the individual sensors good enough Combined they may be unique Uniqueness facilitated by economic incentive Businesses benefit if they are mutually diverse in ambience Ambience diversity helps SurroundSense Summary

48. 48 Conclusion The Virtual Information Telescope  A generalization of mobile, location based, social computing Just developing apps  Not enough Many challenges  Energy  Localization  Privacy  Incentives, data distillation … Internet

49. 49 Conclusion Project Micro-Blog  Addressing the challenges systematically  Building a fully functional system with applications  The project snapshot as of today, includes: Micro-Blog: Overall system and application EnLoc: Energy Efficient Localization SurroundSense: Context aware localization CacheCloak: Location privacy via mobility prediction Micro-Blog: Overall system and application EnLoc: Energy Efficient Localization SurroundSense: Context aware localization CacheCloak: Location privacy via mobility prediction

50. 50 PhonePoint Pens Using phone accelerometers  To write short messages in the air

51. 51 Please stay tuned for more at http://synrg.ee.duke.edu Thank You

52. 52 Several research challenges and opportunities 1.Energy-efficient localization 2.Symbolic localization through ambience sensing 3.Location privacy 4.Incentives 5.Spam 6.Information distillation 7.User Inerfacing … Our Research

53. 53 Disclaimer All of our projects are ongoing, hence not fully mature Today’s talk more about the problems than about solutions

54. 54 Today’s Talk Information Telescope Information Telescope Vision System and Applications System and Applications Challenges/Opporunities Ongoing, Future Work Ongoing, Future Work 1.EnLoc 2.SurroundSense 3.CacheCloak