1 Desiging a Virtual Information Telescope using Mobile Phones and Social Participation Romit Roy Choudhury Asst. Prof. (Duke University)

2 A little bit about ourselves

3 Webpage SyNRG

4 Our Research PHY MAC / Link Network Transport Security Application Incentives Channel fluctuations Spatial Reuse Mobility Energy Savings Eavesdropping Loss Discrimination Privacy Ubiquitous Services Interference Mgmt. What can be enabled (bottom up) What can be enabled (bottom up) What are the visions (top down) What are the visions (top down)

5 Some Ongoing Projects Information Telescope Information Telescope Shuffle Spotlight SmartCast ICNP 2008 MobiSys 2008 Hotnets 2008

6 Today’s Talk Information Telescope Information Telescope Papers: MobiSys 2008 MobiCom workshop (MELT) 2008 Posters, Demos: MobiCom, Sensys, MobiSys

7 Today’s Talk Information Telescope Information Telescope Vision System and Applications System and Applications Challenges/Opporunities Ongoing, Future Work Ongoing, Future Work 1.EnLoc 2.SurroundSense 3.AAMPL 4.CacheCloak

8 Virtual Information Telescope

9 Context Next generation mobile phones will have large number of sensors Cameras, microphones, accelerometers, GPS, compasses, health monitors, …

10 Context Each phone may be viewed as a micro lens Exposing a micro view of the physical world to the Internet

11 Context With 3 billion active phones in the world today (the fastest growing comuting platform …) Our Vision is …

12 Internet A Virtual Information Telescope

13 One instantiation of this vision through a system called Micro-Blog - Content sharing - Content querying - Content floating

14 Content Sharing Virtual Telescope Cellular, WiFi Cellular, WiFi Visualization Service Web Service People Physical Space Phones

15 Content Querying Virtual Telescope Cellular, WiFi Cellular, WiFi Visualization Service Web Service Phones People Physical Space Some queries participatory Is beach parking available? Some queries participatory Is beach parking available? Others are not Is there WiFi at the beach café? Others are not Is there WiFi at the beach café?

16 Content Floating [on physical space] superb sushi Nite? Nite?

17 If designed carefully, a variety of applications may emerge on Micro-Blog

18 Applications Tourism  View multimedia blogs … query for specifics Micro Reporters  News service with feeds from individuals On-the-fly Ride Sharing  Ride givers advertize intension w/ space-time sticky notes  Respond to sticky notes once you arrive there  Negotiate deal on third party server Virtual order on physical disorder  Land in a new place, and get step by step information on your mobile

19 MiroBlog Prototype Nokia N95 phones  Symbian platform  Carbide C++ code

20 Micro-Blog Beta live at

21 Prototype

22 Case Studies Micro-Blog phones distributed to volunteers  12 volunteers 4 phones in 3 rounds 3 weeks  Not great UI Basic training for users  Exit interview revealed useful observations

23 From Exit Interview 1.“Fun activity” for free time Needs much “cooler GUI” 2.Privacy control vital, don’t care about incentives “more interesting to reply to questions … interested in knowing who is asking …” 3.Voice is personal, text is impersonal “Easier to correct text … audio blogs easier but …” 4.Logs show most blogs between 5:00 to 9:00pm Probably better for battery usage as well

24 Thoughts Micro-Blog: Rich space for applications and services But where exactly is the research here ???!!**

25 Several research challenges and opportunities 1.Energy-efficient localization 2.Symbolic localization through ambience sensing 3.Location privacy 4.Incentives 5.Spam 6.Information distillation 7.User Inerfacing … Our Research

26 Disclaimer All of our projects are ongoing, hence not fully mature Today’s talk more about the problems than about solutions

27 Problem I Energy Efficient Localization (EnLoc)

28 To GPS or not to GPS GPS is popular localization scheme  Good error characteristics ~ 10m Apps naturally assume GPS  Shockingly, first Micro-Blog demo lasted < 10 hours

29 Cost of Localization Performed extensive measurements  GPS consumes 400 mW, AGPS marginally better  Idle power consumption 55 mW

30 Alternate Localization WiFi fingerprinting, GSM triangulation  Place Lab, SkyHook … Improved energy savings  WiFi 20 hours  GSM 40 hours At the cost of accuracy  40m +  400m +

31 40 Tradeoff Summary: 20 Research Question: Can we achieve the best of both worlds 400

32 Given energy budget, E, Trace T, and location reading costs, e gps, e wifi, e gsm : Schedule location readings to minimize avg. error Given energy budget, E, Trace T, and location reading costs, e gps, e wifi, e gsm : Schedule location readings to minimize avg. error Formulation L(t 0 )L(t 1 ) L(t 2 ) L(t 3 ) L(t 4 ) L(t 6 ) L(t 7 ) Error t0t0 t1t1 t2t2 t3t3 t4t4 t5t5 t6t6 L(t 5 ) t6t6 Accuracy gain from GPS Accuracy gain from WiFi GPS WiFi

33 Dynamic Program Minimize the area under the curve  By cutting the curve at appropriate points  Number of (GPS + WiFi + GSM) cuts must cost < budget

34 Offline optimal offers lower bound on error Online algorithm necessary Online optimal difficult Need to design heuristics

35 Our Approach Do not invest energy if you can predict (even partially)

36 Predictive Heuristics Prediction opportunities exist  Human users are not in brownian motion (exploit inertia)  Exploit habitual mobility patterns  Population distribution can be leveraged Prediction also incorporated into Dynamic Program  Optimal computed on a given predictor Error t0t0 t1t1 t2t2 t3t3 t4t4 t5t5 t6t6 t6t6 Prediction generates different error curve

37 1. Simple Interpolation Take GPS reading, followed by WiFi  Extrapolate GPS in the direction of WiFi  Reset prediction after threshold time, take GPS again

38 2. Mobility Profiling Build logical mobility tree per-user  Each link an uncertainty point (UP)  Sample location only when uncertain  Location predictable between UPs Exploit acclerometers  Predict traffic turns  Periodically localize to reset errors Home Gym Vine/Mich intersection Vine/Mich intersection Library Grocery Office 8:00 8:15 8:30 12:00 8:05 12:05 3:30 5:30 6:00

39 3. Exploit Group Behavior When user in new mobility path  Exploit mobility patterns of other users Associate probability matrix for traffic intersections  X ij = Probability that car arriving from direction i turns onto j  Use this matrix to predict new user’s behavior Goodwin & Green U-TurnStraightRightLeft E on Green W on Green N on Goodwin S on Goodwin

40 Buy Accuracy with Energy Comparison of optimal with simple interpolation  GPS clearly not the right choice

41 Thoughts Localization cannot be taken for granted  Critical tradeoff between energy and accuracy Substantial room for saving energy  While sustaining reasonably good accuracy However, physical localization  May not be the way to go …  Several motivations to pursue symbolic localization

42 Problem/Opportunity 2 Symbolic localization via ambience sensing (SurroundSense & AAMPL)

43 Symbolic Localization Services may not care about physical location  Symbolic location often sufficient  E.g., coffee shop, movie, park, in-car … Physical to Symbolic conversion possible  Lookup location name based on GPS coordinate  However, risky WalmartStarbucks GPS Error range

44 Our Approach Build symbolic localization algorithms Use low accuracy physical localization as baseline Low accuracy conserves energy

45 SurroundSense Sense ambient light, sound, colors …  Combine sensor readings to generate soft fingerprint For localization, gather fingerprint from mobile  Match with database of fingerprints Of course, fingerprint may not be globally unique  Use rough physical localization as a pre-filter GSM physical location says “you are in the mall” SurroundSense augments that with “you are in Apple Store”

46 SurroundSense Design Prototype on Tmote Invent sensors  Sound and light sensors  Low acoustic frequency range [20, 250] Hz  Currently porting on Nokia N95 phones

47 Fingerprint Extraction Light intensity and sound signals recorded  Fourier transform on sound  Overlapping frequency blocks generated  Each block = 23 bands, 10 Hz each Extract simple features  Each 10 Hz band one feature  Variance of each band another feature  Normalized light intensity another feature  Total - 48 features Train the system with half the data  48 dimensional fingerprint

48 Fingerprint Generation and Matching Match test fingerprint with trained database  Use “Nearest Neighbor” algorithm for classification Location Feature Extract Feature Extract 48 features

49 Results SurroundSense offers consistent localization  Database contains nearby shops in Duke campus  Both sensors > sound > light Pairwise Similarity

50 Symbolic localization can be augmented with phone accelerometers Additional benefits in activity recognition

51 Hypothesis Movement partially indicative of location  People sit in cafes  Run in gyms  Walk up/down aisles in grocery stores  Time duration spent may depend on location Augment location accuracy with acc. signatures  Enable activity recognition as well  E.g., Advertize shoes to users running in the gym

52 AAMPL Acclerometer augmented phone localization Train database with many acc. signatures  Mobile phone 3-axis accelerometer in pockets  Less useful if phone in lady’s handbag Use acc. signatures from individuals in real time  Match against database of signatures  Localize phone  Predict activity Combine accelerometers with sound and light -- fingerprint

53 AAMPL Architecture Client Server based Two-stage classifier  Phones classify sitting/standing  Sends to server  Saves energy  Server classifies location WiFi based basic location  Augmented by AAMPL Server compares classified location with Google’s result

54 AAMPL Classification

55 Evaluation Gathered acc. signatures from many restaurants  Classified with AAMPL, compared with Google Restaurant: Chais Verde Rockfish Fast Food: Chipotle Jimmy John’s Store: Apple Wholefoods Journeys Solstice

56 Evaluation AAMPL classified each location  Compared corresponding location with Google

57 Thoughts Main Idea is that the surrounding is a fingerprint. Effective for separating out nearby contexts. In reality, Spatially clustered shops are diverse by design Aids AAMPL and SurroundSense

58 Problem 3 Location Privacy (CacheCloak)

59 (3) Location Privacy Location information reveals context  Thin line between utility and privacy Pseudonymns  Effective only when infrequent querying from mobiles  Else, spatio-temporal patterns enough to deanonymize Romit’s Office John LeslieJack Susan Alex

60 Location Privacy K-anonymity  Convert location to a space-time bounding box  Ensure K users in the box  Location Service (LS) replies to the boxed region Issues  Not real-time  Poor quality of location  Degrades in sparse regions You Bounding Box K=4

61 Confusion = Privacy Mixing or Cloaking users in space-time  Can offer good QoL Issues  Users need to be present in same space-time location  Else, cloaking needs to be performed post priori A A B B ? ? ? ?

62 Our Objective Real-time, high QoL, entropy guarantees, even in sparse populations

63 Our Approach Exploit mobility prediction to deliberately create mix zones Accurate locations can be revealed that are all confusing to adversary

64 CacheCloak Assume trusted privacy provider  Reveal location to CacheCloak  CacheCloak exposes anonymized location to LS CacheCloak Loc. App1 Loc. App2 Loc. App3 Loc. App4

65 CacheCloak Design User A drives down path P1  P1 is a sequence of locations  CacheCloak has cached response for each location User A takes a new turn (no cached response)  CacheCloak predicts mobility  Deliberately intersects predicted path with other path (P2)  Exposes predicted path to application  Application must reply to queries for entire path Application/adversary confused  New path emanates from both P1 and P2  Not clear where the user came from

66 Example

67 Quantifying Privacy City converted into grid of small sqaures (pixels)  Users are located at a pixel at a given time Each pixel associated with 8x8 matrix  Element (i, j) = probability that user enters i and exits j Probabilities diffuse  At intersections  Over time Privacy = entropy i j pixel

68 Diffusion Probability of user’s presence diffuses  Diffusion gradient computed based on history  i.e., what fraction of users take right turn at this intersection When entropy needs to be increased  Generate spurious branches Time t 1 Time t 2 Time t 3 Road Intersection

69 CacheCloak Benefits Real-time  Response ready when user arrives at predicted location High QoL  Responses can be specific to location  Of course, high overhead due to many responses Entropy guarantees  Entropy increases at traffic intersections  In low regions, desired entropy through false branching Sparse population  Can be handled with dummy users

70 Evaluation Trace based simulation  VanetMobiSim + US Census Bureau trace data  Durham map with traffic lights, speed limits, etc.  Vehicles follow Google map paths  Performs collision avoidance 6km x 6km 10m x 10m pixel 1000 cars 6km x 6km 10m x 10m pixel 1000 cars

71 Results High average entropy  Quite insensitive to user density (good for sparse regions)  Minimum entropy reasonably high

72 Results Length of predictions  Remains reasonably short  Overhead proportional to this length

73 Issues and Limitations CacheCloak overhead  Application replies to lots of queries  However, overhead on wired infrastructure  Caching reduces this overhead significantly CacheCloak assumes same, indistinguishable query  If user asks different query at each road segment  Overhead increases Adaptive branching & dummy users  Offer user-specified privacy guarantee

74 Closing Thoughts Two nodes may intersect in space but not in time Mixing not possible Mobility prediction allows space-time intersections Enables better privacy

75 Conclusion The Virtual Information Telescope  A generalization of mobile, location based, social computing Just developing apps  Not enough Many challenges  Energy  Localization  Privacy  Incentives, data distillation … Internet

76 Conclusion Project Micro-Blog  Addressing the challenges systematically  Building a fully functional system with applications  The project snapshot as of today, includes: Micro-Blog: Overall system and application EnLoc: Energy Efficient Localization SurroundSense & AAMPL: Context aware localization CacheCloak: Location privacy via mobility prediction Micro-Blog: Overall system and application EnLoc: Energy Efficient Localization SurroundSense & AAMPL: Context aware localization CacheCloak: Location privacy via mobility prediction

77 Please stay tuned for more at Thank You