1. 1 1 CSCE 5013: Hot Topics in Mobile and Pervasive Computing Discussion of LOC1 and LOC2 Nilanjan Banerjee Hot Topic in Mobile and Pervasive Computing University of Arkansas Fayetteville, AR nilanjan.banerjee@gmail.com Acknowledgment: Romit Roychoudhuri for the slides

2. 2 2 LOC2: SurroundSense

3. 3 Location-Based Applications (LBAs) For Example:  GeoLife shows grocery list when near Walmart  MicroBlog queries users at a museum  Location-based ad: Phone gets coupon at Starbucks iPhone AppStore: 3000 LBAs, Android: 500 LBAs

4. 4 Most emerging location based apps do not care about the physical location GPS: Latitude, Longitude

5. 5 Most emerging location based apps do not care about the physical location Instead, they need the user’s logical location GPS: Latitude, Longitude Starbucks, RadioShack, Museum, Library

6. 6 Physical Vs Logical Unfortunately, most existing solutions are physical  GPS  GSM based  Google Latitude  RADAR  Cricket  …

7. 7 Given this rich literature, Why not convert from Physical to Logical Locations?

8. 8 Physical Location Error

9. 9 Pizza HutStarbucks Physical Location Error

10. 10 Pizza HutStarbucks Physical Location Error The dividing-wall problem

11. 11 SurroundSense: A Logical Localization Solution

12. 12 It is possible to localize phones by sensing the ambience It is possible to localize phones by sensing the ambience Hypothesis such as sound, light, color, movement, WiFi …

13. 13 It is possible to localize phones by sensing the ambience It is possible to localize phones by sensing the ambience Hypothesis such as sound, light, color, movement, WiFi …

14. 14 Multi-dimensional sensing extracts more ambient information Any one dimension may not be unique, but put together, they may provide a unique fingerprint

15. 15 SurroundSense Multi-dimensional fingerprint  Based on ambient sound/light/color/movement/WiFi Starbucks Wall Pizza Hut

16. 16 B A C D E Should Ambiences be Unique Worldwide? F G H J I L M N O P Q Q R K

17. 17 Should Ambiences be Unique Worldwide? B A C D E F G H J I K L M N O P Q Q R GSM provides macro location (strip mall) SurroundSense refines to Starbucks

18. 18 Economics forces nearby businesses to be diverse Not profitable to have 3 adjascent coffee shops with same lighting, music, color, layout, etc. SurroundSense exploits this ambience diversity Why does it work? The Intuition:

19. 19 + + Ambience Fingerprinting Test Fingerprint Sound Acc. Color/Light WiFi Logical Location Matching Fingerprint Database = = Candidate Fingerprints GSM Macro Location SurroundSense Architecture

20. 20 Fingerprints Sound: (via phone microphone) Color: (via phone camera) Amplitude Values -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 Normalized Count 0.14 0.12 0.1 0.08 0.06 0.04 0.02 0 Acoustic fingerprint (amplitude distribution) Color and light fingerprints on HSL space Lightnes s 1 0.5 0 Hue 0 0.5 1 0 0.2 0.4 0.6 0.8 1 Saturation

21. 21 Fingerprinting Sound Fingerprint generation : Signal amplitude  Amplitude values divided in 100 equal intervals  Sound Fingerprint = 100 normalized values value X = # of samples in interval x / total # of samples Filter Metric: Euclidean distance  Discard candidate fingerprint if metric > threshold г Threshold г  Multiple 1 minute recordings at the same location  d i = max dist ( any two recordings )  г = max ( d i of candidate locations )

22. 22 Fingerprinting Color Floor Pictures  Rich diversity across different locations  Uniformity at the same location Fingerprint generation: pictures in HSL space  K-means clustering algorithm  Cluster’s centers + sizes Ranking metric

23. 23 Fingerprints Movement: (via phone accelerometer) CafeteriaClothes Store Grocery Store Static Moving

24. 24 Fingerprints Movement: (via phone accelerometer) CafeteriaClothes Store Grocery Store Static Queuing Seated Moving

25. 25 Fingerprints Movement: (via phone accelerometer) CafeteriaClothes Store Grocery Store Static Pause for product browsing Short walks between product browsing Moving

26. 26 Fingerprints Movement: (via phone accelerometer) CafeteriaClothes Store Grocery Store Static Walk more Quicker stops Moving

27. 27 Fingerprints Movement: (via phone accelerometer) WiFi: (via phone wireless card) CafeteriaClothes Store Grocery Store Static ƒ (overheard WiFi APs) Moving

28. 28 Fingerprinting WiFi Fingerprint generation: fraction of time each unique address was overheard Filter/Ranking Metric  Discard candidate fingerprints which do not have similar MAC frequencies

29. 29 Discussion Time varying ambience  Collect ambience fingerprints over different time windows What if phones are in pockets?  Use sound/WiFi/movement  Opportunistically take pictures Fingerprint Database  War-sensing