1. SurroundSense: Mobile Phone Localization via Ambience Fingerprinting MARTIN AZIZYAN, IONUT CONSTANDACHE, ROMIT ROY CHOUDHURY Presented by Lingfei Wu

2. Outline  Introduction  SurroundSense Architecture  System Design  Implementation  Evaluation  Conclusion

3. Introduction  Notion of location  Physical coordinates(latitude/longitude)  Logical labels(like Starbucks, Mcdonalds)  Many applications based on logical location Application of logical localization

4. Introduction  Physical coordinate can be reversed to logical location.  However, it often causes error !  Why not compute logical location directly?

5. Relative work  Active RF  Install special hardware  Ultrasound, Bluetooth  Passive RF  GPS, GSM or WIFI based  Behavior Sensing  Imaging matching 1. Lack accuracy 2. Need pre-installed infrastructure

6. Motivation  Combine effect of ambient sound, light, color, user motion  Sound (microphone)  Starbucks VS Bookstore  Light / Color (camera)  Different thematic light, colors and floors.  Human movement (accelerometer)  Wal-Mart VS McDonald  Place may not be unique based on any one attribute  The combination can be unique enough for localization  In this paper, we propose SurroundSense for logical localization. Starbucks McDonald’s Bookstore Wal-Mart

7. SurroundSense Architecture 1.Xxx 2.Yyy 3.zzz Candidate list 1.Xxx 2.Yyy 1.Xxx 2.Yyy 1.Xxx 2.Yyy 1.Xxx

8. System Design  Fingerprint generation  Fingerprinting sound  Fingerprinting motion using accelerometers  Fingerprinting color/light using cameras  Fingerprinting Wi-Fi  Fingerprint matching  Wi-Fi filter  Sound filter  Motion filter  Color/light Match

9. Fingerprinting sound  Convert signals to time domain  100 normalized values as feature of sound  Similarity of fingerprints  Compute 100 pair-wise distance between test fingerprint and all candidate fingerprint 50 0 -50 Normalized amplitude value Normalized occurrence count time amplitude value time Dim123……100 A0.10.20.1……0.05 B0.60.30.2……0.1

10. Fingerprinting Sound  Unreliable to be a matching scheme  Sound from the same place can vary over time.  Only use as a filter  If distance > threshold τ then discard from the candidate list

11. Fingerprinting Motion  Use support vector machine(SVM) as classifier  Sequence of states as user’s moving pattern  Movement is prone to fluctuation  In a clothing store, Some users browse for a long time while others purchase clothes in haste.  Only use as a filter SVM Raw data moving stationary 1

12. Fingerprinting Motion  Compute motion fingerprint: Ratio = t moving / t static  Bucket 1: 0.0 <= Ratio <= 0.2 Sitting (cafe)  Bucket 2: 0.2 <= Ratio <= 2.0 Browsing (clothing)  Bucket 3: 2.0 <= Ratio <= ∞ Walking (grocery) SittingBrowsingWalking

13. Fingerprinting Color / Light  Thematic color and lighting in different stores  Where to capture the picture?  random picture of surrounding  floor  Advantages of taking floor pictures  Privacy concern  Less noisy  Rich diversity in floor color  Easy to obtain too much noise

14. Fingerprinting Color / Light  How to extract colors and light intensity?  RGB  HSL(Hue-Saturation-Lightness)  Find color cluster and its size using K-means clustering algorithm k=2 s k -s k-1 < t k-mean clustering k++ no yes s k : the sum of distance from all pixels to their (own cluster’s) centroid. t: convergence threshold Bean Trader’s Coffee shop too much noise

15. Fingerprinting Color / Light  Similarity of fingerprints  Assume C 1 = {c 11, c 12, …, c 1n }; C 2 = {c 21, c 22, …, c 2m }  Fingerprint matching  The candidate list with maximum similarity is declared to the matching fingerprint Total size in C 1 or C 2 distance of centroid

16. Fingerprinting Wi-Fi  Wi-Fi fingerprint  Record MAC address from APs every 5 second Fingerprint tuple: <{AP1_MAC_Addr, AP1_fraction_time}, {AP2_MAC_Addr, AP2_fraction_time}, {AP3_MAC_Addr, AP3_fraction_time}>

17. Fingerprinting Wi-Fi  Similarity of fingerprints  Use as filter/matching module  In the absence of light/color, we use it as matching module.  Accuracy depend on location of shops. M: union of MAC address of fingerprints f1 and f2 fraction of time

18. Implementation  Client and server  Client: Nokia 95 phones using Python as client  Server: Matlab and Python code and some data mining tools for fingerprinting algorithms.  Fingerprint database  Labor-intensive war-sensing at 51 stores  Store location: 46 business location in university town, 5 location in India

19. Implementation

20. Evaluation  SurroundSense(SS) test environment  War-sensed 51 shops organized in 10 clusters  4 students visited the first nine clusters in university town, while 2 students visited the tenth cluster in India.  4 localization models:  Wi-Fi only (Wi-Fi)  Sound, Accelerometer, Light and color ( Snd-Acc-Lt-Clr)  Sound, Accelerometer, Wi-Fi (Snd-Acc-Wi-Fi)  SurroundSense (SS)

21. Evaluation – Per-Cluster Accuracy Best represented Restaurant Similar hardwood floor in strip mall Same AP False negative Snd and Acc No Wi-Fi

22. Evaluation – Per-Shop Accuracy  To understand the localization accuracy on a per-shop basis 47% shops 30% shops SS: 92% Snd-Acc-WiFi: 92% Snd-Acc-Lt-Clr: 75% WiFi: 75%

23. Evaluation – Per-User Accuracy  Simulate 100 virtual user, each assign 4~8 stores from cluster 1~9

24. Evaluation – Per-Sensor Accuracy  Hand-picked 6 samples to exhibit the merits and demerits of each sensor false negative Percentage localized using special sensors Number of shops left after special filter

25. Conclusion  Presented SurroundSense, a non-conventional approach for logical localization.  Created fingerprints about ambient sound, light, color, movement and Wi-Fi and match them with fingerprint database to realize accurate logical localization.  The evaluation achieved an average location accuracy of over 85% using all sensors.

26. Discussion  The GPS 10 m, Wi-Fi and GSM 40m and 400m respectively. Why not use Wi-Fi to get initial location instead of using GSM?  Support vector machines (SVM), K-means clustering algorithm are used in paper, do you have any better machine learning methods? Such as Kalman filter, Particle filter, and Wavelet Transform?  Can other sensors help? Such as compass and Bluetooth?  Energy consideration? Non-business location?