SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Martin Azizyan, Ionut Constandache, Romit Roy Choudhury Mobicom 2009

Table of Contents 1. Introduction 2. SurroundSense Architecture 3. System Design a. Sound b. Motion using Accelerometers c. Color/Light using Cameras d. Wi-Fi 4. Evaluation 5. Conclusion

INTRODUCTION ▣ Introduction  Today’s technologies cannot provide logical localization. –Most emerging location based apps need place of user (logical location), not physical location. –Problem  Mobile phones can sense the ambience via sensors.  SurroundSense can realize the indoor logical location via sensors. –Sound, light, color, movement.

SURROUNDSENSE ARCHITECTURE

SYSTEM DESIGN ▣ Fingerprinting Sound  The ambient sound in a place can be suggestive of the type of place.  Amplitude values divided in 100 equal intervals.  Sound Fingerprint = 100 normalized values – = the number of samples in interval x / the total number of samples  Discard candidate fingerprint –If distance > threshold  How do we choose the Threshold( ) ?

SYSTEM DESIGN ▣ Fingerprinting Sound  How do we choose Threshold( ) ? –Multiple 1 minute recordings at the same location – = max distance ( any two recordings ) – = max ( of candidate locations )  Example  is largest distance. So,  Acoustic fingerprints in A Store A Store B Store C

SYSTEM DESIGN ▣ Fingerprinting Motion using Accelerometers  The nature of service in a place influences the type of human movements in that place.  We decided to identify two simple states –Stationary, Motion –Using SVM(support vector machines) Stationary : -1, Moving : +1

SYSTEM DESIGN ▣ Fingerprinting Motion using Accelerometers –0.0 ≤ R ≤ 0.2 sitting (café) –0.2 < R ≤ 2.0 browsing (clothing) –2.0 < R < ∞ walking (grocery)

SYSTEM DESIGN ▣ Fingerprinting Color/Light using Cameras  A large number of stores have a thematic color and lighting as part of their décor (wall, floor).  A hypothesis –automatic pictures taken from different spots in a store are likely to reflect this theme.  Random pictures –Capture a variety of store items. But, it can make the pictures noisy.

SYSTEM DESIGN ▣ Fingerprinting Color/Light using Cameras  The benefits of the floor pictures 1.Avoid privacy concerns When the phone camera is pointing towards the floor, the concerns are partly alleviated. 2.Low noise Dominant colors extracted from floor pictures are expected to be less noisy. 3.Rich diversity in the colors of floor This diversity is beneficial to localization. 4.Uncommon Users may often point their cameras downward while using their phone.

SYSTEM DESIGN ▣ Fingerprinting Color/Light using Cameras  Our goal is to extract dominant colors and light intensity from pictures of floors. –Hue-Saturation-Lightness(HSL) –K-mean algorithm Centroids and Sizes of each cluster. S12 : similarity between test and candidate fingerprint(F 1 : test, F 2 : candidate) δ(i,j) : centroid distance between the i th cluster of F 1 and the j th cluster of F 2 T 1, T 2 : the total number of pixels in F 1 and F 2 SizeOf(C 1i ) : the number of pixels in cluster C 1i

SYSTEM DESIGN ▣ Fingerprinting Wi-Fi  The MAC addresses of visible APs are some indication of the phone’s location.  If Wi-Fi is available –The phone records MAC addresses of APs from received beacons every 5 seconds.  S : similarity f 1, f 2 : the fraction of times M : union of MAC address in f 1 and f 2 m : MAC Address

EVALUATION

CONCLUSION  The main idea is to fingerprint a location based on its ambient sound, light, color, RF, as well as the user movement. –This fingerprint is used to identify the user’s location.  SurroundSense may not qualify as a stand-alone localization technique. –However, in conjunction with GSM based macro-localization, SurroundSense can perform micro-localization based on the inherent properties of the ambience.  Further research in fingerprinting techniques, sophisticated classification, and better energy management schemes could make SurroundSense a viable solution of the future.