SurroundSense Mobile Phone Localization via Ambience Fingerprinting Scott Seto CS 495/595 November 1,

Introduction Mobile phones are becoming people-centric Location-based advertising is coming soon There is an absense of well- established logical localization schemes Physical localization does not work well indoors

What is SurroundSense? Uses the overall ambience of a place to create a unique fingerprint for localization Fingerprint location based on ambient sound, light, color, RF, etc. Sensor data is distributed to different modules

Motivation Installing localization equipment in every area is unscalable A scheme with accuracy of 5 meters may not place a person on the correct side of a wall

Challenges Fingerprints from various shops vary over time Colors may be different based on daylight or electric light A sound fingerprint from a busy hour might not match a low-activity period

SurroundSense Architecture

Detecting Sound Ambient sound can be suggestive of the type of place Use sound as a filter Eliminate outliers Compute the pair-wise Euclidean distance between candidate and test fingerprints

Detecting Motion People are stationary for a long period in restaurants and less in grocery stores Place motion fingerprints into buckets Differentiate between sitting and moving places

Detecting Color/Light Extract dominant colors and light intensity from pictures of floors Translate the pixels to the hue- saturation-lightness (HSL) to decouple the actual floor colors from the ambient light intensity

Fingerprinting Wifi Adapt existing WiFi based fingerprinting to suit logical localization Use the MAC addresses of visible APs as an indication of the phone’s location Avoid false negatives

Implementation Groups of students visited 51 stores using a Nokia N95 phone running SurroundSense Collected fingerprints from each store Visited each of them in groups of 2 people (4 people in total). Keep the camera out of pocket

Implementation While in the store, try to behave like a normal customer Went to different stores so that the fingerprints were time-separated Mimiced the movement of another customer also present in that store No atypical behavior: one may interpret the results to be partly optimistic

Future Work Independent research on energy efficient localization and sensing Use the compass to correlate geographic orientation to the layout of furniture and shopping aisles in stores Group logical locations into a broader category

Conclusion SurroundSense fingerprinted a logical location based on ambient sound, light, color, and human movement Created a fingerprint database and performed fingerprint matching for test samples Localization accuracy of over 85% when all sensors were employed for localization

Questions?