1. University of Massachusetts, Amherst Ferret: RFID Localization for Pervasive Multimedia Xiaotao Liu, Mark Corner, Prashant Shenoy

2. Scenario: I’ve Lost my Keys People frequently misplace common items books, keys, tools, clothing, etc. difficult due to the sheer scale: we interact with >1000s of items Need a system to find objects quickly and efficiently then tell the user where the object is

3. Problems Tracking objects can be broken into sub-problems Locate: find position, perhaps not exact, but a general idea Store: keep object locations in a convenient place Update: when objects move, need to change store Display: Present locations to user in a helpful way

4. Solution: Ferret Provides a real-time augmented reality service locates, stores, updates, and displays object locations intended for nomadic objects not mobile ones Leverage passive RFID, multimedia, and location systems passive RFID: inexpensive, scalable, maintenance-free multimedia systems: provide convenient display and storage location systems: bootstrap process of finding locations Goal is to pack all functions into a hand held device including RFID detection, storage, and display a combination of video camera and RFID reader

5. Outline Motivation and Applications Overview of Use Design of Ferret Sensor model Offline location algorithm Online location algorithm Display In paper: Storage, Update for nomadic objects Prototype implementation Experiments Speed and accuracy Robustness to different movement patterns Related Work Conclusions

6. Overview of Operation User selects some object(s) that she is looking for She wanders around a room, or building, holding Ferret system During this process, the reader scans for nearby RFID tags Ferret detects the RFID tag of interest, localizes tag It then displays an outline of where the object is on the screen willing to settle for a probable region of where the object is depend on human skill to find the exact location refine region as system runs present improved results in real-time

7. RFID Localization Passive RFID tags are not self-locating Instead we depend on the handheld to locate tags Passive RFID tags have significant error rates false negatives are frequent false positives due to reflections Locate using probabilistic model inspired by [Hähnel et. al] RFID reader 1. energy 3. id 2. use RF energy to charge up

8. Bayesian Probability Model Goal: p(x|D 1:n ): Probability of tag at x given readings Initially, without readings, p(x|D 0 ) is uniformly distributed Assume we have p(x|D 1:n ) Positive reading p(D n+1 =True|x) Bayes’ rule p(x|D 1:n+1 ) = α p(x|D 1:n ) p(D n+1 |x) α – normalization factor Similarly, for negative readings p(D n+1 =False|x) = 1 - p(D n+1 =True|x)

9. Tag Detection Probability Manually measure probability of detecting tag (positive reading) p(D =True|x) x – tag’s position

10. Ferret Localization Algorithm (+ reading) Multiple readings come from user mobility, previous, or shared readings

11. Ferret Localization Algorithm (- reading) Repeated intersection of positive and negative readings

12. Offline Algorithm Complexity We refer to the previous algorithm as the “offline” algorithm Each + or - reading Ferret performs O(n^3) operations n is the number of sample points it must rotate, translate the RFID sensor model multiply each sample point against every other sample point must do this for each object! Computational requirements at least 0.7s on a laptop reader is producing at least 4 readings per second some readings include multiple objects Algorithm most useful for back-annotating video

13. Online Algorithm To address real-time concerns use an “online” algorithm instead of intersecting all interior points, just find convex intersection only uses positive readings, not negative ones (keeps shape convex!) Complexity reduced to O(n^2) or 6ms per reading

14. Display Each RFID location is a 3-D shape To display we simply project this 3-D shape onto a 2-D screen

15. Ferret Prototype ThingMagic Mercury4 RFID reader 30dBm (1 Watt), monostatic circular antenna Alien Technology “M” RFID Tag EPC Class 1, 915 MHz Sony Motion Eye web-camera 320x240 at 12fps Cricket Ultrasound 3-D locationing system global location not necessary, but need relative locations at least Sparton SP3003 Digital Compass Pan, tilt, and roll Software translate between coordinate systems, rotate, and display

16. Ferret Prototype Cricket locationing sensor Compass RFID antenna ThingMagic RFID reader Built-in Camera

17. Evaluation Evaluation metrics: Size of location region for many objects Speed of localization for a particular object Robustness of localization to mobility patterns Evaluation setup for many objects: Place 30+ objects with passive tags around the room Move Ferret system around the room by human for 20 minutes CDF of localization over 30 objects Evaluation setup for single object: Place single object in room with passive tag Move Ferret system in and out of view randomly and using a specific pattern Size of localization after some amount of time

18. Online Vs Offline (CDF-30 Objects) Offline algorithm outperforms online, but most objects localized to 0.2 m^3

19. Refinement: Relative Volume (1 Object) Volume size drops down 100 times to 0.02m3 in 2 mins When starting with previous readings, localization is faster

20. Refinement: Relative Projection Area Final projection area decreases 33 times in 2 mins to a 54 pixel diameter circle

21. Different Movement Patterns StraightHead-onz-LineRotateCircle online Volume (m^3) 0.0200.00420.0230.0260.032 offline Volume (m^3) 0.00150.00300.00170.00110.026 Offline/ Online 13.331.4013.5223.631.23 Circular motion pattern performs the worst: no diversity in views Offline algorithm’s advantage comes from negative readings so head-on and circular perform similarly

22. Related Work Grown out of our work on Sensor Enhanced Video Annotation SEVA ACM Multimedia 2005 (Best Paper Award) Used active sensors for location RFID Localization inspired by techniques from [Hähnel et. al] 2-D sensor model, application of Bayes rule positive readings we add 3-D model, negative readings, and online technique focuses on SLAM/localizing reader, we focus on reverse LANDMARC and SpotON RFID locationing active RFID and signal strength

23. Conclusions Ferret: a scalable, RFID-based, augmented reality system localize objects augmented with passive RFID tags display probable location regions to a user in real-time Uses two algorithms: online and offline both are accurate and efficient (localizes objects to 0.2m^3 in minutes) robust to a variety of user mobility patterns Ferret lays the ground work for other augmented reality applications

24. University of Massachusetts, Amherst Ferret: RFID Localization for Pervasive Multimedia Xiaotao Liu, Mark Corner, Prashant Shenoy

26. Location Storage Locations (3-Dimensional probability maps) Storage on reader simple to implement, but must acquire readings as it goes Database any Ferret readers can take advantage of prior knowledge also permits offline searching, but privacy/authorization concerns Storage on writable tags tags self-locating and provide locations to non-Ferret systems

27. What if objects move? Nomadic objects may have moved since previous readings when online algorithm detects empty intersection, reset offline algorithm more complex, uses a probability threshold

28. Bayesian Locationing Module Device Drivers for Cricket and Compass RFID Module (operate RFID reader) Ferret Software Architecture Ferret System Video Recording Visualization Module (modified from FFmpeg) via TCP, Use SQL- like language Deal with large amount of data, Optimized for real-time usage Use optics model Intercept original display function Fuse video, tag’s location together Compute projection of location estimates Display projection boundary

29. [Hähnel et. al] “To each of the randomly chosen potential positions we assign a numerical value storing the posterior probability p(x | z1:t) that this position corresponds to the true pose of the tag. Whenever the robot detects a tag, the posterior is updated according to Equation (1) and using the sensor model described in the previous section.” In this paper we analyze whether recent Radio Frequency Identification (RFID) technology can be used to improve the localization of mobile robots and persons in their environment.