1. Robot Compagnion Localization at home and in the office Arnoud Visser, Jürgen Sturm, Frans Groen University of Amsterdam Informatics Institute

2. Overview Mobile robotics Mobile robotics Robot localization Robot localization Presentation of the panorama approach Presentation of the panorama approach Results Results Demonstration videos Demonstration videos

3. Mobile robotics SICO at Kosair Children's Hospital Dometic, Louisville, Kentucky Sony Aibos playing soccer Cinekids, De Balie, Amsterdam Robot cranes and trucks unloading ships Port of Rotterdam RC3000, the robocleaner Kärcher

4. The localization problem Robot localization Robot localization.. is the problem of estimating the robot’s pose relative to a map of the environment. Position tracking Position tracking Global localization Global localization Kidnapping problem Kidnapping problem

5. Localization Sensors Sensors Odometry, GPS, Laserscanner, Camera.. Odometry, GPS, Laserscanner, Camera.. Feature space Feature space World representation World representation Topological graphs, grid-based maps Topological graphs, grid-based maps Filters Filters Kalman filters, particle filters Kalman filters, particle filters

6. Classical approaches Special environments Special environments (Visual) landmarks (Visual) landmarks (Electro-magnetic) guiding lines (Electro-magnetic) guiding lines Special sensors Special sensors GPS GPS Laser-scanners Laser-scanners Omni-directional cameras Omni-directional cameras Special requirements Special requirements Computationally heavy (offline computation) Computationally heavy (offline computation)

7. New approach Natural environments Natural environments Human environments Human environments Unstructured and/or unknown for the robot Unstructured and/or unknown for the robot Normal sensors Normal sensors Camera Camera Reasonable requirements Reasonable requirements Real-time Real-time Moderate hardware requirements Moderate hardware requirements

8. Platform: Sony Aibo Internal camera 30fps 208x160 pixels Computer 64bit RISC processor 567 MHz 64 MB RAM 16 MB memorystick WLAN Actuators Legs: 4 x 3 joints Head: 3 joints

9. Demo: Compass Library, University of Amsterdam

10. Synopsis

11. Color segmentation Sidetrack: Color Calibration Robot collects colors from environment Robot collects colors from environment Colors are clustered using an EM algorithm Colors are clustered using an EM algorithm Color-to-Colorclass lookup table is created for faster access Color-to-Colorclass lookup table is created for faster access Raw image Color class image

12. Mathematics rotation translation feature vector ideal world model learned world model

13. Feature space conversion

14. Feature vectors and world model World model distribution Feature vector consists of color transition counts between the n color classes

15. Feature space conversion (2) Raw image Color class image Sector-based feature vectors

16. Learning Update distribution of single color class transition by updating the constituting counters

17. Matching Likelihood of Single sector Rotation estimate Confidence estimate Adjacent sectors

18. Post-processing: Compass Idea: smooth rotational estimate over multiple frames + removes outliers + stabilizes estimate + integrates (rotational) odometry

19. Results: Compass Brightly illuminated living room

20. Results: Compass Daylight office environment

21. Results: Compass Outdoor soccer field

22. Results: Compass Robocup 4-Legged soccer field

23. Signal degradation (w.r.t. distance) Robocup 4-Legged soccer field

24. Post-processing: Grid localization Idea: learn multiple spots, then use confidence value to estimate the robot‘s position in between – fixed grid (better: self-learned graph based on confidence) – difficult to integrate odometry + proof of concept

25. Demo: Grid localization Robocup 4-Legged soccer field

26. Results: Grid localization Robocup 4-Legged soccer field -100 -75 -50 -25 0 25 50 75 100 -100-75-50-250255075100 x [cm] y [cm] Positioning accuracy Robot walks back to center after kidnap

27. Conclusions Novel approach to localization: Novel approach to localization: Works in unstructured environments Works in unstructured environments Tested on various locations Tested on various locations Interesting approach for mobile robots at home and in the office Interesting approach for mobile robots at home and in the office

28. Questions?