1 Panoramic University of Amsterdam Informatics Institute

2 Localization

3 with a Sony Aibo by Jürgen Sturm

4 RoboCup 4-Legged League Sony Aibo Robots 4 vs. 4 robots play fully autonomously Soccer Games Context: Mobile Robots

5 home real-world applications human-machine interaction Fully autonomous robots have to master challenges in unknown & unstructured environments Follow a human, navigate, etc.

6

7 Traditional approaches Aibos / 4-Legged league uses landmarks with known positions, known shape and known color (manually calibration taking hours) General solutions (SLAM) use better hardware Laser range finders Omnidirectional cameras Robots with better odometry (wheels) The problem: Mobile robot localization (estimating the robot’s position)

8 Features of new approach Real-time localization on a Sony Aibo Take advantage of natural features of a room –Independency of artificial landmarks –Auto-calibrating in new environments Idea: –Learn a panoramic model of the surroundings of the robot for localization

9 Color clustering Collect interesting colors (around the robot) Determine 10 most characteristic colors (using an EM clustering algorithm) Raw image (208x160, YCbCr)

10 Sector appearance Divide in vertical slices, called sectors (360° correspond to 80 sectors) Count color transitions per sector (between the 10 most char- acteristic colors of the scene) Approach Building an virtual panoramic wall Raw image (208x160, YCbCr)

11 Learning the panorama model Image features (10-12 sectors/image, 10x10 frequencies/sector) Learn panorama model (estimate frequency distributions per sector) Panorama model (80 sectors, 10x10 distributions, each defined by 5 bins)

12 Alignment and Localization Image features (10-12 sectors/image, 10x10 frequencies/sector) Align with stored panorama model (find shortest path) Output (Rotational estimate Signal-to-noise ratio Confidence range) After learning from 131 frames Robot rotated 45° to the left

13 Experiments in human environments Rotational test in living room (at night) Results Learning of the appearance of unknown & unstructured environments

14 4-Legged soccer field, indoors, single learned spot Translational test on soccer fields Human soccer field, outdoors, single learned spot

15 Multi-spot learning Aibo trained on 4 different spots, yielding 4 different panoramas Aibo kidnapped and placed back at arbitrary positions on the field Aibo tries to walk back to center spot

16 Possibilities for the 4-Legged league Getting rid of all artificial landmarks 11 vs. 11 games (bigger field) Outdoor demonstrations become possible Conclusions

17 Possible usage for the home league Distinguish living room from kitchen or garden Rough but quick map building Find relative position of the TV/stove/etc on this map

18 Other applications CareBot: navigation in a closed indoor environment Mobile applications (for example on cellular phones) for quick positional estimates (tourism)

19 Rotational estimate and Confidence range in numbers

20 Architecture Overview Camera images Odometry updates Sector- based feature extraction and SNR Align with panorama model 0º 90º 180º 270º Orientation buffer + confidence range..

21 Conclusions Accurate estimate of the rotation from a single learned spot (up to 40 meters) A good estimate of the relative distance from a single learned spot (up to 40 meters) Rough estimate of the absolute position from multiple trained spots

22 University of Amsterdam Informatics Institute Panoramic Localization with a Sony Aibo by Jürgen Sturm User manual Head button always resets robot and triggers autoshutter & color clustering Press front button to manually trigger color clustering In training mode: Press middle button to start learning of the first spot Press middle button again to continue learning on more spots Press back button to switch to localization mode In localization mode: Press front button to switch between rotational and translational mode Press middle button to reset panorama and start learning Press back button to switch between find and set-reference mode Fullly working memorystick image can be downloaded from