1. Visual Tracking on an Autonomous Self-contained Humanoid Robot Mauro Rodrigues, Filipe Silva, Vítor Santos University of Aveiro CLAWAR 2008 Eleventh International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines 08 – 10 September 2008, Coimbra, Portugal

2. 11th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines University of Aveiro 08-10 September 2008 2 Outline Overview Objectives Self-Contained Platform Vision System Experimental Results Conclusions

3. 11th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines University of Aveiro 08-10 September 2008 3 Overview Humanoid Platform Humanoid Robot developed at University of Aveiro Ambition is participation at Robocup Platform composed of 22 DOF’s Head on a PTU arrangement Up to 70 cm height and a mass of 6,5 kg

4. 11th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines University of Aveiro 08-10 September 2008 4 Overview Distributed Control Architecture Master/Multi-Slave configuration on CAN Bus Central Processing Unit: Image processing and visual tracking External computer interaction for monitorization, debug or tele- operation Master CPU/Slaves communication interface Slaves Interface with actuators and sensors

5. 11th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines University of Aveiro 08-10 September 2008 5 Objectives Central Processing Unit Integration Computational autonomy Development environment Vision System Development Visual Tracking Approach Detection and tracking of a moving target (ball)

6. 11th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines University of Aveiro 08-10 September 2008 6 Self-Contained Platform CPU standard PCI-104 AMD Geode LX-800 @ 500MHz 512Mb RAM SSD 1Gb Video Signal Capture PCMCIA FireWire board Dual PCMCIA PC104 module UniBrain Fire-i @ 30fps (640x480) Camera Development Environment Linux based OpenCV

7. 11th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines University of Aveiro 08-10 September 2008 7 Vision System

8. 11th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines University of Aveiro 08-10 September 2008 8 Vision System Acquisition Mask Segmentation - H, S and V Components Object Location Pre-processing

9. 11th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines University of Aveiro 08-10 September 2008 9 Vision System Dynamic Region of Interest (ROI) Reduced noise impact Faster calculus With ROI No ROI

10. 11th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines University of Aveiro 08-10 September 2008 10 Vision System Dynamic Region of Interest (ROI)

11. 11th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines University of Aveiro 08-10 September 2008 11 Vision System Visual Tracking Approach Keep target close to image centre Image based algorithm Fixed gains proportional law,, joint increment vector, constant gain matrix, error vector defined by the ball’s offset Variable gains nonlinear law,

12. 11th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines University of Aveiro 08-10 September 2008 12 Experimental Results Self-Contained Platform Acquisition libdc1394 based library Acquisition @ 320x240 with down-sampling: ~24ms Processing Without dynamic ROI: 15ms With dynamic ROI: 11ms Total = ~40ms 25Hz Times (ms) Max. (ms)Min. (ms)Avg. (ms)St. Dev. acquisition32,402011,878013,68202,0275 pyr down25,90509,47309,84321,6330 segmentation41,60309,33209,84562,4185 centroid location3,25500,39701,30790,4478 control0,15900,01400,01540,0093 actuation37,24602,16704,48502,6913 total118,660035,906039,15207,1468

13. 11th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines University of Aveiro 08-10 September 2008 13 Experimental Results Visual Tracking Ball alignment ~1s Stationary error (~7 pixels)

14. 11th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines University of Aveiro 08-10 September 2008 14 Experimental Results Visual Tracking Pan tracking with fixed gains Pan tracking Error increases in frontal area of the robot

15. 11th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines University of Aveiro 08-10 September 2008 15 Experimental Results Visual Tracking Pan tracking with variable gains Frontal area error reduced Fixed GainsVariable Gains

16. 11th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines University of Aveiro 08-10 September 2008 16 Experimental Results Visual Tracking Tilt tracking with variable gains Tilt tracking Error similar to the pan tracking Trunk increases the error

17. 11th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines University of Aveiro 08-10 September 2008 17 Experimental Results Visual Tracking

18. 11th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines University of Aveiro 08-10 September 2008 18 Conclusions Implemented architecture separates the high-level vision processing from the low-level actuators control Dynamic Region of Interest guarantees a greater noise immunity and faster calculus Low error location and alignment with stationary target, fast convergence Tracking error reveals the need of a more sophisticated control Autonomous Self-Contained Humanoid Platform 25Hz average processing rate, sufficient to deal with fast-stimuli and other quick changing visual entries

19. 11th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines University of Aveiro 08-10 September 2008 19 Future Work Validate ball detection through shape detection Recognition of other elements, such as the ones present at the Robocup competition Explore alternative techniques of Visual Servoing Study the influence of the robot’s movement on the visual information and on the tracking system’s performance

20. 11th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines University of Aveiro 08-10 September 2008 20 Thank you for your atention