1. INSTITUTO DE SISTEMAS E ROBÓTICA http://www.isr.ist.utl.pt/labs/vislab Computer and Robot Vision Laboratory OMNIVIEWS DEMOS SURVEILLANCE DEMOS TRANSMISSION DEMO NAVIGATION DEMOS

2. INSTITUTO DE SISTEMAS E ROBÓTICA http://www.isr.ist.utl.pt/labs/vislab Computer and Robot Vision Laboratory Visual Tracking with Omnidirectional Sensors Alexandre Bernardino, José Santos-Victor Instituto Superior Técnico The algorithm: Minimize the SSD. Test hypothesis of main translational motion. Consider other small deformations (e.g. rotation, scale) to cope with deviations from model. Applications: Robot navigation: track landmarks. Robot heading is directly related to target image position. Video Conferencing/Surveillance: keep attention focus on people/faces.

3. INSTITUTO DE SISTEMAS E ROBÓTICA http://www.isr.ist.utl.pt/labs/vislab Computer and Robot Vision Laboratory Visual Tracking with Omnidirectional Sensors Advantages over cartesian sensors: –Target never moves out of horizontal field of view –No moving parts required (pan/tilt) Disadvantages –Shape variation even with simple cartesian transformations (e.g. translation)

4. INSTITUTO DE SISTEMAS E ROBÓTICA http://www.isr.ist.utl.pt/labs/vislab Computer and Robot Vision Laboratory Addressing shape variance problems. –Polar sensor: Horizontal translation invariance –Purposive mirror design Vertical translation invariance Visual Tracking with Omnidirectional Sensors

5. INSTITUTO DE SISTEMAS E ROBÓTICA http://www.isr.ist.utl.pt/labs/vislab Computer and Robot Vision Laboratory The algorithm: –Minimize the SSD (sum of squared differences) of selected target region on incoming images. –Test hypothesis of main translational motion. –Consider some other small deformations (e.g. rotation, scale) to cope with deviations from model. Applications: –Robot navigation: track landmarks. Robot heading is directly related to target image position. –Video Conferencing/Surveillance: keep attention focus on people/faces. Visual Tracking with Omnidirectional Sensors

6. INSTITUTO DE SISTEMAS E ROBÓTICA http://www.isr.ist.utl.pt/labs/vislab Computer and Robot Vision Laboratory Demos: –Robot Navigation –Visual Surveillance Visual Tracking with Omnidirectional Sensors

7. INSTITUTO DE SISTEMAS E ROBÓTICA http://www.isr.ist.utl.pt/labs/vislab Computer and Robot Vision Laboratory Motion detection Stefan Gaechter, Tomas Pajdla, Branislav Micusik Czech Technical University in Prague - Motion detection by background subtraction - OMNIVIEWS 110 x 252 uniform resolution images - Matlab implementation ~ 2-3 fps

8. INSTITUTO DE SISTEMAS E ROBÓTICA http://www.isr.ist.utl.pt/labs/vislab Computer and Robot Vision Laboratory OMNIVIEWS DEMOS SURVEILLANCE DEMOS TRANSMISSION DEMO NAVIGATION DEMOS

9. INSTITUTO DE SISTEMAS E ROBÓTICA http://www.isr.ist.utl.pt/labs/vislab Computer and Robot Vision Laboratory Transmission Demo Laptop IP Service Provider VSR - Server Omniviews Camera (Observit office) Omniviews Camera (Observit office) Telephone Line ADSL Internet Connection Use industry standard codec tools (VSR from Observit) Images are processed as conventional one Connection with telephone line H263 Compression

10. INSTITUTO DE SISTEMAS E ROBÓTICA http://www.isr.ist.utl.pt/labs/vislab Computer and Robot Vision Laboratory OMNIVIEWS DEMOS SURVEILLANCE DEMOS TRANSMISSION DEMO NAVIGATION DEMOS

11. INSTITUTO DE SISTEMAS E ROBÓTICA http://www.isr.ist.utl.pt/labs/vislab Computer and Robot Vision Laboratory Visual-based Robot Navigation & Teleoperation using the Omniviews Camera Niall Winters, José Gaspar & José Santos-Victor Instituto Superior Técnico Topological Navigation Teleoperation Visual Path Following 93 cm

12. INSTITUTO DE SISTEMAS E ROBÓTICA http://www.isr.ist.utl.pt/labs/vislab Computer and Robot Vision Laboratory 1 - Topological Navigation Closest Match Current Image Bird’s eye view

13. INSTITUTO DE SISTEMAS E ROBÓTICA http://www.isr.ist.utl.pt/labs/vislab Computer and Robot Vision Laboratory 2 - Teleoperation method (a): Target direction Target forward direction Forward direction

14. INSTITUTO DE SISTEMAS E ROBÓTICA http://www.isr.ist.utl.pt/labs/vislab Computer and Robot Vision Laboratory Forward direction Target forward direction 2 - Teleoperation method (a): Target direction

15. INSTITUTO DE SISTEMAS E ROBÓTICA http://www.isr.ist.utl.pt/labs/vislab Computer and Robot Vision Laboratory Forward direction Target forward direction 2 - Teleoperation method (a): Target direction

16. INSTITUTO DE SISTEMAS E ROBÓTICA http://www.isr.ist.utl.pt/labs/vislab Computer and Robot Vision Laboratory Bird’s Eye View Panoramic View 2 - Teleoperation method (b): Target position

17. INSTITUTO DE SISTEMAS E ROBÓTICA http://www.isr.ist.utl.pt/labs/vislab Computer and Robot Vision Laboratory 3) Navigation: Visual Path Following