1. Robot Vision SS 2007 Matthias Rüther 1 710.088 ROBOT VISION 2VO 1KU Matthias Rüther

2. Robot Vision SS 2007 Matthias Rüther 2 Administrative Things VO: Tuesday 14:30-16:00 HS i11 Strongly coupled with KU!! www.icg.tu-graz.ac.at/courses Exam: Written Exam Oral Exam if Requested KU: Groups of three students Each group does the same project Effort: ~1week per student

3. Robot Vision SS 2007 Matthias Rüther 3 Time Table

4. Robot Vision SS 2007 Matthias Rüther 4 Literature Sciavicco, L., Siciliano, B., Modelling and Control of Robot Manipulators 2nd Ed., Springer, 2000 Sonka M., Hlavac V., Boyle Image Processing, Analysis and Machine Vision, Chapman Hall, 1998 Hartley R., Zissermann A., Multiple View Geometry in Computer Vision, Cambridge, 2001.

5. Robot Vision SS 2007 Matthias Rüther 5 Student Project  Solve a Computer Vision Problem –From Hardware selection over 3D Measurement to Live Test

6. Robot Vision SS 2007 Matthias Rüther 6 Goal  Measure 3D Geometry of Electrical Discharges Impact Area C1 C2 C3 C4

7. Robot Vision SS 2007 Matthias Rüther 7 Tasks  Workpackage 1: select hardware, acquire images, segment flash (x i, y i )

8. Robot Vision SS 2007 Matthias Rüther 8 Tasks  Workpackage 2: camera calibration and pose estimation Impact Area C1 C2 C3 C4 R W, T W R 21, T 21 R 31, T 31 R 41, T 41 K1K1 K2K2 K3K3 K4K4

9. Robot Vision SS 2007 Matthias Rüther 9 Tasks  Workpackage 3: correspondence & triangulation (x i, y i ) (x j, y j ) 3D

10. Robot Vision SS 2007 Matthias Rüther 10 Organization  The Project is divided in three workpackages which have to be delivered during the term: –30.3.2007 –1.6. 2007 –22.6. 2007  Each group (3 students) does all three workpackages.  The workpackages build on top of the previous ones. After submission, the workpackages are published.  Each group is allowed to use previous workpackages of any other group.

11. Robot Vision SS 2007 Matthias Rüther 11 Example WP1: WP2: WP3: Group 1Group 2 … Group n NO Collaboration during workpackage Group 1Group 2 … Group n Group 1Group 2 … Group n YES Each group may reuse previous workpackages of other groups

12. Robot Vision SS 2007 Matthias Rüther 12 Rules  No collaboration between groups during a workpackage. Copying groups are removed from the KU.  Every group member is held responsible for every task in every workpackage.  Code reuse has no influence on the grade.  Each group must deliver at least two workpackages.  A “Sehr Gut” on the Project gives a 25% Bonus on the Lecture exam on 3.6.2007.

13. Robot Vision SS 2007 Matthias Rüther 13 Robotics  What is a robot? "A reprogrammable, multifunctional manipulator designed to move material, parts, tools, or specialized devices through various programmed motions for the performance of a variety of tasks" Robot Institute of America, 1979 … in a three-dimensional environment.  Industrial –Mostly automatic manipulation of rigid parts with well-known shape in a specially prepared environment.  Medical –Mostly semi-automatic manipulation of deformable objects in a naturally created, space limited environment.  Field Robotics –Autonomous control and navigation of a mobile vehicle in an arbitrary environment.

14. Robot Vision SS 2007 Matthias Rüther 14 Experimental/Industrial/Commercial Robots

15. Robot Vision SS 2007 Matthias Rüther 15 Industrial Robots

16. Robot Vision SS 2007 Matthias Rüther 16 Challenging Environments

17. Robot Vision SS 2007 Matthias Rüther 17 Service and Assistance

18. Robot Vision SS 2007 Matthias Rüther 18 FRIEND Project

19. Robot Vision SS 2007 Matthias Rüther 19 Robot vs Human  Robot Advantages : –Strength –Accuracy –Speed –Does not tire –Does repetitive tasks –Can Measure  Human advantages: –Intelligence –Flexibility –Adaptability –Skill –Can Learn –Can Estimate

20. Robot Vision SS 2007 Matthias Rüther 20 Robotics: Goals and Applications  Goal: combine robot and human abilities.  Applications: –Automation (Production) –Inspection (Quality control) –Remote Sensing (Mapping) –Man-Machine interaction („Cobot“) –Robot Companion (Physically challenged people) –See [Brady, M. et. al. (eds). „Robot Motion: Planning and Control“]

21. Robot Vision SS 2007 Matthias Rüther 21 Statistics Yearly installations of industrial robots, 2003-2004 and forecast for 2005-2008

22. Robot Vision SS 2007 Matthias Rüther 22 Statistics Estimated operational stock of industrial robots 2003-2004 and forecast for 2005-2008

23. Robot Vision SS 2007 Matthias Rüther 23 Statistics Number of robots per 10,000 production workers in the motor vehicle industry 2002 and 2004

24. Robot Vision SS 2007 Matthias Rüther 24 Statistics Service robots for professional use. Stock at the end of 2004 and projected installations in 2005-2008

25. Robot Vision SS 2007 Matthias Rüther 25 Statistics Service robots for personal and domestic use. Stock and value of stock at the end of 2004 and projected installations in 2005 - 2008

26. Robot Vision SS 2007 Matthias Rüther 26 What can Computer Vision do for Robotics?  Accurate Robot-Object Positioning  Keeping Relative Position under Movement  Visualization / Teaching / Telerobotics  Performing measurements  Object Recognition (see LV „Bildverarbeitung u. Mustererkennung“, „Bildverstehen“, „AK Computer Vision“)  Registration Visual Servoing

27. Robot Vision SS 2007 Matthias Rüther 27 Computer Vision  What is Computer Vision? "Computer Vision describes the automatic deduction of the structure and the properties of a (possible dynamic) three- dimensional world from either a single or multiple two-dimensional images of the world" [Nalva VS, "A Guided Tour of Computer Vision"]  Measurement –Measure shape and material properties in a 3D environment. Accuracy is important.  Recognition –Cognitive systems interpret a 3D environment (object classification, categorization). Systems are allowed to fail to a certain extent (similar to humans).  Navigation –Navigation Systems orient themselves in a 3D environment. Robustness and time are important.

28. Robot Vision SS 2007 Matthias Rüther 28 Shape from Stereo

29. Robot Vision SS 2007 Matthias Rüther 29 Shape from Stereo

30. Robot Vision SS 2007 Matthias Rüther 30 Shape from Focus

31. Robot Vision SS 2007 Matthias Rüther 31 Shape from Structured Light  Structured Light Sensor Figures from PRIP, TU Vienna

32. Robot Vision SS 2007 Matthias Rüther 32 Shape from Shading

33. Robot Vision SS 2007 Matthias Rüther 33 Navigation  SLAM: Simultaneous Localization and Mapping. –Where am I on my map? –If the place is unknown, build a new map, try to merge it with the original map.  Visual Odometry: calculate the relative motion of the camera between two frames. Summing up the motion gives the camera path. Error propagation!  Visual Servoing: move to / maintain a relative position between robot end effector and an object.  Tracking: continuously measure the position of an object within the sensor coordinate frame.

34. Robot Vision SS 2007 Matthias Rüther 34 SLAM Mapping:

35. Robot Vision SS 2007 Matthias Rüther 35 SLAM The final map:

36. Robot Vision SS 2007 Matthias Rüther 36 SLAM Navigation:

37. Robot Vision SS 2007 Matthias Rüther 37 Visual Odometry

38. Robot Vision SS 2007 Matthias Rüther 38 Visual Servoing

39. Robot Vision SS 2007 Matthias Rüther 39 Tracking

40. Robot Vision SS 2007 Matthias Rüther 40 Registration  Registration of CAD models to scene features: Figures from P.Wunsch: Registration of CAD-Models to Images by Iterative Inverse Perspective Matching