1. Hand-Eye Coordination and Vision-based Interaction http://www.cs.ualberta.ca/~jag/courses/HandEye / cmput610 Martin Jagersand

2. Today: 1. Fun (hopefully) intro to some of the topics in the course. Give flavor of course. 2. Suggestions for some exciting projects you can choose. You are also welcome to propose your own. 3. (Boring) administrative details: Course schedule, room, participation and examination, class list etc…

3. Main focus of course Most other courses are about static vision. What is in the image? This course: Vision for regestering, understanding and generating motion. How to act based on visual information. ImProcessing Symbol Interp Images Agent doing physical motion Real world

4. Where/What or Action/Perception division also in biological vision

5. Main topics we will cover 1. Motion in the real world and its image projection 2. Vision and other sensory based control 3. Biological motor control. 4. User interfaces for computers, robots and other machines.

6. Motion estimation: Image subtraction 50 Candidate areas for motion Detecting motion:

7. Motion estimation 2 A vector flow is a more useful representation

8. Motion estimation 3 Non-rigid motions are also important

9. Motion estimation 4 Generalize to many freedooms (DOFs)

10. One application: Tracking Goal: Stabilizing motion. Find move params

11. Vision and Sensory based motion control

12. Uncalibrated Visual Servoing Let y = visual observation; x = motor control Linear system model: y= Linear p-controller Estimate the Visual-Motor Jacobian

13. Shortcut in Image based visual servoing

14. Biological sensory-motor control Sensory and motor areas in cortex

15. Example: Eye movements Relatively well studied area Yet complex nonlinear kinematics Interesting adaption behaviour Paths: 1 flash Multple targets Remembered location

16. User Interface applications: Gesture and motion tracking

17. User Interfaces 2: Robot Control Vision based “Tele Assistance” Describe task and objects by gestures and pointing Visual language maps to physical actions

18. Composite Task: Solving a Puzzle

19. Proposed projects 1 Neurosci Georgopoulos, Kalaska, Shieber... PercAct Goodale, Lomis, Characterizing motion: Activity recognition: Nelson,Alimonios Fleet, Jepson... Jagersand Matlab implementaton Animating motion. Gleicher, Jagersand, Thalman

20. Proposed projects 2 Tracking: Hager, Black, XVision: practical experiments with. Filtering, Bar-Shalom: Robotics: Controller HW and SW for the small robot arm (or the PUMA's) Real time systems Real time linux

21. Participation and Examination Course readings Class discussion, active class participation Present some papers of your choice Do your own reading or practical project and a writeup

22. Administrative Register for cmput610 Proposed schedule: Tue, Thu 9-10am Room TBA Course literature and readings are given on-line when possible. Announcements via email. Make sure you are on the email list. Course web page (important): http://www.cs.ualberta.ca/~jag/courses/HandEye /