Math. Meth. in Vision and Imaging cmput 613/498 Lecture 1: Introduction and course overview Martin Jagersand

Administrivia Classes: Tue, Thu 8-9:20Classes: Tue, Thu 8-9:20 Instructor: Martin Jagersand Martin Jagersand Labs: csc 1-05Labs: csc 1-05 Prerequisite: Compulsory linear algebra and calculusPrerequisite: Compulsory linear algebra and calculus Textbook: Multiple View Geometry, Hartley and Zisserman. Available: Bookstore or webTextbook: Multiple View Geometry, Hartley and Zisserman. Available: Bookstore or web Webpage:Webpage: –

Administrivia Grading Homework: 32% (4 written and lab assignments, 8% each) Exams: In class, Feb 26, Mar 27, 15% each Reading and presentation: In class, 8% Project. Due end of classes 30% Lab: Room csc1-05, SW: Matlab + Mexvision under Linux

What is Computer Vision? How to infer salient properties of 3-D world from 2-D images or video ¤ What is salient? ¤ How to deal with loss of information going from 3-D to 2-D projection?

Biological vision For humans and animals vision is powerful yet effortless.For humans and animals vision is powerful yet effortless. A large part of the brain is devoted to visual processing (40%)A large part of the brain is devoted to visual processing (40%)

Computer Vision For machines: There are methods for several subdomains e.g.There are methods for several subdomains e.g. 1.Object, person identification 2.3D modeling from points 3.Visual motion control But general, human-like vision unsolved/hardBut general, human-like vision unsolved/hard

How much “3D” do we really see? Will the scissors cut the paper in the middle?Will the scissors cut the paper in the middle?

ambiguity Will the scissors cut the paper in the middle? NO!Will the scissors cut the paper in the middle? NO! ambiguity

Size illusions

Tasks in Vision Perception: What: Label what is in a scene: What people, animals, objects…What: Label what is in a scene: What people, animals, objects…Action: Where: Determine the coordinates of where something is. (Mishkin, Ungleider)Where: Determine the coordinates of where something is. (Mishkin, Ungleider) How: Use vision to perform some physical manipulation task. (Goodale etc.)How: Use vision to perform some physical manipulation task. (Goodale etc.)

Compare in Human vision: Dorsal and Ventral Pathways

The “Vision Problem” InputOutput Vision Algorithm

The “Vision Problem” InputOutput Vision Algorithm

Allies and Inspiration Math/geometry Image Processing Cognitive Science Physics, Optics InputOutput Vision Algorithm Engineering Biology, Psychology Computer Science Graphics, AI, NNets,...

What is an image? What is an object? Recognizing objects

What: Recognizing people and objects How do we determine that these are the same objects? Two approaches: 1.Shape 2.Texture

Model based recognition: Find and match the shape Example: Find the outlines of objects. Try to generate 3D or some pseudo 3D geometric description Check database for a similar geometric object

Model based recognition: Techniques. Stereo: From two (or more) images, determine the geometry of the scene by matching corresponding areas of the imagesFrom two (or more) images, determine the geometry of the scene by matching corresponding areas of the images RIGHT IMAGE PLANE LEFT IMAGE PLANE RIGHT FOCAL POINT LEFT FOCAL POINT BASELINE d FOCAL LENGTH f

Apperance based recognition Match the 2D visual “texture” Methods: 1.Spatial: Determine “likeness” by convolution. 2.“Frequency”: Transform and measure in fourier or KL space.

What is the appearance under all different light directions? Space of images at light varies Space of images at light varies Sample images with different light directions [Debevec et al]

RIGHT IMAGE PLANE LEFT IMAGE PLANE RIGHT FOCAL POINT LEFT FOCAL POINT BASELINE d FOCAL LENGTH f Where Use stereo to determine 3D location with respect to camera.Use stereo to determine 3D location with respect to camera. If we know where the camera is we can determine 3D world positionIf we know where the camera is we can determine 3D world position

How (to act in the world)?

Motion estimation and tracking 50 Candidate areas for motion Detecting motion:

Moving the camera Like stereo!Like stereo! Locations of points on the object (the “structure”) The change in spatial location between the two cameras (the “motion”)

Tracking Goal: Stabilizing motion.Goal: Stabilizing motion. Method:Find same region in consecutive imagesMethod:Find same region in consecutive images

Applications of Computer Vision: Medical Imaging

Applications of Computer Vision: Image Databases From a search for horse pix in 100 horse images and 1086 non-horse images (Courtesy D. Forsyth & J. Ponce)

Applications of Computer Vision: Data Acquisition

3D Computer vision uses Use increases every year. Current and emerging applications are e.g.: 1.Scientific, engineering, industrial 2.Movie special effects 3.Computer games 4.Tele-presence 5.Capture and visualization of cultural and natural heritage. etc… =+

Space Vision and Robotics Canada’s contribution Neptec and Optec vision systemsNeptec and Optec vision systems Canada-arm 1 and 2Canada-arm 1 and 2 Canadian Space Agency vision and robotics operation supportCanadian Space Agency vision and robotics operation support

Sony’s Eye Toy: Computer Vision for the masses Background segmentation/ motion detection Color segmentation …

Vision-based modeling and rendering 1.Collect input views 2.Compute 3D geometry and appearance 3.Integrate models 4.Render new views See: poses structure Structure from motion algorithm

Case Study: Virtual heritage Modeling Inuit Artifacts Results: 1.A collection of 3D models of each component 2.Assembly of the individual models into animations and Internet web study material. animationsInternet web study material animationsInternet web study material

Questions? More information on 3D vision: Downloadable renderer+models+infoDownloadable renderer+models+infowww.cs.ualberta.ca/~vis/ibmr Papers: Capturing software + IEEE VR03 tutorial textCapturing software + IEEE VR03 tutorial textwww.cs.ualberta.ca/~vis/VR2003tut More on applications and uses next class