1. 1 Visual Perception in Humans and Machines Kostas Daniilidis Assistant Professor GRASP Lab University of Pennsylvania

2. 2 Examples How do we (humans) recognize faces ? Make a machine find President Clintons face in the web

3. 3

4. 4 An interdisciplinary definition Computer Vision is devoted to the discovery of algorithms, representations, and architectures that embody the principles of visual capabilities. What are visual capabilities? –Recognizing objects and faces –Estimating shapes and distances –Moving, grasping

5. 5 Relation to other fields Computer Vision is inspired from Biological Vision (Phenomenology and Models in Psychophysics and in Neurobiology) but does not try to imitate the nature's architecture or algorithms. Biological Vision and Psychophysics may find computational models discovered in Computer Vision useful for explaining nature.

6. 6 Target problem in computer vision Compute properties of the 3D world from one or more digital images These properties may be –dynamic (observer and object motion) –geometric (distances, object shape) –enabling recognition The result may be an action (grasp an object, avoid an obstacle)

7. 7 What is an image ? A gray-value image is just a set of numbers (usually from 0 to 255)

8. 8 An image is a set of numbers 175 189 190 188 199 197 196 193 181 189 191 194 198 196 191 179 189 191 197 198 200 195 173 129 192 194 198 200 194 161 116 116 198 200 200 190 152 113 116 119 201 202 185 135 105 103 114 119 205 180 121 89 104 101 109 114 177 105 88 90 100 103 101 105

9. 9 An image is a surface I(x,y)

10. 10 Basic image processing operations Smoothing and Noise Removal

11. 11 Blur removal

12. 12 Edge detection X derivativeX derivative Y derivativeY derivative Gradient magnitudeGradient magnitude After thresholdingAfter thresholding

13. 13 (Sub) sampling Shannon Theorem: Sampling frequency must be greater than the maximal frequency in the image (therefore smooth before subsample)

14. 14 Brightness perception

15. 15 How do we perceive distances? Perspective distortion in texture, contour, shading, and a-priori knowledge Stereopsis (what most people believe) Motion

16. 16 The Eye as a Pinhole Camera: Perspective Projection u = X/Z Z X

17. 17 Ames Illusion

18. 18 Perspective Illusions A-priori-knowledge bias

19. 19 Quiz From which points in space is a rectangle viewed as a square (more difficult: an ellipse viewed as a circle ? Be careful: The center ofBe careful: The center of the ellipse in the image is not the projection of the center of the ellipse on the floor!

20. 20 The power of vanishing points Perspective projection preserves cross-ratio = AC/AD : BC/BD is the same on the street and in the image. If A is a vanishing point AC/AD = 1. A B C D We measure A,B,C,D in pixels in the image and form cross ratio for image and for the street. BC is computed from the equality of the two ratios.

21. 21 Stereopsis Infer depth from the disparity between the positions of the same feature in left and right image

22. 22 Stereo Reconstruction

23. 23 Stereo-disparity estimation Search at every pixel for candidates of maximum correlation between left and right Estimate 3D-coordinates of point

24. 24 The power of visual motion Most of the animals have monocular vision (left and right visual fields do not overlap) 8% of the population can not see stereo Stereopsis is limited to a very short depth of field (10m).

25. 25

26. 26 Kinetic depth effect (moving dots)

27. 27 Self- and object-motion

28. 28 Motion artifacts

29. 29 Structure from Motion Given a sequence of images find 1. Ego-motion 2. 3D-structure 3. Independent motions applying only the assumption of rigidity.

30. 30 Motion Field and Heading Direction

31. 31 Depth map

32. 32 Temporal aliasing Wagon Wheel Illusion: A wheel with a periodic radial pattern is perceived to move backwards depending on the relation between the speed, the radius of the wheel, and the period of the pattern (www.cstr.ed.ac.uk/~rjc/wagonWheel)

33. 33 Aperture problem Inside a small aperture displaying a small line we can estimate only the motion direction perpendicular to the line.

34. 34 My wife and my mother-in-law The role of the focus of attention