1. Computational Vision Jitendra Malik University of California, Berkeley

2. What is in an image? The input is just an array of brightness values; humans perceive structure in it.

3. From Pixels to Perception Tiger Grass Water Sand outdoor wildlife Tiger tail eye legs head back shadow mouse

4. If visual processing was purely feedforward…(it isn’t) Pixels Local Neighborhoods Contours Surfaces Tiger Grass Water Sand Objects Scenes Low-level Image Processing Mid-level Grouping Figure/Ground Surface Attributes High-level Recognition

5. Boundaries of image regions defined by a number of attributes  Brightness/color  Texture  Motion  Binocular disparity  Familiar configuration

8. Grouping is hierarchical A BC A,C are refinements of B A,C are mutual refinements A,B,C represent the same percept Image BGL-birdR-bird grass bush head eye beak far body head eye beak body Perceptual organization forms a tree: Two segmentations are consistent when they can be explained by the same segmentation tree

9. Humans assign a depth ordering to surfaces across a contour  R1 appears in front of R2  R2 appears in front of R3 This can be done for images of natural scenes …

10. Figure-Ground Labeling - - red is near; blue is far

11. Figure/Ground Organization  A contour belongs to one of the two (but not both) abutting regions. Figure (face) Ground (shapeless) Figure (Goblet) Ground (Shapeless) Important for the perception of shape

12. Some other aspects of perceptual organization Good continuation Amodal completion Modal completion

13. What do we see here?

14. And here?

15. Some Pictorial Cues

16. Support, Size ? ? ? 1 3 2

17. Cast Shadows

18. Shading

19. Measuring Surface Orientation

20. Binocular Stereopsis

21. Optical flow for a pilot

22. Object Category Recognition

23. Shape variation within a category  D’Arcy Thompson: On Growth and Form, 1917  studied transformations between shapes of organisms

24. Attneave’s Cat (1954) Line drawings convey most of the information

25. Objects are in Scenes

26. Human stick figure from single image Input imageStick figureSupport masks

27. This is hard…  Variety of poses  Clothing  Missing parts  Small support for parts  Background clutter

28. Taxonomy and Partonomy  Taxonomy: E.g. Cats are in the order Felidae which in turn is in the class Mammalia  Recognition can be at multiple levels of categorization, or be identification at the level of specific individuals, as in faces.  Partonomy: Objects have parts, they have subparts and so on. The human body contains the head, which in turn contains the eyes.  These notions apply equally well to scenes and to activities.  Psychologists have argued that there is a “ basic-level ” at which categorization is fastest (Eleanor Rosch et al).  In a partonomy each level contributes useful information for recognition.

29. Visual Control of Action  Locomotion  Navigation/Way-finding  Obstacle Avoidance  Manipulation  Grasping  Pick and Place  Tool use

31. Camera Obscura (Reinerus Gemma-Frisius, 1544)

32. Camera Obscura (Angelo Sala, 1576-1637)