1 Mid level vision, neglected yet still important Ken Nakayama Harvard University

2 21st C challenge Existence and variation of occlusion and variable sources of illumination pose unsolved problems for vision

3 Object representation needs an intermediate level format Low level vision alone is not even explanatory for wide range of visual processes (motion, stereo, search) Missing -- a satisfactory scientific description of surface level vision

4 1950s 1970s Visual take-over of the whole brain 1980s visual Half of primate brain and substantial fraction of human brain devoted to vision

5 Macaque monkey brain flattened Visual regions shown in color

6 Global division of the visual system dorsal ventral (where, how) what

7 action object recognition BYPASS?

8 action object recognition motion search depth attention motion search depth attention

9 Kanizsa Phenomenology, reviving the Gestalt approach Level: surfaces Method: phenomenology Practitioner: Gaetano Kanizsa new concepts: amodal and modal completion

10 Amodal competion (behind) modal competion (in front)

11 Inferences, but at what level ?

12

13 Amodal completion trumps knowledge of horses Suggests that there is a completion process within the visual system

14 Amodal completion allows fragments to be grouped and thus recognized (strongest evidence) letter B spot the 5 letter Bs From Bregman, 1990 same fragments

15 Occlusion and the problem of segmentation for object recognition Border ownership issues - for 3-D scenes, borders cannot be shared. Border dispute needs resolution Rule - border belongs to the closest surface What belongs together ?

16 Problem of segmentation Kanisza’s figure Normal or amputee ?

17 Border ownership dictated by “lines” prevents modal and amodal completion

18 New sources of evidence Surface in front “owns” the border. Thus face on right is broken up, on left is OK Stereoscopic disparity Nakayama et al.Perception ‘89 - faces easier to recognize on left

19 Stereoscopic depth also determines border ownership between regions. Nearer surface will own the border (for opaque surfaces) Nakayama & Shimojo stereo demonstrations

20 Image level can’t even explain much lower level vision Deployment of attention, motion perception, texture, visual search

21 image surfaces where what dorsal parietal ventral temporal how

22 Surfaces needed for much lower visual function

23 He and Nakayama search task Nature (1992) Used stereo vision

24

25 Random dot stereogram unpaired points The correspondence problem: an image based problem

26 L.E. onlyR.E. only

27 invisible to right eye What would happen if we presented unpaired points by themselves? What gives rise to unpaired points? occluding surfaces

28 DaVinci stereopsis (Nakayama &Shimojo)

29  Scene depth from unpaired gaps Gillam and Nakayama, 1999

30 Forest vs plane A plane is a surface Which can occlude, a set of random sticks cannot

31 Planes vs sticks Gillam and Nakayama, 200

32 Level of processing high or low level inference? Hypothesis - inferences learned via associative cortical learning

33 generic view principle when faced with more than one surface interpretation of an image, the visual system assumes it is viewing the scene from a generic, not accidental, vantage point. Nakayama and Shimojo

34 LERE folded wings? folded cards? Why don’t we interpolate depth and see folded wings and cards? Some counterintuitive observations

35

36 Accidental vs generic vantage points

37 accidental view generic view

38 cube square (surface) (volume) surfaces images viewing sphere

39 generic view principle when faced with more than one surface interpretation of an image, the visual system assumes it is viewing the scene from a generic, not accidental, vantage point.

40

41 LERE folded wings? folded cards? Why don’t we interpolate depth and see folded wings and cards? Some counterintuitive observations

42 LERE conclusion: this is a generic view of crossed bars not wings

43 this is the generic view of transparent surface in front, not a folded card

44 neural mechanisms of surface representation ? Cells in V2 respond to subjective contours Strategy: vary stimuli in ways that lead to Appearance and disappearance of subjective contours

45 Recordings from a single cell in area V2 of monkey Physiological correlates of illusory contours in single unit recordings

46 Bakin, Nakayama, and Gilbert, 2000

47 Edgar Rubin figure and ground Edge labeling? contrast polarity vs edge labeling Cells coding Border ownership? Von der Heydt Et al.

image based response

49 Border ownership based response

50 Border ownership cells Von der Heydt and colleagues

51 Von der Heydt (1984) Bakin, Nakayama, Gilbert (2000) DaVinci stereopsis Border ownership cells (V2) yes Mechanistic account of surface representation? --> V2

52 21st C challenge Existence and variation of occlusion and variable sources of illumination pose unsolved problems for vision

53 Object representation needs an intermediate level format Low level vision alone is not even explanatory for wide range of visual processes (motion, stereo, search) Missing -- a satisfactory scientific description of surface level vision -- demos the importance for illumination for object recognition

54 importance of shadow processing Ted Adelson

55 outline no shadow face yes reduce contrast yes Shadow processing in object recognition

56 reduce contrast yes add outline no Outline is very destructive to seeing regions as shaded. Line is interpreted as a bounding Contour of an object

57