1. Ch 51 Sensation & Perception Ch. 5: Perceiving Objects and Scenes © Takashi Yamauchi (Dept. of Psychology, Texas A&M University) Main topics The challenge of object perception Inverse projection problem Gestalt psychology Perceptual organization Figure-ground segregation Biederman’s Recognition by Components theory

2. Ch 52 Minsky’s estimation Why is it so difficult for the computer to recognize objects as we do? http://web.media.mit.edu/~minsky/

3. Ch 53 What AI can do 50 th anniversary in 2006 (from Rodney Brooks) –The most advanced AI can barely beat a 2- year-old child in object recognition.

4. Ch 54 Why AI betrayed? Recognizing objects –is much more than just detecting lines, colors, or shapes. –requires a large amount of background knowledge.  both implicit and explicit knowledge

5. Ch 55 The challenge of object perception The stimulus on the receptors is ambiguous –Inverse projection problem: an image on the retina can be caused by an infinite number of objects Objects can be hidden or blurred –Occlusions are common in the environment

6. Ch 56 Figure 5.3 The principle behind the inverse projection problem. The small square stimulus creates a square image on the retina. However, this image could also have been created by the other two shapes and many other stimuli. This is why we say that the image on the retina is ambiguous.

7. Ch 57 Fig. 5-4, p. 96

8. Ch 58 The Challenge of Object Perception - continued Objects look different from different viewpoints –Viewpoint invariance: the ability to recognize an object regardless of the viewpoint The reasons for changes in lightness and darkness in the environment can be unclear

9. Ch 59 Fig. 5-6a, p. 96

10. Ch 510 Fig. 5-8, p. 97

11. Ch 511 Part + Part = Whole? Gestalt psychology

12. Ch 512 Group discussion Given the pictures shown in the slides, write down everything you see

13. Ch 513 Fig. 1.

14. Ch 514 Fig. 2

15. Ch 515 Fig. 3

16. Ch 516 Fig 4

17. Ch 517 Fig. 5

18. Ch 518 Fig. 7a Fig. 7b Fig. 7c

19. Ch 519 Three Musicians. P. Picasso

20. Ch 520 Woman with a book: P. Picasso

21. Ch 521 La lectrice: P. Picasso

22. Ch 522 Music. H. Matisse

23. Ch 523 Dance. H. Matisse

24. Ch 524 View of Toledo: El Greco

25. Ch 525 Baptism of Christ: El Greco

26. Ch 526 The Burial of Count Orgaz: El Greco

27. Ch 527 Metamorphosis of narcissus: S. Dali

28. Ch 528 War: S. Dali

29. Ch 529 Mont Sainte-Victoire (Cezanne)

30. Ch 530 Hokusai Hiroshige

31. Ch 531 What do these pictures tell us? Perceiving things is a lot more than just detecting lines, colors, motion, so on. We see –Face, body, hands, apples, mountains, legs, tables, houses, streets,…… –motion, movement, action, rhythm, expansion, contraction, upheaval,

32. Ch 532 How come? Your eyes receive 2-D flat information. All you got are activation of neurons. How come you perceive faces, trees, apples…?

33. Ch 533 Why & how do we see those? Perception involves –Organization We see a woman sitting on a sofa because we organize visual information in a certain way.

34. Ch 534

35. Ch 535 How do we organize visual information? Are there any principles behind it? What principles do we follow? Gestalt laws of organization

36. Ch 536 Tell me what you see.

37. Ch 537 Tell me what you see.

38. Ch 538 Perception is not just detection Perception is not just about detecting color or shape. Perception is about organizing visual information. How do we organize visual information?

39. Ch 539 Or when do we fail to organize visual information?

40. Ch 540 Gestalt psychology “Gestalt” means “whole.” Organizational principles: –Similarity –Proximity –Continuity And more (see the textbook)

41. Ch 541 Law of similarity Similar things are put together

42. Ch 542

43. Ch 543 Items with similar colors are put together

44. Ch 544 Do you see 3 musicians? Why? Why not? Do you see a table? Do you see a sofa? Do you see a blouse and a blue skirt?

45. Ch 545

46. Ch 546 Law of proximity Things that are close to each other are put together.

47. Ch 547 There are two separate worlds --- this world and that world?

48. Ch 548

49. Ch 549 Law of good continuity We tend to put things together when they show nice continuity.

50. Ch 550 Do you see a spinal motion? This picture is called “Dance.” Do you see why?

51. Ch 551 Do you feel an upward motion or a downward motion?

52. Ch 552

53. Ch 553 There are many many more ways to organize visual information (see the textbook).

54. Ch 554 Perceptual Segregation Figure-ground segregation – determining what part of environment is the figure so that it “stands out” from the background

55. Ch 555 Figure-Ground segregation Distinguishing a figure from the ground. We know the difference between the figure and the ground

56. Ch 556 How does the perceptual system distinguish the figure from the ground?

57. Ch 557 Which one is the figure and which one is the ground? Depending on the locus of attention, the figure and the ground switch rapidly.  Attention plays some role in determining the figure and the ground.

58. Ch 558 The figure represents “some thing.” The contours belong to the figure rather than to the ground.

59. Ch 559 Which one if the figure and which is the ground? This is easy. The figure tends to have solid and continuous surface.

60. Ch 560 Which one is the figure and which one is the ground? Symmetric items tend to be seen as a figure.

61. Ch 561 What did you see? Some arrows and what? Do you see something else? Meaningful items are seen as figure

62. Ch 562

63. Ch 563

64. Ch 564 What does this tell? Figure-ground segregation is influenced by our knowledge about the world in general. –What things are, how they look like, –Two things can’t occupy the same space simultaneously.

65. Ch 565 What does this tell? Gestalt laws of organization –Similarity, continuity, proximity,.. Figure-ground segregation Perceptual organization is based on our world knowledge (what we know about the world).

66. Ch 566

67. Ch 567

68. Ch 568

69. Ch 569

70. Ch 570

71. Ch 571

72. Ch 572

73. Ch 573

74. Ch 574 Input & output to and from LGN

75. Ch 575 Any idea? Gestalt laws of organization –Similarity, continuity, proximity,.. Figure-ground segregation What do they tell you? Some top-down processes are going on.

76. Ch 576

77. Ch 577 Modern research on object perception The perceptual system is tuned to capture the regularities in the environment. Biederman’s Recognition by Components model

78. Ch 578 How do we store objects in the brain? See an object. Put it into your brain Take out the one you stored earlier. compare it to what you saw. How do you store what you saw before?

79. Ch 579 You see billions of objects every day Does your brain have enough space to store them? Storing everything is ineffective (very expensive). So, you got to do something different. Better way to store things?

80. Ch 580 How do you do that?

81. Ch 581 Biederman’s Recognition by Components You just need about 36 components to represent millions of different objects.

82. Ch 582 Biederman’s Recognition by Components (RBC) Objects are described and stored by simple geometric components (geons). There are about 36 geons. To represent objects, we use geons and their arrangements.

83. Ch 583 Geons

84. Ch 584

85. Ch 585 Combinations of geons

86. Ch 586 Combinations of geons Combining 4 geons can yield more than 1 million objects. (36x36x36x36)

87. Ch 587

88. Ch 588

89. Ch 589

90. Ch 590

91. Ch 591 Biederman & Ju (1988) Cognitive Psychology, 20, 38-64 Do people need information more than geons provide? Contrast subjects’ performance for object recognition when two types of pictures (actual pictures and schematic pictures) are shown one by one.

92. Ch 592 Schematic pictures depicted by geons

93. Ch 593 Actual pictures

94. Ch 594 Experiment: The subject indicated the name of the object shown on the screen. In one case, an actual picture of the object was shown. In the other case, a schematic illustration of the object (depicted by geons) was shown.

95. Ch 595 Questions/design: Do subjects name the objects more quickly and accurately when actual pictures were shown or when schematic pictures (depicted by geons) were shown? Each picture (either actual/schematic) flashed on the computer screen only for 50ms, 60ms, or 400ms.

96. Ch 596 Why 50ms? Or 400ms? The task needs to be not too easy but not too difficult. –Test college students’ math ability. –test adding and subtracting? –Test high school students’ math ability. –Give quantum mechanics questions?

97. Ch 597 Results:

98. Ch 598

99. Ch 599

100. Ch 5100 The intelligence of human object perception Why are humans much better than computers at object recognition?

101. Ch 5101 Theory of unconscious inference –Human object perception is like problem solving. We make an unconscious inference. –Likelihood principle objects are perceived based on what is most likely to have caused the pattern. –Humans have a vast array of knowledge (intelligence) that can disambiguate ambiguous stimuli.

102. Ch 5102

103. Ch 5103

104. Ch 5104

105. Ch 5105

106. Ch 5106

107. Ch 5107

108. Ch 5108

109. Ch 5109

110. Ch 5110

111. Ch 5111

112. Ch 5112

113. Ch 5113

114. Ch 5114

115. Ch 5115

116. Ch 5116

117. Ch 5117

118. Ch 5118

119. Ch 5119

120. Ch 5120

121. Ch 5121

122. Ch 5122

123. Ch 5123

124. Ch 5124

125. Ch 5125

126. Ch 5126