1. Categorization: Scenes & Objects (P) Lavanya Sharan March 16th, 2011

2. Last time What is category? Functional vs. communicational Basic-level categories (Rosch) Entry-level categories and prototypicality

3. Visual perception of categories Objects (e.g., animal vs. non-animal, cars vs. houses, German shepherds vs. other dogs etc.) Kirchner & Thorpe, 2005 Grill-Spector & Kanwisher, 2005

4. Visual perception of categories Scenes (e.g., desert vs. canyon, low openness vs. high openness etc.) Oliva & Schyns, 2000 Greene & Oliva, 2009

5. Why does the choice of category matter? Object Non-object Detection task: Should be easiest and fastest.

6. Why does the choice of category matter? House Dog Categorization task: Should be harder and slower? Image sources: dogbreedinfo.com, cambridge2000.com

7. Why does the choice of category matter? Danish Farm Dog Old Danish Chicken Dog Categorization task: This one should be hardest and slowest? Image source: dogbreedinfo.com,

8. Classic results in object recognition Mostly from psychophysical experiments. We can tell object from non-object in a brief glance. We can tell scene categories in a brief glance. Beyond categories, details about scenes and objects can be inferred in a brief glance. Short reaction times and ERP data suggests we can do these tasks quickly. Objects: Thorpe et al. 1996, Grill-Spector & Kanwisher 2005, Kanwisher et al. 1997, Potter 1975, Rosch 1978, Nakayama et al. 1995, Biederman 1987, Intraub 1981, Peterson & Gibson 1993,… Scenes: Biederman 1972, Potter 1975, 1976, Intraub 1981, Oliva and Schyns 2000, Oliva & Torralba 2001, Rousselet et al. 2005, Evans & Treisman 2005, Fei-fei et al. 2007, Greene & Oliva 2009,...

9. Today: Object recognition & fMRI Slide source: Jody Culham

10. Today: Object recognition & fMRI Slide source: Jody Culham

11. Why care about what fMRI has to say? Slide source: Jody Culham

12. Category-specific regions Image source: Grill-Spector 2008 Blue = Object > Scrambled objects Lateral Occipital Complex (LOC) Red = Faces > Non-face objects Fusiform Face Area (FFA) + few others Green = Places > Objects Parahippocampal place area (PPA) + few others Magenta = Faces + Objects Dark Green = Places + Objects Also regions for body parts, letter vs. textures, tools vs. animals etc.

13. LOC: Basics Two regions: LO + pFus/OTS Shape, surfaces, contours. Not low-level features (e.g., colors, textures). Global shape, not local contours. Image source: Grill-Spector 2008

14. LOC: Basics Image source: Grill-Spector 2008

15. Objection recognition implies invariances Size Position Rotation Illumination... Can fMRI activations tell us how these invariances are achieved? Unlikely. But, let’s study how invariant LOC responses are. Is it truly correlated with successful object recognition?

16. Measuring position invariance in LO Image source: Grill-Spector 2008

17. Position effects larger than category effects in LO! Image source: Grill-Spector 2008 For pFus/OTS greater position invariance than LO (Schwarzlose et al. 2008)

18. Rotation sensitivity in LOC Image source: Grill-Spector 2008 Andreson et al. 2009 used fMRI adaptation. If a region is sensitive to rotation, then repeating the same (or similar view) will cause a change in response. Mixed story. Rotation sensitivity depends on categories and brain regions.

19. Implications for object recognition theories/models View dependence. What is the representation for object recognition? View-sensitive neurons (low-level representations) and population coding? Or view-invariant neurons outside regions measured? Do we really need view-invariant representations for object recognition?

20. Implications for object recognition theories/models Domain specialization. Given that some object categories seem special, what does that imply for object recognition theories? General-purpose computations vs. specialized features and computation? Alternative interpretation of fMRI data: These regions are really about expertise with a visual category rather than the category itself (e.g., faces).

21. Implications for object recognition theories/models Distributed processing. It is possible to predict (using machine learning techniques on fMRI data) which object category a person is looking at (even when FFA etc. are not considered). Advantage of distributed code, recognize more objects?

22. Implications for object recognition theories/models Effect of experience. Size of category-specific regions changes as children mature to become adults. These changes are not simply geometric scaling. Learned representations vs. innate modules?