1. Conceptual Processing and Semantics (also more on the Mirror Neuron/Action Perception System)
A.P. Saygin Cogs 172 UCSD

2. The Nature of Conceptual Representations
How is semantics organized?
Sensory/Functional
Visual properties in a visual subsystem, auditory properties in an auditory subsystem, and so on…
Domain-specific
Organized by domains such as animals, plants, faces… Independent of sensory and motor representations.

3. Semantic Categories Specific deficit for living things
JBR, SBR, KB, ING (Warrington & McCarthy, 1983; Warrington & Shallice, 1984); TOB - auditory (McCarthy & Warrington, 1988)
Tasks used
Producing definitions
Matching names and pictures
Picture naming
Elizabeth Warrington

4. Semantic Categories Specific deficit for artifacts/non-living things
YOT (Warrington & McCarthy, 1987)
Tasks
Matching a spoken name to one of several pictures (among an array of choices)

5. Semantic Categories and Deficits
Specific deficit for artifacts/non-living things
YOT (Warrington & McCarthy, 1987)
Task: Matching a spoken name to one of several pictures (among an array of choices)
Can be more specific (e.g., musical instruments, food)
Tranel, Damasio & Damasio (1997): Meta-analysis of category specific deficits

6. Semantic Categories and Deficits
Patient EW (Caramazza & Shelton)
Naming test
100% Fruits & vegetables
92% Tools
34% Animals
Other tasks as well
PS: Caramazza overheard saying “The lesion can be in the toe!” – interested in the theory, not the neural bases

7. Living Nonliving Cases Animals Fruits/ Vegetables Artifacts X
Warrington & Shallice (1984), Silveri & Gainotti (1988)
Warrington & McCarthy (1983), Sheridan & Humphreys (1993)
Hart & Gordon (1992), Caramazza & Shelton (1998)
Hillis & Caramazza (1991)
Hart, Berndt & Caramazza (1985), Farah & Wallace (1992)
Not observed (so far)
X: Disproportionate impairment
See readings Caramazza and Martin et al. for more examples

8. How are living things and artifacts different?
Note: Warrington et al. did not necessarily view the patient deficits as evidence for domain-specificity
Instead discussed modality-specificity of semantic system
Visual vs. Functional
We may distinguish living things more by visual properties
- How they look, how they move, colour, etc.
Artifact knowledge more functional
- What is it for? How is it used?
Important to make sure the task does not confound the studies

9. However, Caramazza argues, there are patients whose deficits reveal category effects even when the tasks require functional judgments. (Visual judgments: Do ducks have long ears? Functional judgment: Are roses given on Valentine’s day?)
Does this argument convince you? Even if the judgment is functional, could the results be consistent with the concept requiring sensory (here, visual) representations to be used? Could this explain the patient data?
Living
NonLiving
Patient
Visual
Functional EW 67 74 96 99 GR 55 58 91 84 FM 73 69
JEN 42 51 75 83 DB
100 RC 70 79 90

10. Emergent Category Structure
Categories can “emerge” from a system that isn not domain-specific.
Semantic space is “lumpy” – i.e., things that move in a certain way tend to be made of similar materials.
Items that share many properties ~ highly correlated properties will cluster in the brain
What is the difference?
- Domain-specific vs. emergent categories

11. Alex Martin et al. Sensorimotor/Functional model Distributed networks
Embodiment
Animate objects –visual properties
Tools – functional properties

12. Sensory/Functional Account
Evidence for modality effects
Patients who cannot recognize objects in one modality, but can in another (Shallice, 1987; McCarthy & Warrington, 1988; Saffran & Schwartz, 1994)

13. Neuroimaging Studies in the Healthy Brain
Silently naming animal drawings: occipital
Verbalizing the action: Motor and superior temporal areas
Verbalizing the color: Ventral areas near regions involved in color perception
Silently naming tool drawings: motor
See Martin, Ungerleider & Haxby chapter – This is pretty old for neuroimaging and methods have advanced since then but these studies illustrate the general points (& the debate is ongoing)

14. Viewing and naming manipulable objects activates left premotor and parietal cortex (Chao & Martin, 2000)

15. Why?
We talked about mirror neuron theory (and biological motion). A brief overview again

16. Understanding Others Theory Neuroscience
3rd person (“other”) actions: Often visually perceived
1st person (“self”) actions: Rarely visually perceived
Yet we are able to know what we perceive (Barresi & Moore, 1996, BBS)
Neuroscience
Visual hypothesis – Based on a visual analysis of the elements (body parts, objects, motion, etc). No sensorimotor involvement is required
Simulation hypothesis – Analysis-by-synthesis. We map the visual representation onto our own sensorimotor representations

17. Frontal and posterior cortical regions are heavily interconnected
Perception and Action
Frontal and posterior cortical regions are heavily interconnected
“Networks”

18. Mirror Neurons
Frontal area F5 in the macaque (Rizzolatti lab, ca 1996)
Later also found in parietal cortex
Neuroimaging in humans: The mirror neuron system
Inferior frontal cortex
Inferior parietal cortex
Superior temporal sulcus (STS)

19. Theory
An individual can understand others’ actions by mapping the visual representation of the observed action onto his/her sensorimotor representation of the same action, thus using his/her own embodied experience of the world.
“An action is understood when its observation causes the motor system of the observer to ‘resonate’ ” (Rizzolatti, Fogassi, & Gallese, 2001).

20. Mirror Neuron System
Seen as a possible neural resource for linking between self and other, and for empathy, and social cognition.
The specific properties of the system and the possible involvement of the mirror neuron system in brain disorders (from autism to psychiatric disorders) is currently of interest in cognitive neuroscience.

21. Mirror Neuron System
Seen as a possible neural resource for linking between self and other, empathy, and social cognition.
That’s important stuff. So mirror neurons have been popular!
The involvement of the mirror neuron system in associated disorders (from autism to psychiatric disorders) is currently of intense interest.
The mirror neuron system is:
- Not a silver bullet
Have sometimes been overhyped
Functional properties still need to be specified

22. Embodiment We can be more agnostic about “mirror neurons” per se
From mirror neurons to theories of perceptual and conceptual processing…

23. Mirror-like responses: Empathy for pain
Pain observation increases activity in affective pain processing areas
Singer et al., 2004, Science

24. Mirror-like responses: Disgust
or olfactory stimulation
Wicker et al., 2003, Neuron

25. Mirror-like responses: Listening to and producing speech
UCLA/UCSD Study
Listen to monosyllables /pa/ /gi/
Produce monosyllables /pa/ /gi/
Overlap found in premotor cortex.
Note: This is a more superior area than Broca’s area.
Wilson et al., 2004, Nature Neurosci.

26. Link to Semantics
Activations for tools Chao & Martin could reflect automatic engagement of action/tool-use related systems Frontal ~ canonical neurons (manipulable objects)

27. Sensorimotor/Functional Theory of Semantics
Semantic representations are related to sensorimotor properties
Automatic activation of semantic representations
Imaging, patient, TMS and behavioral studies
Color - ventral temporal anterior to color perception areas
Motion - lateral temporal lobe
Superior: biological motion
Middle: artifact motion
Actions: Motor, premotor, parietal areas
Modulations in sensory or motor regions can be evoked using linguistic stimuli
Action words (Hauk, Johnsrude, & Pulvermuller, 2004; Tettamanti et al., 2005); odour-related words (Gonzalez et al., 2006)

28. Somatotopy in action observation
Buccino et al 2001, Eur. J Neurosci

29. Somatotopy in Language Semantics
Foot action words
Hand action words
Mouth action words
Hauk, et al., 2004, Neuron

30. Motion Semantics and Vision
FMRI study – we localized motion-sensitive area V5/MT+ and face-sensitive area FFA
Motion sentences: “I drove from Modesto to Fresno”
Static sentences: “Modesto and Fresno are in California”
Fictive motion sentences: Use of a motion verb when there is no physical occurrence of motion (Talmy 1996, 2000)
“The highway runs from Modesto to Fresno”
“The mountain range goes from Mexico to Canada”

31. Saygin, McCullough, Alac & Emmorey, 2010, J. Cognitive Neuroscience

32. Readings: Caramazza, 1999 and Martin, Ungerleider & Haxby, 1999 What do you think?