WORD SEMANTICS 2 DAY 27 – OCT 30, 2013 Brain & Language LING NSCI Harry Howard Tulane University

Course organization The syllabus, these slides and my recordings are available at If you want to learn more about EEG and neurolinguistics, you are welcome to participate in my lab. This is also a good way to get started on an honor's thesis. The grades are posted to Blackboard. 10/30/13Brain & Language - Harry Howard - Tulane University 2

REVIEW 10/30/13Brain & Language - Harry Howard - Tulane University 3

Linguistic model, Fig. 2.1 p /30/13Brain & Language - Harry Howard - Tulane University 4 Discourse model Syntax Sentence prosody Morphology Word prosody Segmental phonology perception Segmental phonology perception Acoustic phonetics Feature extraction Segmental phonology production Segmental phonology production Articulatory phonetics Speech motor control INPUT Sentence level Word level

10/30/13Brain & Language - Harry Howard - Tulane University 5

Semantic networks 10/30/13Brain & Language - Harry Howard - Tulane University 6

LEXICAL SEMANTICS 2 Ingram: III. Lexical semantics, §10. 10/30/13Brain & Language - Harry Howard - Tulane University 7

The linkages in such a network are … associative … established by the fact that certain words are often used together, such as pig and farm; these are ‘accidental’, in the sense that there is nothing in the meaning of pig that requires them to be associated with farms; they are often defined in a free association test, by giving a subject the prime word and asking her to say the first word that comes mind; or semantic … the relationships of meaning mentioned yesterday, such as part- whole; these are necessary, in the sense that a hand is by definition made up of fingers. 10/30/13Brain & Language - Harry Howard - Tulane University 8

Caveat I grant that the distinction between associative and semantic relationships can be difficult to pin down. Note that psychologists would call semantic networks ‘semantic memory’, while linguists would say that most of these networks contain real- world knowledge, which is different from linguistic semantics. So let us look at an experiment that tries to tease these two domains apart. 10/30/13Brain & Language - Harry Howard - Tulane University 9

Semantic + associative vs. non-associative prime-probe relations Table 10.4, Moss et al. (1995) Semantic relation Category coordination [taxonomy] Function NaturalArtifactInstrumentalScripted Associated cat – dogboat – shipbow – arrowtheater – play brother – sistercoat – hatumbrella – rainbeach – sand Non- associated aunt – nephewairplane – trainknife – breadparty – music pig – horseblouse – dressstring – parcelzoo – penguin 10/30/13Brain & Language - Harry Howard - Tulane University 10 Increased priming with respect to control condition in which there is no relationship between prime and probe: unrelated (control) < semantic + non-associative < semantic + associative

Leftovers The modality of presentation has a large influence. Auditory priming fades much more quickly than visual priming. Priming has shown that multiple word meanings are activated before a word is actually recognized. This reminds of the TRACE model, which is reviewed in the next slide. 10/30/13Brain & Language - Harry Howard - Tulane University 11

An alternative: the TRACE II model 10/30/13Brain & Language - Harry Howard - Tulane University 12

Activation in a semantic network 10/30/13Brain & Language - Harry Howard - Tulane University 13

Some results from brain imaging I have mentioned a few times a general division of the brain into posterior or sensory cortex (occipital, temporal & parietal lobes) and anterior or motor cortex (frontal lobe). Should this have any relevance for language? Nouns vs verbs Many nouns have ‘high imageability’ and so should require more activation from visual cortex (temporal-occipital lobes) Verb should require more activation from motor cortex (frontal lobe) Not all results are consistent, but by and large this is true. 10/30/13Brain & Language - Harry Howard - Tulane University 14

Levels of categorization On a scale of 1 to 7, rate the following items as a good example of the category furniture. 1 chair 1 sofa 3 couch 3 table 5 easy chair 6 dresser 6 rocking chair 8 coffee table 9 rocker 10 love seat 11 chest of drawers 12 desk 13 bed 10/30/13Brain & Language - Harry Howard - Tulane University 15

Hierarchy of categories furniture | chair | bench domain-level | basic/prototype | subordinate 10/30/13Brain & Language - Harry Howard - Tulane University 16

Basic is special 1. Response Times: in which queries involving a prototypical member (e.g. is a robin a bird) elicited faster response times than for non-prototypical members. 2. Priming: When primed with the higher-level (superordinate) category, subjects were faster in identifying if two words are the same. Thus, after flashing furniture, the equivalence of chair-chair is detected more rapidly than stove-stove. 3. Exemplars: When asked to name a few exemplars, the more prototypical items came up more frequently. 10/30/13Brain & Language - Harry Howard - Tulane University 17

Basic is really special 1) It is the highest level at which a single mental image can represent the entire category (you can’t get a mental image of vehicle or furniture). 2) It is the highest level at which category members have a similarly perceived overall shape. 3) It is the highest level at which a person uses similar motor actions for interacting with category members. 4) It is the level at which most of our knowledge is organized. 10/30/13Brain & Language - Harry Howard - Tulane University 18

The fMRI experiment 10/30/13Brain & Language - Harry Howard - Tulane University 19

10/30/13Brain & Language - Harry Howard - Tulane University 20 Why we are interested in vision The easiest stimuli to use are visual, so we will be gathering information about vision anyway pictures (name this picture) text Reading disorders (dyslexia) have a linguistic component We are ultimately more interested in audition, however, but perhaps some of what we learn from vision will generalize to it

10/30/13Brain & Language - Harry Howard - Tulane University 21 Early and late vision early vision is beneath the surface; late vision is on it

10/30/13Brain & Language - Harry Howard - Tulane University 22 Norman (2002) Constructivist approach ventral ~ identification the stimulation is inherently insufficient, necessitating an “intelligent” perceptual system that relies on inference perception is indirect/multistage process between stimulation and percept memory, stored schemata, and past experience play an important role in perception excels at analyzing the processes and mechanisms underlying perception Ecological approach dorsal ~ visual control of motor behavior the information in the ambient environment suffices and is not equivocal, and thus, no “mental processes” are needed to enable the pick-up of relevant information perception is direct/single- stage process no role for memory or related phenomena excels at the analysis of the stimulation reaching the observer affordances

10/30/13Brain & Language - Harry Howard - Tulane University 23 The what / ventral pathway (Palmeri & Gauthier 2004)

CATEGORY-SPECIFIC DEFICITS 10/30/13Brain & Language - Harry Howard - Tulane University 24

THE FUNCTIONAL ORGANIZATION OF THE VENTRAL VISUAL PATHWAY AND ITS RELATIONSHIP TO OBJECT RECOGNITION Grill-Spector /30/13Brain & Language - Harry Howard - Tulane University 25

10/30/13Brain & Language - Harry Howard - Tulane University 26 Introduction Humans recognize objects and faces instantly and effortlessly. What are the underlying neural mechanisms in our brains that allow us to detect and discriminate among objects so efficiently? Here we examine whether the human ventral stream is organized more around stimulus content or recognition task.

10/30/13Brain & Language - Harry Howard - Tulane University 27 Previous discoveries Multiple ventral occipitotemporal regions anterior to retinotopic cortex respond preferentially to various objects compared to textures. Functional imaging studies have revealed that some of these regions respond maximally to specific object categories, such as: faces (fusiform face area) places (parahippocampal place area) body parts letter strings tools animals These results suggest that areas that elicit a maximal response for a particular category are dedicated to the recognition of that category.

10/30/13Brain & Language - Harry Howard - Tulane University 28 Problems Comparing activation between a handful of object categories is problematic because it depends on the choice of categories. While there is maximal activation to one category the activation to other categories is not negligible Comparing the amplitude of activation to object categories does not exclude the possibility that the underlying representation might not be of whole objects. Objects from different categories differ in many dimensions and it is possible that the source of higher activation for a category is not restricted to visual differences.

10/30/13Brain & Language - Harry Howard - Tulane University 29 Three ways to represent objects Kanwisher (2000): ventral temporal cortex contains a limited number of modules specialized for the recognition of special categories (faces, places, body parts) and the remaining cortex is a general-purpose mechanism for the perception of any shape Haxby et al.(2001): occipitotemporal cortex is organized according to form attributes. The representation of an object is reflected by a distinct pattern of response across ventral cortex, and this distributed activation produces the visual percept Tarr and Gauthier (2000): occipitotemporal cortex is organized according to the perceptual processes carried out and not by the content of information processed – different cognitive processes require different computations

10/30/13Brain & Language - Harry Howard - Tulane University 30 Experimental tasks Detection subjects decide whether or not a gray-scale image contains an object Identification subjects discriminate between objects belonging to the same basic level category for instance, a particular subordinate member of a category (e.g. electric guitar) from other members of that category (e.g. acoustic guitars)

10/30/13Brain & Language - Harry Howard - Tulane University 31 Results When the category was held constant but subjects performed different recognition tasks (detection vs. identification) similar regions in the human ventral stream were activated. When the task was kept constant and subjects were required to identify different object categories, different regions of the human ventral stream were activated. This suggests that the human ventral stream is organized more around visual content than visual process.

NEXT TIME Q7 Continue with word semantics 10/30/13Brain & Language - Harry Howard - Tulane University 32