Word recognition in normal reading Sara C. Sereno Collaborators: RAs/PGs: Paddy O’DonnellSébastien Miellet Hartmut LeutholdGraham Scott Christopher Hand

Word Recognition What factors affect word recognition? How can word recognition processes be accurately measured? How can effects be interpreted?

Orthography of language –English vs. Hebrew or Japanese Intraword (sublexical) variables –word-initial bi/tri-gramsclown vs. dwarf –spelling-to-sound regularityhint vs. pint –neighborhood consistencymade vs. gave –morphemes prefix vs. pseudo-prefixremind vs. relish compound vs. pseudo-compoundcowboy vs. carpet What factors affect word recognition?

Word (lexical) variables –word lengthduke vs. fisherman –word frequencystudent vs. steward –AoArabbit vs. violin –expert vocabularyvoxel –syntactic classopen/closed-class; A,N,V –ambiguitybank vs. edge, brim –concreteness/imagabilitytree vs. idea –animacydog vs. cup –affective tonelove vs. farm vs. fire

What factors affect word recognition? Extraword (supralexical) variables –Contextual predictability Neutral He bought a large plant for his garden. Biasing Terry went to the new gardening centre. He bought a large plant for his garden. –Syntactic complexity Trans. Mary took the book VERBMary knew the book Intrans. Mary hoped the book on the table. was good. on the table. was good. on the table. was good.

Extraword (supralexical) variables –Discourse factors Focus The dog chased the cat today. The cat was chased by the dog today. What the dog chased was the cat today. It was the cat that was chased by the dog today. Elaborative inferences & anaphora What factors affect word recognition? weapon … The mugger her with his weapon… He threw the knife into the bushes and ran away. stabbed assaulted

What factors affect word recognition? Language skill –beginning (novice) vs. skilled (expert) readers –normal vs. dyslexic vs. neuropsychological patient How can word recognition processes be accurately measured?

Measure Task Time Res. “electrical” imaging single word presentation ~ ms (EEG, MEG) word-by-word reading (P1,N1,EPN,N400) Eye movements in fixation time, location & ~250 ms normal reading sequence of EM’s Stnd. word recogn. naming ~500 ms ± priming, masking, lexical decision RT ~600 ms lateralized present. categorization ~800 ms “blood flow” imaging single word presentation seconds (PET, fMRI)

Thisisawordbywordpresentationofasentenceatafastreading-likerate. Word-by-word reading: 200 ms per word

Thisisawordbywordpresentationofasentenceataslowratetypically usedinERPstudies. Word-by-word reading: 600 ms per word

Normal Reading

This is an approximation of normal reading in real time. *

This is an approximation of normal reading in real time. *

This is an approximation of normal reading in real time. *

This is an approximation of normal reading in real time. *

This is an approximation of normal reading in real time. *

This is an approximation of normal reading in real time. *

This is an approximation of normal reading in real time. *

This is an approximation of normal reading in real time. *

This is an approximation of normal reading in real time. *

This is an approximation of normal reading in real time. *

The importance of making eye movements in normal reading Cond1 There was a box of… Cond2 There was an enormous box of… Cond1 She saw a cat in the… Cond2 She saw a cup in the… The measure affects what is being measured: –Perception of text influences how EMs made. –Location/duration of EMs affect perception.

Theoretical approaches –Interactive (top-down) vs. Modular (bottom-up) Additive factors How can effects be interpreted? Stimulus Quality Context Frequency RT

Modelling How can effects be interpreted?

Modelling –Repeated measures multiple regression analyses: Oculomotor-related factors launch distance to word location of fixation within word number of fixations on word word length word frequency contextual predictability Language-related factors How can effects be interpreted?

FactorsMeasuresApproach orthography bi-/tri-grams regularity neighborhood morphology length frequency jargon word class ambiguity imagability animacy emotionality predictability syntactic prefs. focus inference anaphora skill ERPs + word-by-word (slow) presentation Eye movements + normal reading EM-ERP co-registration? Additive factors Repeated measures multiple regression

Distributed hierarchical visual processing in primates lexicalhumans higher-level semantics syntax meanings word forms letters features

Why? Precisely delineate the time course of different levels of linguistic processing. Help inform a temporally realistic neural circuitry of normal reading.

The importance of making eye movements in normal reading Cond1 There was a box of… Cond2 There was an enormous box of… Cond1 She saw a cat in the… Cond2 She saw a cup in the… The measure affects what is being measured: –Perception of text influences how EMs made. –Location/duration of EMs affect perception.

Theoretical approaches –Interactive (top-down) vs. Modular (bottom-up) Additive factors How can effects be interpreted? Stimulus Quality Context Frequency RT

How can effects be interpreted? Modelling approach

Modelling –Repeated measures multiple regression analyses: Oculomotor-related factors launch distance to word location of fixation within word number of fixations on word word length word frequency contextual predictability Language-related factors How can effects be interpreted?

Oculomotor-related factors launch distance to word location of fixation within word number of fixations on word word length word frequency contextual predictability Language-related factors How can effects be interpreted?

Why is word recognition important? By precisely delineating the time course of different levels of linguistic processing, it is possible to: Determine which aspects of processing are Interactive (top-down) or Modular (bottom-up). Help inform a temporally realistic neural circuitry of normal reading.

This is an approximation of normal reading in slow motion.

Word Recognition What factors affect word recognition? How can word recognition processes be accurately measured? How can effects be interpreted?

Word Recognition What factors affect word recognition? How can word recognition processes be accurately measured? What can accurately measured factors of word recognition tell us?

What factors affect word recognition? Intraword (sublexical) variables Word (lexical) variables Extraword (supralexical) variables

Emotion Word Processing: Scott, O’Donnell, Leuthold, & Sereno RTs, Eye Movements, & ERPs University of Glasgow Glasgow Language Processing

Emotion Words What are emotion words? –Express an emotional state (e.g., angry, happy) –Elicit an emotional state (e.g., snake, puppy) 2 traditional dimensions of emotion words –Arousal ≈ internal activation –Valence ≈ value or worth

valence arousal low (negative) high (positive) low highsexwarnun

Early Emotion Word Processing Lexical Decision (LD) / ERP experiment Eye Movement (EM) reading experiment Word frequency –High-frequency (HF) words are read more quickly than low-frequency (LF) words. –A word frequency effect (HF<LF) is used as a marker (index) of successful word recognition (lexical access).

Past Behavioural Experiments Stimuli negative vs. neutral words positive vs. neutral words emotional state words In general, stimuli are not well-controlled for psycholinguistic variables such as word length and frequency.

Past Behavioural Experiments Task Manipulations lexical decision masking emotional decision priming recollection mood induction odd-ball paradigms lateralised presentation forced-choice tasks stimulus repetition self-referential judgments blocked presentation

Lexical Decision Emotion (Pos, Neg, Neut) x Frequency (HF, LF) 40 words of each type (240 total words) HF: Pos, Neg, Neut LF: Pos, Neg, Neut 240 length-matched non-words pronounceable pseudowords (blimble)

Lexical Decision Norms Arousal & Valence: ANEW (1000 words) Frequency: BNC (90 million written words) Randomised presentation – NO repetition, lateralised presentation, priming, masking, self- referential judgments, mood induction… 26 participants

Arousal Valence Freq Length (lo-hi, 1-9) (neg-pos, 1-9) (per mill) (char) LD: Stimulus Specifications (N=40) Pos LFPos Neut Neg Pos HFPos Neut Neg

rage venom detest hawk truck invest fame cheer elated destroy jealousy scissors ketchup admired ecstasy destroy jealousy scissors ketchup admired ecstasy suffocate leprosy pamphlet highway valentine treasure suffocate leprosy pamphlet highway valentine treasure shark slap rude muddy cane lump glory sexy lust ambulance torture lighthouse privacy millionaire miracle fire anger danger book hotel fabric joke brave pretty cancer divorce manner avenue travel birthday cancer divorce manner avenue travel birthday accident violent reserved village beautiful success accident violent reserved village beautiful success abuse fear burn bench bowl rock happy kiss gift suspicious panic concentrate glass excitement desire

LF: Pos=Neg < Neut HF: Pos < Neut=Neg

Measurement EMs = best on-line measure of visual word recognition in the context of normal reading ERPs = best real-time measure of brain activity associated with the perceptual and cognitive processing of words

(Sereno & Rayner, Trends in Cognitive Sciences, 2003)

Sereno, Rayner, & Posner (1998). NeuroReport. Sereno, Brewer, & O’Donnell (2003). Psych. Sci.

Past ERP Experiments Kanske & Kotz (2007) –Abstract & concrete Pos, Neg, Neut words –Lateralised presentation; stimulus repetition –Blocked: [Pos & Neut]; [Neg & Neut] –P2 ( ms) effects, but not with go/nogo LD Herbert et al. (2006) –Pleasant, unpleasant, and neutral adjectives –Task: emotionally evaluate and memorise words –Startle response induced on 1/3 of trials! –P2 ( ms) effects

ERP Experiment Same Materials/Design as in Lexical Decision Same 26 participants as in Lexical Decision Apparatus: BIOSEMI Active-Two amps 70 electrodes presentation controlled by ERTS N1 component: ms post-stimulus

N1 Topography ms

LF: Pos=Neg < Neut HF: Pos=Neut < Neg

Eye Movement Experiment No past EM emotion word experiments. Emot (Pos, Neg, Neut) x Freq (LF, HF) 15 words of each type → 90 expt sentences Measure fixation time on target words in neutral sentences via Dual Purkinje Eyetracker. 48 participants

Pos LFPos Neut Neg Pos HFPos Neut Neg Arousal Valence Freq Length (lo-hi, 1-9) (neg-pos, 1-9) (per mill) (char) EM: Stimulus Specifications (N=15)

Tom delivered the _____ with great care and attention. Phoebe discussed the _____ at great length with her friends. Michelle dreamt about the _____ every night for weeks. bombnewskiss spidercamelpuppy Lisa read about the _____ in her animal book. A sturdy creature, the _____ can survive in many habitats. The documentary on the _____ was very interesting. LF: HF:

Results: Fixation Time Measures Early –First fixation duration (FFD) –Single fixation duration (SFD) –Gaze duration (GD) Later –Next forward-going fixation after target (‘spillover’) –Total Fixation Time (TT) reject 6% skip10% 1 fix71% 2+ fix13%

LF: Pos=Neg < Neut HF: Pos < Neut=Neg

Conclusion We examined HF and LF Pos, Neg, Neut words across 3 measures: LexDec, ERP, and EM reading measures. Reponses were modulated by 3 factors: Frequency, Arousal, and Valence. First to show early lexical effects of emotion.

Lexical Decision LFHF Pos Neut Neg Single Fixation Duration LFHF Pos Neut Neg282286