Word Recognition (Sereno, 4/04) How long does it take to recognise a visual word? –What is meant by “recognition” or “lexical access”? –Can lexical access be accurately measured? –What factors affect lexical access and when? The “magic moment” (Balota, 1990) of lexical access: “At this moment, presumably there is recognition that the stimulus is a word, and access of other information (such as the meaning of the word, its syntactic class, its sound, and its spelling) would be rapid if not immediate.” (Pollatsek & Rayner, 1990)

Background: Basic Units of Language A. ~5,000 languages phonemes  morphemes  sentences  conversations (sounds) & words B. Phonemes = elementary sounds of speech phonemes are not letters... to, too, two, through, threw, shoe, clue, view vowel & consonant phonemes phonemes in any given language English has ~ 40; Hawaiian has ~16 combining phonemes is rule-governed

Wordness: For each row of 3 possible new words, which one will probably never make it : ( blicksplungerlight sbarmwumpleturl mancernserhtcrelurious intheriwhucrneen shacefringngout

Basic Units of Language C. Morphemes = smallest meaningful unit of lang. can be a word, word stem, or affix (prefix, suffix) word:help, love word stem:spir, ceive, duce prefix/suffix:re-, dis-, un- / -less, -ful, -er derivational & inflectional morphemes derivational – change the grammatical class V + -able = Adj (adorable, believable) V + -er= N (singer, runner) inflectional – grammatical markers V + -ed= past tense (walked) N + -s= plural (cows) “free” { “bound” {

Basic Units of Language C. Words Content vs. function (open- vs. closed-class) words Content words = carry the main meaning nouns, verbs, adjectives, adverbs Function words = grammatical words articles (a, the, this), conjunctions (and, but), prepositions (in, above) Psychological reality of the content-function word distinction in aphasia  selective impairment of content (Wernicke’s) or function words (Broca’s aphasia) Cattell (1886) & Stroop (1925)

Word superiority effect (Cattell, 1886) –Reicher (1969); Wheeler (1970) –tachistoscopic presentation –more accurate identification of the letter when stimulus is a word –pseudoword superiorty effect --- dkdk word  dkdk d 

RED BLUE BLACK GREEN RED GREEN BLACK BLUE RED BLACK BLUE BLACK BLUE GREEN BLUE GREEN RED GREEN NAME THE COLOUR OF THE INK

Basic Units of Language C. Words Ambiguity 1 word form, but 2 (or more) word meanings Ex: bank (N-N, “money” vs. “river”) watch(N-V, “clock” vs. “look”) bass (N-N, “guitar” vs. “fish”) 2 word forms, but 1 pronunciation Ex: sail/sale, right/write Generally unaware of ambiguity... even though it is quite pervasive even though it affects behaviour (RT, etc) homographs homophones

Basic Units of Language D. Sentences Syntax = the rule-governed system for grouping words together into phrases and sentences Sentences introduce a concept that they are about, the subject (or noun phrase), and then propose something about that concept, the predicate (or verb phrase). Ex:“The boy hit the ball.” doer act done-to (thematic roles) subject predicate

Basic Units of Language D. Sentences Same deep structure, different surface structure “The boy hit the ball.”(active) “The ball was hit by the ball.” (passive) Same surface structure, different deep structure [The French bottle] NP [smells.] VP [The French] NP [bottle smells.] VP THEY are boring. VISITING THEM is boring. cf. ambig. figures in perception: 1 form, 2 interpretations “The French bottle smells.” “Visiting relatives can be boring.”

Necker cube

New obesity study looks for larger test group Reagan wins on budget, but more lies ahead Man struck by lightening faces battery charge Enraged Cow Injures Farmer with Axe Milk Drinkers Are Turning to Powder Local High School Dropouts Cut in Half British Left Waffles on Falklands Dealers Will Hear Car Talk at Noon Miners Refuse to Work after Death Beating Witness Provides Names Squad Helps Dog Bite Victim Kids Make Nutritious Snacks Headlines

Stolen Painting Found by Tree Prostitutes Appeal to Pope Red Tape Holds up Bridge Deer Kill 17,000 Teenage Prostitution Problem is Mounting Child Stool Great for Use in Garden Shouting Match Ends Teacher’s Hearing Man Robs then Kills Himself Lung Cancer in Women Mushrooms Mondale’s Offensive Looks Hard to Beat Tuna Biting off Washington Coast Chinese Apeman Dated Headlines

Basic Units of Language D. Sentences Syntactic ambiguities “She hit the boy with the big stick.” “She hit the boy with the runny nose.” Interpretation depends on structural preferences (certain constructions used more often, favoured), as well as the prior discourse context.

Word Recognition (Sereno, 4/04) Measures Components Models Eye movements (EMs) Event-related potentials (ERPs) Word frequency & lexical ambiguity

Measures Standard behavioural techniques –lexical decision, naming, categorisation; also RSVP, self-paced reading –priming, masking, lateralised presentation –Donders (1868): subtractive method assumes strictly serial stages of processing additive vs. interactive effects –automaticvs. strategic (Posner & Snyder, 1975) unconscious exogenous bottom-up benefit controlled endogenous top-down cost & benefit

Measures Eye movements Neuroimaging –“Electrical”: EEG, MEG –“Blood flow”: PET, fMRI

MEASURE Normal reading TASK fixation duration (as well as location and sequence of EMs) TIME RES. GOOD POOR “blood flow” imaging: fMRI, PET “electrical” imaging: EEG, MEG various word tasks ms-by-ms seconds various word tasks naming categorisation lexical decision Standard word recognition paradigms (± priming, ± masking): RT ~500 ms ~600 ms ~800 ms ~250 ms

Components Orthography of language –English vs. Hebrew or Japanese Language skill –beginning (novice) vs. skilled (expert) reader –easy vs. difficult text

Components Intraword variables –word-initial bi/tri-gramsclown vs. dwarf –spelling-to-sound regularityhint vs. pint –neighborhood consistencymade vs. gave –morphemes prefix vs. pseudoprefixremind vs. relish compound vs. pseudocompoundcowboy vs. carpet

Components Word variables –word lengthduke vs. fisherman –word frequencystudent vs. steward –AoAdinosaur vs. university –ambiguitybank vs. edge, brim –syntactic classopen vs. closed; A,N,V –concretenesstree vs. idea –affective tonelove vs. farm vs. fire –etc.

Components Extraword variables –contextual predictability The person saw the... moustache. The barber trimmed the... –syntactic complexity Mary took the book.*Mary took the book was good. Mary knew the book. Mary knew the book was good. *Mary hoped the book. Mary hoped the book was good. –discourse factors (anaphora, elaborative inferences) He assaulted her with his weapon.... knife... stabbed

Models Dual-route account (Coltheart, 1978) Direct route (addressed) phonologysemanticsorthography Indirect route (assembled)

Models Interactive (Morton, 1969; Seidenberg & McClelland, 1989) /m A k/ phonology meaningorthography M A K E context

Models Modular (Forster, 1979; Fodor, 1983) decision output Lexical processor Syntactic processor Message processor General Problem Solver input features

Models Hybrid –2-stage: generate candidate set  selection –(Becker & Killion; Norris; Potter)

Word Recognition (Sereno, 4/04) Measures Components Models Eye movements (EMs) Event-related potentials (ERPs) Word frequency & lexical ambiguity

MEASURE Normal reading TASK fixation duration (as well as location and sequence of EMs) TIME RES. GOOD POOR “blood flow” imaging: fMRI, PET “electrical” imaging: EEG, MEG various word tasks ms-by-ms seconds various word tasks naming categorisation lexical decision Standard word recognition paradigms (± priming, ± masking): RT ~500 ms ~600 ms ~800 ms ~250 ms

Tools of choice: Recording eye movements in reading Recording ERPs in language tasks

Eye Movements (EMs) Best on-line measure of visual word recognition in the context of normal reading: Fast (avg fixation time ≈ 250 ms) Ecologically valid task Eye-mind span is tight

P1 N1 P300 N400 Number of trials EEG ERP

ERPs Best real-time measure of brain activity associated with the perceptual and cognitive processing of words: Continuous ms-by-ms record of events Early, exogenous components (before 200 ms) should reflect lexical processing

(Sereno & Rayner, Trends in Cognitive Sciences, 2003)

DIVERSION High-density ERP Analysis: A case of “too many notes”?

High-density ERP Analysis: Typical approaches for space & time Pick ‘n choose favourite electrode and ERP component

High-density ERP Analysis: Typical approaches for space & time Pick ‘n choose favourite electrode and ERP component Hunt down where/when the effect is strongest and gather data from those electrodes/time window

High-density ERP Analysis: Typical approaches for space & time Pick ‘n choose favourite electrode and ERP component Hunt down where/when the effect is strongest and gather data from those electrodes/time window Procrustean regions analysis (turtle shell) or series of pre-set time windows (eg, 50, 100, 200 ms)

High-density ERP Analysis: Typical approaches for space & time Pick ‘n choose favourite electrode and ERP component Hunt down where/when the effect is strongest and gather data from those electrodes/time window Procrustean regions analysis (turtle shell) or series of pre-set time windows (eg, 50, 100, 200 ms) Spatial and/or temporal principal component analysis (PCA)

Scalp topography of the ms SF1 loadingsVoltages (Sereno, Brewer, & O’Donnell, Psychological Science, 2003)

Word frequency Word frequency effect represents the differential response to commonly used high- frequency (HF) words versus low-frequency (LF) words that occur much less often. Presence of word frequency effects is used as a marker of successful word recognition or lexical access.

553 ms 490 ms 259 ms 275 ms 280 ms 293 ms (Sereno & Rayner, Trends in Cognitive Sciences, 2003)

Comparative EM & ERP studies Word frequency Word frequency X Context The sore on Tam-Tam’s was swollen. (LF) rump (HF) back (Neutral context) To our surprise we saw a (Biasing context) Flying to its nest was a hawk. (LF) (Neutral context) She looked over the (Biasing context) She read the new book. (HF) (Sereno, Rayner, & Posner, NeuroReport, 1998) (Sereno & Rayner, Perception & Psychophysics, 2000) (Sereno, Brewer, & O’Donnell, Psychological Science, 2003)

More comparative EM & ERP studies Word frequency X Orthographic regularity Lexical ambiguity X Context (Neutral context) James peered over at the (Biasing context) The mud was deep along the bank. Mike wasted the whole and then regretted it. ( LF-Reg) cask (LF-Exc) pint (HF-Reg) week (HF-Exc) hour (Sereno, Pacht, & Rayner, Psychological Science, 1992) (Sereno, JEP: Learning, Memory, and Cognition, 1995) (Sereno, Rayner, & Posner, NeuroReport, 1998) (Sereno & Rayner, Perception & Psychophysics, 2000) (Sereno, Brewer, & O’Donnell, Psychological Science, 2003)

(Sereno & Rayner, Trends in Cognitive Sciences, 2003)

Lexical Ambiguity Resolution Interactive position –Access is selective: Context guides access towards the appropriate sense of an ambiguous word; while both senses may be initially activated, only the contextually appropriate sense is fully accessed. Modular position –Access is exhaustive: All meanings of ambiguous words are automatically accessed; context cannot directly affect lexical processing, but instead operates on the output of the lexical processor to select the appropriate sense.

Ambiguity: Cross-modal priming Paradigm: Aud amb Visual Auditory context prime target “The building was infested with BUGS” ANT “and it…” SPY SEW Results: –In general, support the modularity of lexical processing. (Swinney, 1979)

Ambiguity: ERP unimodal priming Paradigm: –The only ERP ambiguity study employed a unimodal (visual) version of the cross-modal paradigm (Van Petten & Kutas, 1987). –Measured ERPs to targets that followed presentation of the ambiguous word prime. Results: –Support an interactive account of lexical processing.

Ambiguity: EM “fast priming” Paradigm: –Measured fixation duration on targets that followed presentation of the “fast” ambiguous word prime across various context conditions...

Ambiguity: EM “fast priming” Priming Effect ~30 ms n.s. (Sereno, JEP: LMC, 1995)

Ambiguity: EM “fast priming” Results: Support a modified interactive account of lexical processing - “reordered access” - in which both (1) meaning frequency (Dom vs. Sub) (2) prior context affect access speed. Specifically, (1) Alternative meanings become activated in order of their meaning frequency (2) Context can “boost” the activation of one of the meanings, possibly reordering access procedures

AmbiguousThe mud was deep along the bank... (subordinate) LFThe mud was deep along the brim... HFThe mud was deep along the edge... (Sereno, Pacht, & Rayner, Psych Sci, 1992) Fixation time:Amb = LF > HF “Spillover” time:Amb > LF > HF Support modified interactive account of access: “reordered access”. Paradigm: Results: Ambiguity: EM normal reading

Critique of methods: Cross-modal priming RT (lexical decision, naming) –Slow ( ms) compared to speed of lexical access (~ ms); susceptible to response bias. Secondary (indirect) measure –Effects of context on ambiguous word gauged by priming effects on target downstream.

Critique of methods: ERP unimodal priming Time-locked averages of the EEG –Ms-by-ms voltage fluctuations reflect processing in real time. Secondary (indirect) measure –Effects of context on ambiguous word gauged by priming effects on target downstream.

Critique of methods: EM “fast priming” Fixation time –Relatively fast (~375 ms) and on-line, but can reflect lexical and post-lexical integration effects. Secondary (indirect) measure –Although much quicker time course than cross- modal or ERP unimodal, effects of context on ambiguous word still gauged by priming effects on target downstream

Critique of methods: EM normal reading Fixation time –Fast (~250 ms) and on-line, but can reflect lexical and post-lexical integration effects. Primary (direct) measure –Effects of context on ambiguous word gauged by comparing its fixation time to control word.

ERP Ambiguity Experiment Biased ambiguous words were presented in neutral and biasing contexts in a word-by-word sentence reading paradigm. Biasing contexts always instantiated the subordinate sense. ERPs on the ambiguous words, themselves, were measured. ERPs to ambiguous words were then directly compared to ERPs to unambiguous control words. Control words - matched either to the dominant (HF) sense of the ambiguous word or to the contextually instantiated subordinate (LF) sense of the ambiguous word - were presented in neutral and biasing contexts. Comparisons across conditions were made at an early, lexical stage of processing (N1, ms).

Example Stimuli

Scalp topography of the ms SF1 loadingsVoltages

Scalp topography of the ms SF1 loadingsVoltages ± 0.7 factor loading contours

Factor scores for SF1 Voltages (electrodes with SF1 loading > +0.7 or SF1 loading < -0.7) ms

(Sereno & Rayner, Trends in Cognitive Sciences, 2003)

Summary From the EM record, we can infer when lexical processing should occur (~ ms). From the ERP record, we can see when certain differences first appear in real time. Effects of word frequency as well as context initially appear very early in the ERP record 132 ms post-stimulus). We can begin to establish a realistic time-line of word recognition in reading.

ERP Word Recognition Words Reg Exc LF cask pint HF time hour PseudoWords welf Consonant Strings fhvr LexicalityW vs PW vs ms FrequencyLF vs ms RegularityLF Reg vs LF ms Stimuli Results (Sereno, Rayner, & Posner, NeuroReport, 1998)

CS-WPW-WCS-PWp < Lexicality Effects: P1 ( ms) (Sereno, Rayner, & Posner, NeuroReport, 1998)

Frequency Effects: N1 ( ms) LF-HF (Sereno, Rayner, & Posner, NeuroReport, 1998)

Regularity Effects: P2 ( ms) LF Exc -LF Reg Ss with RT effectSs with no RT effect (Sereno, Rayner, & Posner, NeuroReport, 1998)

Scalp topography of the ms SF1 loadingsVoltages ± 0.7 factor loading contours (Sereno, Brewer, & O’Donnell, Psychologcial Science, 2003)

Emotion words Valence Arousal + ve – ve LoHi peacelove boredfire Neutral controls:hotel, farm

Questions of interest: How fast are words recognised? What factors affect lexical access? How early do these factors operate?

Models Interactive vs. Modular –Logogen (Morton, 1969) –PDP (Seidenberg & McClelland, 1989) Read-out –Search (Forster & Bendall, 19xx; ) Hybrid –2-stage: generate candidate set  selection –Becker & Killion, 19xx; Norris, 1984; Potter

Q: Why are camels called “ships of the desert?” A: Because they’re always filled with Arab sea-men. Q: Would you prefer roses on your piano or tulips on your organ? Q: What’s the difference between a rolling stone and a Scotsman? A: A Rolling Stone says “Hey you get off of my cloud!” and a Scotsman says “Hey McLeod get off my ewe!” Q: Why is men’s ‘sea-men’ white and their urine yellow? A: So they can tell if they’re coming or going. Jokes

Current Directions Emotion word processing Contextual constraint

ERP Ambiguity Experiment Design/Stimuli 6 experimental conditions: Word Type X Context LF Neutral HF Biasing Amb Word specifications: LF = 6 per million HF = 60 per million Amb = 63 per million (Dominant sense = 89 % Subordinate sense = 9%)

Spatial Principal Components Analysis Sample-by-sample voltage data at all 129 electrodes for all 14 Ss in all 6 conditions were submitted to a spatial PCA with a Quartimax rotation (cf. Spencer, Dien, & Donchin, 1998). The first Spatial Factor (SF1) accounted for a high degree of the variance (44%). 3(Word Type) x 2(Context) ANOVA was performed on the factor scores.