Psy1302 Psychology of Language Lecture 8 & 9 Mental Dictionary – The Lexicon Spoken Word Recognition

Agenda of the day Two related questions How are words organized and stored in our brain? How do we retrieve, look up words from memory? Other organizations of the dictionary: (1) Filed by last letter of the word instead of first letter, (2) filed first by word length and then letter, etc.

The Lexicon A lexicon is usually a list of words together with additional word-specific information, i.e., a dictionary.

Question 1 How are words organized and stored in our brain?

Subtext How might psychology experiments inform us of our mental processes help us create models of our mental representations and of how our mind process information?

Lexical Decision Task A Classical Reaction Time (RT) Paradigm Task: Respond to “Is it a word?” Measure: RT for deciding “yes” or “no” “blicket” is not a word. Respond: NO “lick” is a word. Respond: YES RT (i.e., speed of look-up) informs us of the organization of the mental lexicon

Lexical Decision Task CLAM NUMBER BLUTY PLUP SNARL + ASK RASK PURCE

Lexical Decision Task Frequency Effect - Results (Embick et al., 2001)

Lexical Decision Task Frequency Effect - Summary In a lexical decision task, responses are faster for more frequent words Implies lexicon organized by frequency

Lexical Decision Task Semantic Effect Semantically Related Word Pairs butter milk doctor nurse hand finger speak talk sound volume book volume

Lexical Decision Task Semantic Effect - Design BREAD BUTTER NURSE BUTTER WINE PLAME YES Top: Word Bottom: Associate PLAME WINE YES Top: Word Bottom: Non-associate PABLE REAB NO Top: Word Bottom: Nonword NO Top: Nonword Bottom: Word NO Top: Nonword Bottom: Nonword Meyer, D. E., & Schvaneveldt, R. W. (1971)

Lexical Decision Task Semantic Effect - Results RT Associated words < RT of ALL other conditions Of the NO responses of First word is a Nonword < RT First word is a word Meyer, D. E., & Schvaneveldt, R. W. (1971)

Lexical Decision Task Semantics Effect - Summary In a lexical decision task, responses are faster for semantically associated words [Terminology: when a target (e.g., BUTTER) is preceded by a stimulus (e.g., BREAD) and the target is processed faster, this is called “PRIMING” BREAD primes BUTTER BREAD is the “prime”.] Implies lexicon organized by meaning

Variants of Priming Present the words serially prime = CAPTAIN target = BOAT Time (lags vary) CAPTAIN #### (sometimes introduce masking or other words introduced in between) BOAT http://opl.apa.org/Experiments/Start.aspx?EID=8

Variants of Priming Cross-modal Priming “captain” BOAT Sometimes hear sentences as opposed to words in isolation

Cross-Modal Priming (Marslen-Wilson & Zwitserlood) Prediction of RT? CAPTAIN PAMPHLET FAST BOAT BOAT SLOW < Things to control about the primes (CAPTAIN & PAMPHLET): frequency, syllable duration, and # of letters are comparable.

Lexical Decision Task Phonology effects Design your own experiment How might you test whether the lexicon is organized by phonology? For Example: When you are hearing “spinach” do you access words like “spinster” or “spinach”?

Cross-modal Priming (Marslen-Wilson & Zwitserlood) Hear word through earphone. E.g. “Captain” Judge whether string presented on monitor is a word or not a word. Does hearing “captain” which sounds like “capital” PRIME “money”? KAPITEIN KAPITAAL

Cross-modal Priming (Marslen-Wilson & Zwitserlood) Vocabulary in Dutch KAPITEIN: captain BOOT: boat KAPITAAL: capital GELD: money PAMFLET: pamphlet

Cross-Modal Priming (Marslen-Wilson & Zwitserlood) “KAPITEIN” (captain) Hear Prime: GELD KAPITAAL-GELD Lexical Decision: (capital-money) KAPITEIN & KAPITAAL controlled for frequency (captain) (capital)

Cross-Modal Priming (Marslen-Wilson & Zwitserlood) “PAMFLET” (pamphlet) Hear Prime: GELD PAMFLET-GELD Lexical Decision: (pamphlet-money) KAPITEIN & PAMFLET controlled for frequency (captain) (pamphlet)

Cross-Modal Priming (Marslen-Wilson & Zwitserlood) Prediction of RT if KAPITEIN activates KAPITAAL? KAPITEIN PAMFLET (captain) (pamphlet) BOOT BOOT FAST SLOW (boat) < (boat) = =< GELD GELD MEDIUM < SLOW (money) (money)

Summary of Experiments The 3 Lexical Decision & Cross-modal Priming Tasks show: lexicon organized by frequency lexicon organized by semantic relatedness lexicon organized by phonological relatedness

Question 2 How do we retrieve, look up words from memory?

Two Historical Models of Organization and Retrieval Serial Models (Forster) Parallel Models (Morton)

Serial Search Models Forster 1989 Serially search to match word Frequency ordered phonological list Explains more frequent words are accessed first Semantically associate list Priming & Context plays a role and makes less frequent words be accessed faster

Serial Search Models Forster’s Model

Serial Search Models Forster 1989 Serially search to match word Frequency ordered phonological list Explains more frequent words are accessed first Semantically associate list Priming & Context plays a role and makes less frequent words be accessed faster

Serial Search Models Evaluation Physiological unrealistic Neuron transmission slow 100 time steps can be computed in ½ second (Feldman & Ballard, 1982) But retrieval of a word is FAST If we know ~75K words, searching serially is time consuming.

Parallel Search Models Morton’s Logogen model (1964, 1979). (father of other models to come) Next few slides adapted from J. Simner.

Parallel Search Model Morton’s Logogen Model The lexical entry for each word comes with a logogen unit Input such as [k] of “cat” activate the relevant units in parallel. The lexical entry becomes available once the logogen ‘fires’ (i.e., reaches threshold) INPUT [k] cat skip cot camp

Parallel Search Model Morton’s Logogen Model Threshold Analogy of logogen unit ‘firing’. INPUT /kaet/ cat skip cot camp

Parallel Search Model Morton’s Logogen Model ‘cat’ [kæt] ‘cot’ [kot] High frequency words have a lower threshold for firing, so it takes less to activate. lower threshold = shorter RT

Network connection of Logogens canary bird animal ostrich mammal yellow doctor dentist fever green baby cradle bed hospital sun rain heat grass nurse delirium

Spreading Activation cradle baby bed hospital nurse animal dentist “doctor” spread activation to “nurse” so “nurse” becomes semi-activated. NURSE now requires less to fire. baby bed hospital nurse animal dentist doctor mammal bird canary rain fever heat delirium sun ostrich green grass yellow

Parallel Search Models Evaluation Accounting for Data High frequency words have lower threshold Multiple Networks: Words related by… meaning are linked in a semantic network sound are linked in a phonological network How to deal with ability to judge something as a non-word? Impose a timer. After a certain time period, decide not a word Appeal - Efficiency Lexical access is in parallel

In-Class Discussion Time (Discussion of HW assignment) Design a program to recognize words in speech when given the phonetic transcription. Simple Case: One word Hard Case: string of multiple words (word segmentation problem)

Recognizing a single word The Simplest Idea Take the string of phonemes/syllables in order and match the string to find a word in our mental lexicon (In the same way we look up words in a dictionary – alphabet by alphabet.)

Cohort of Candidates S ... song story sparrow saunter slow secret sentry ... (i.e., words beginning w/ the sound heard so far) Slides adapted from P. Collins.

Cohort of Candidates SP spice spoke spare spin splendid spelling spread ... (i.e., words beginning w/ the sound heard so far)

Cohort of Candidates SPI spit spigot spill spiffy spinaker spirit spin ...

Segmenting a string of words (hw Q1B) THEREDONATEAKETTLEOFTENCHIPS THE RED ON A TEA KETTLE OFTEN CHIPS THERE, DON ATE A KETTLE OF TEN CHIPS PROCEDURE?

or Alignment Problem ThesKyisfalling! The sky is falling! This guy is falling!

Segmenting a string of words THEREDONATEAKETTLEOFTENCHIPS THE REDONATEAKETTLEOFTENCHIPS THE RED ONATEAKETTLEOFTENCHIPS THE RED ON ATEAKETTLEOFTENCHIPS THE RED ON A TEAKETTLEOFTENCHIPS … THE RED ON ATE AKETTLEOFTENCHIPS THE REDO NATEAKETTLEOFTENCHIPS THE REDONATE AKETTLEOFTENCHIPS THERE DONATEAKETTLEOFTENCHIPS ... (so on and so forth… MANY ALTERNATIVES TO GO THROUGH!!!) (Is this how our brains segments speech?)

Some Design Issues Depth first vs. Breadth first Search Pruning?

Outline for rest of today and next lecture We are fast at speech recognition. How do we achieve speed? Contextual Effects Terminologies and Ideas Two Classic Models Cohort Model TRACE Model [and many experimental paradigms and findings]

Speech Recognition is FAST Intuitively immediate. Comprehension happens around 200-300ms after hearing the word. Eye-tracking Studies Shadowing Studies Gating Studies Word Monitoring Studies

Eye-tracking Technology “Pick up the square”

Computer records scene plus eye fixation Eye camera Scene camera “Pick up the candle” http://www.psych.upenn.edu/trueswell_videos/adult1u.mov

Eye-tracking Technology Same Command: “Pick up the candle.” Two Different Scenes Competitor ABSENT Competitor PRESENT COMPETITOR of “candle” (phonologically) + + TARGET

Results Latency from onset of spoken target It takes 200ms to program an eye movement. So processing time here is about 200ms to 250ms for word comprehension. FAST!!!

Digression: Typical Eyetracking Data Presentation

Speech Recognition How do we achieve speed? But… Parallel search of words I.e. Activation of potential candidates in parallel But… Parallel search leads to many likely candidates to consider at the same time. If there are many possible likely candidates, how do we QUICKLY select the right candidate among the choices?

Speech Recognition CONTEXTUAL EFFECTS? Suppose you hear: Popeye loves to eat… Have you already guessed what the next word is going to be?

Shadowing Task Message entering participant’s ears. Participant repeat (speak) what s/he hears while listening – SHADOWING! The participant’s speech is recorded for analysis. Record Sound

Shadowing Task (Marslen-Wilson 1973) Message played into ears (160 words/min): Susan and Jill wanted… Tone synchronized with start of every word of message sent to recorder. Shadowed Speech Measure onset of each word shadowed relative to the tone.

Shadowing Task (Marslen-Wilson 1973) Susan and Jill wanted… Time unfolding… Message Played: Susan and Jill wanted… tone Susan and Jill wanted… Message Shadowed: onset time Measure onset of each word shadowed relative to the tone.

Shadowing Task (Marslen-Wilson 1973) Finding: About 10% of the population can shadow with 250ms delay. Most people shadow between 800ms-1000ms. Record Shadowed Message

Shadowing Task (Marslen-Wilson 1973) You might ask: Are they really retrieving words and processing what they heard that quickly? (1) Post-reading, participants were good at answering content questions. (2) Errors found were structurally appropriate: e.g. insertion of “that” From: It was beginning to be light enough so I could see… To: It was beginning to be light enough so that I could see… Record Shadowed Message

In-Class Shadowing Demo 2 Volunteers One to read a passage One to shadow a passage