PSY 369: Psycholinguistics Language Comprehension: Introduction & Perception of language

Announcements Homework 2 is due on Thursday Read: Samuel, A. (2001). Knowing a word affects the fundamental perception of the sounds within it. Psychological Science, 12(4) (you can download it from the assignment page on the course website) After you've read the article, write a summary of it. Make sure that you include the following in your summary: what is/are the issues what is/are the hypotheses what methodology was used what were the basic findings what conclusions were drawn

Different signals Visual word recognitionSpeech Perception Some parallel input Orthography Letters Clear delineation Difficult to learn Serial input Phonetics/Phonology Acoustic features Usually no delineation “Easy” to learn Where are you going

Different signals Visual word recognitionSpeech Perception Some parallel input Orthography Letters Clear delineation Difficult to learn Serial input Phonetics/Phonology Acoustic features Usually no delineation “Easy” to learn Where are you going

Why so much research using visual language? We do use it Easy to use in research The parts Letters Words Eye movements Visual perception of language

Same object category (‘G’) may have different shapes, sizes, and orientations G G G G G G G G G G G G G G G G G G G G G G Perhaps the brain is able to represent these objects in a way that is “translationally invariant” and “size invariant”. Invariance a problem in vision too?

Letter Recognition How do we recognize a group of lines and curves as letters? Two common explanations: Template matching Feature detection Okay, I’m going to show you a stimulus really fast and you need to tell me what it was

Template matching Store in brain a copy of what every possible input will look like. Match observed object to the proper image in memory

Perceptual Representation Memory Representations Template matching

Problems with Template matching Costly: Massive numbers of templates are required (remember all those E’s?).. Normalization before matching - ”mentally cleaning it up” before matching to templates Predicts no transfer to novel views of the same object Objects are often obstructed/occluded E FROG

Prolblems with Template matching E FROG Objects are often disamiguated by context Costly: Massive numbers of templates are required (remember all those E’s?).. Normalization before matching - ”mentally cleaning it up” before matching to templates Predicts no transfer to novel views of the same object Objects are often obstructed/occluded

Read aloud the following word CT

TE

So what is the middle letter? TE CT Clearly, top-down influences. However it is unclear how this works with template matching

Feature detection Analysis-by-synthesis 1. Letter broken down to its constituent parts 2. List of parts compared to patterns in memory 3. Best matching pattern chosen

A fixed set of elementary properties are analyzed Independently and in parallel across visual field. Possible examples Line Orientations: Different Sizes: Curvature: +45deg. -10deg. Free line endings: Colors: Feature detection

Perceptual Representation 3 Horizontal lines 1 Vertical line 4 Right angles Memory Representation 3 Horizontal lines 1 Vertical line 4 Right angles E F 2 Horizontal lines 1 Vertical line 3 Right angles A simple theory of Feature detection

Evidence for Features: The visual search task is straightforward, you are given some target to look for, and asked to simply decide, as quickly as possible, whether the target is present or absent in a set of objects. For example, let’s try a few searches to give you a feel for this. Search 1 - Is there an O present in the following displays?

Is an O present? T T O T T T

T T T T TTT T T T O TTT T TT TT T TT T T TT T T T TT T TTT T TT Is an O present?

Q Q Q O Q Q Q Q Is an O present?

Q QQ Q Q QQQ QQQ Q QQQ Q O Q Q QQ Q Q QQ Q Q Q QQQ Q QQQQ Q Is an O present?

T T O T T T T T T T TTT T T T O TTT T TT TT T TT T T TT T T T TT T TTT T TT Q Q Q O Q Q Q Q Q QQ Q Q QQQ QQQ Q QQQ Q O Q Q QQ Q Q QQ Q Q Q QQQ Q QQQQ Q

A theory of Feature detection Selfridge’s Pandemonium system, 1959

Another theory of Feature detection

Interactive Activation Model (AIM) McClelland and Rumelhart, (1981) Nodes: (visual) feature (positional) letter word detectors Inhibitory and excitatory connections between them. Previous models posed a bottom-up flow of information (from features to letters to words). IAM also poses a top-down flows of information

Inhibitory connections within levels If the first letter of a word is “a”, it isn’t “b” or “c” or … Inhibitory and excitatory connections between levels (bottom-up and top-down) If the first letter is “a” the word could be “apple” or “ant” or …., but not “book” or “church” or…… If there is growing evidence that the word is “apple” that evidence confirms that the first letter is “a”, and not “b”….. Interactive Activation Model (AIM)

+ Until the participant hits some start key The Word-Superiority Effect (Reicher, 1969)

COURSE Presented briefly … say 25 ms The Word-Superiority Effect (Reicher, 1969)

U &&&&& A Mask presented with alternatives above and below the target letter … participants must pick one as the letter they believe was presented in that position. The Word-Superiority Effect (Reicher, 1969)

+ E E & T + PLANE E &&&&& T + KLANE E &&&&& T Letter only Say 60% Letter in Nonword Say 65% Letter in Word Say 80% Why is identification better when a letter is presented in a word?

IAM & the word superiority effect We are processing at the word and letter levels simultaneously Letters in words benefit from bottom-up and top- down activation But letters alone receive only bottom-up activation.

Other Relevant Findings? Bias towards “well-formed” stimuli Bisidentify words with uncommon spelling patterns BOUT as BOAT misidentify nonwords (e.g., SALID) as words that are like it (SALAD). Difficulty identifying nonwords with irregular spelling patterns (e.g., ITPR) more than those with regular spelling patterns (e.g., PIRT).

Sublexical units bigger than phonemes and graphemes? onsets and rimes onset: initial consonant or consonant cluster in a word or syllable rime: following vowel and consonants if words broken at onset-rime boundary, resulting letter clusters more easily recognized as belonging together than if broken at other points example: FL OST ANK TR vs. FLA ST NK TRO Sublexical units

Adding a bigram level By adding a frequency-sensitive bigram level, we can account for the findings of well-formedness along with the others.

Summing up Word recognition is based on a feature-detector system Biased to perceive common or recently occurring features

Studying Word Identification Generally people ask: what makes word identification easy or difficult? The assumption: Time spent identifying a word can be a measure of difficulty Measures of identification time are usually indirect

Some Identification Time Measures Measure how long people take to say a string of letters is (or is not) a word (lexical decision) Measure how long people take to categorise a word (“apple” is a fruit) Measure how long people take to start saying a word (naming or pronunciation time) Measure how long people actually spend looking at a word when READING Line by line reading Word by word reading using eye movement monitoring techniques

Line-by-line A banker is a fellow

Line-by-line who lends you his umbrella

Line-by-line when the sun is shining

Line-by-line but wants it back

Line-by-line the minute it begins to rain.

Line-by-line Problem: Overall reading time for entire sentence or phrase need for more “on-line” measurements Timing on a smaller scope See effects at level of word

Word-by-word RSVP (rapid serial visual presentation)

Word-by-word A

lie

Word-by-word can

Word-by-word travel

Word-by-word halfway

Word-by-word around

Word-by-word the

Word-by-word world

Word-by-word while

Word-by-word the

Word-by-word truth

Word-by-word is

Word-by-word putting

Word-by-word on

Word-by-word its

Word-by-word shoes.

Word-by-word Moving window

Word-by-word I xxxx xxxxx xxx xx xxxxxxxxx xxxxxxxxx xxxx xx xxxxxxxxx.

Word-by-word x have xxxx xxx xx xxxxxxxxx xxxxxxxxx xxxx xx xxxxxxxxx.

Word-by-word x xxxx never xxx xx xxxxxxxxx xxxxxxxxx xxxx xx xxxxxxxxx.

Word-by-word x xxxx xxxxx let xx xxxxxxxxx xxxxxxxxx xxxx xx xxxxxxxxx.

Word-by-word x xxxx xxxxx xxx my xxxxxxxxx xxxxxxxxx xxxx xx xxxxxxxxx.

Word-by-word x xxxx xxxxx xxx xx schooling xxxxxxxxx xxxx xx xxxxxxxxx.

Word-by-word x xxxx xxxxx xxx xx xxxxxxxxx interfere xxxx xx xxxxxxxxx.

Word-by-word x xxxx xxxxx xxx xx xxxxxxxxx xxxxxxxxx with xx xxxxxxxxx.

Word-by-word x xxxx xxxxx xxx xx xxxxxxxxx xxxxxxxxx xxxx my xxxxxxxxx.

Word-by-word x xxxx xxxxx xxx xx xxxxxxxxx xxxxxxxxx xxxx xx education.

Word-by-word A couple of methods RSVP (rapid serial visual presentation) Moving window Better, more “on-line” But, these measures are also a little bit unnatural (especially RSVP) e.g., Don’t allow regressions (looking back)

Eye-movements Clothes make the man. Naked people have little or no influence on society.

Eye-movements Clothes make the man. Naked people have little or no influence on society.

Eye-movements Clothes make the man. Naked people have little or no influence on society.

Eye-movements Clothes make the man. Naked people have little or no influence on society.

Eye-movements Clothes make the man. Naked people have little or no influence on society.

Eye-movements Clothes make the man. Naked people have little or no influence on society.

Eye-movements Clothes make the man. Naked people have little or no influence on society.

Eye-movements Clothes make the man. Naked people have little or no influence on society.

Eye-movements Clothes make the man. Naked people have little or no influence on society.

Eye Movements Limitations of the visual field 130 degrees vertically 180 degrees horizontally Clothes make the man. Naked people have little or no influence on society.

Retinal Sampling

Eye Movements Within the visual field, eye movements serve two major functions Fixation – Position target objects of interest on the fovea Tracking – Keep fixated objects on the fovea despite movements of the object or head

Eye movements Fixations Movements Saccades Smooth Pursuit

Fixations The eye is (almost) still – perceptions are gathered during fixations The most important of eye “movements” 90% of the time the eye is fixated duration: 150ms - 600ms Consists of a series of small eye movements (tremor, drift, microsaccades)

Saccades Saccades are used to move the fovea to the next object/region of interest. Connect fixations Duration 10ms - 120ms Very fast (up to 700 degrees/second) No visual perception during saccades Vision is suppressed

Saccades

Saccades are used to move the fovea to the next object/region of interest. Connect fixations Duration 10ms - 120ms Very fast (up to 700 degrees/second) No visual perception during saccades Vision is suppressed Ballistic movements (pre-programmed) About 150,000 saccades per day

Smooth Pursuit Smooth movement of the eyes for visually tracking a moving object Cannot be performed in static scenes (fixation/saccade behavior instead)

Smooth Pursuit versus Saccades Saccades Jerky No correction Up to 900 degrees/sec Background is not blurred (saccadic suppression) Smooth pursuit Smooth and continuous Constantly corrected by visual feedback Up to 100 degrees/sec Background is blurred

Purkinje Eye Tracker Laser is aimed at the eye. Laser light is reflected by cornea and lens Pattern of reflected light is received by an array of light- sensitive elements. Very precise Also measures pupil accomodation No head movements Measuring Eye Movements

Video-Based Systems Infrared camera directed at eye Image processing hardware determines pupil position and size (and possibly corneal reflection) Good spatial precision (0.5 degrees) for head-mounted systems Good temporal resolution (up to 500 Hz) possible