Sensation & Perception Exam 3: Review © Takashi Yamauchi (Dept. of Psychology, Texas A&M University) 1

Sensation & Perception Ch. 7: Action © Takashi Yamauchi (Dept. of Psychology, Texas A&M University) Main topics Ecological approach Optic flow The physiology of navigation Skilled action Linking sensory and motor functions

Perception reconsidered J. J. Gibson’s ecological approach All the studies we have seen so far are conducted in laboratory settings, in which stationary stimuli were given. Perception is not a stationary experience. Perception should be studied as it occurs in the natural environment.

Optic flow and a gradient of flow A gradient of flow gives a strong cue for motion and depth perception

Demonstration: http://www.rdg.ac.uk/arl/clips/demo_of_displays.htm#opticflow

The Physiology of Navigation Optic flow neurons - neurons in the medial superior temporal area (MST) of monkeys respond to flow patterns

Neuron 1 in the monkey’s MST responds to an expanding stimulus but not a stimulus with circular movement. Neuron 2 responds to circular movement but not to expansion.

Physiological Links Between Sensory and Motor Functions Optic ataxia Patients suffering from this syndrome produce inaccurate reaching movements towards a target or object in space.

Experiment by Schindler et al. Participants:2 groups Patients with parietal lobe damage and normal control participants. Participants performed two tasks Bisection task - point to position between cylinders Reaching task - reach between cylinders and touch a gray strip Bisection task In each trial, you saw two cylinders that were shifted different positions. You had to point a place exactly midway between the cylinders. The results of this study suggest that the damage to the dorsal stream keeps such patients from being able to avoid obstacles. However, further research is needed to provide more evidence that this is the case.

Experiment by Schindler et al. Trial 1 Bisection task In each trial, you saw two cylinders that were shifted different positions. You had to point to a place exactly midway between the cylinders. So, for each trial, you had to point different positions. point Trial 2 The results of this study suggest that the damage to the dorsal stream keeps such patients from being able to avoid obstacles. However, further research is needed to provide more evidence that this is the case. point

Experiment by Schindler et al. Trial 1 Reaching task In each trial, you saw two cylinders that were shifted different positions. You had to touch a gray strip that were 20 cm behind the cylinders. So, for each trial, you had to extend your hand to different positions. touch Trial 2 The results of this study suggest that the damage to the dorsal stream keeps such patients from being able to avoid obstacles. However, further research is needed to provide more evidence that this is the case. touch

Results Pointing task Reaching task Trial 1 The patients with parietal lobe damage and normal control were equally accurate Reaching task The patients kept reaching the same place even though cylinders were shifted in each trial. touch Trial 2 touch

Mirror neurons

Mirror neurons Mirror neurons respond to a particular “kind” of action (e.g., grasping) AND observing someone doing the action. The activity of a particular neuron in the premotor area of a monkey.

Video clip from NOVA (PBS) 15 min http://www.pbs.org/wgbh/nova/sciencenow/3204/01.html

Sensation & Perception Ch. 8: Motion © Takashi Yamauchi (Dept. of Psychology, Texas A&M University) Main topics The functions of motion perception The direction of movement

4 different ways to create the perception of movement Real movement The object is physically moving Apparent movement Displacement of objects Induced movement Movement aftereffect

Functions of Movement Perception Survival in the environment Predators use movement of prey as a primary means to location in hunting Motion agnosia Damage to the cortex resulting in inability to perceive movement Extremely debilitating and dangerous for the patient Instructor can use the case of the woman in the text as an example of motion agnosia.

Functions of Movement Perception - continued Perceiving objects Movement of objects or the observer’s movement through objects help perceive the 3D organization of stimuli Kinetic depth effect - movement of an object’s 2-D shadow can change into perception of a 3-D object This is an example of structure-from-motion Perceptual organization

Figure 9.4 Setup similar to the one used by Wallach and O’Connell (1953) to demonstrate the kinetic depth effect.

Demonstration: structure-from-motion http://www.cs.ubc.ca/nest/imager/contributions/flinn/Illusions/Illusions.html

Visual Pathway (MT)

The Physiology of Navigation Optic flow neurons - neurons in the medial superior temporal area (MST) of monkeys respond to flow patterns

Neurons that are sensitive to a specific direction of motion. Inhibitory connections

The intelligence of movement perception Top-down processes Certain stimuli (like the human body) have special meaning that affects perception

This slide can be used to talk about the experimental evidence for biological motion. Figure 9.18 Frames from the stimuli used by Grossman and Blake (2001). (a) Sequence from the point-light walker stimulus. (b) Sequence from the scrambled point-light stimulus.

Biological Motion (Demonstration) http://www.biomotionlab.ca/

Depth and size Monocular depth cues Binocular depth cues

Monocular pictorial cues Occlusion Relative height Relative size Familiar size Atmospheric perspective Linear perspective

Leonardo da Vinci (1452-1519) Mona Lisa (1503) There has never been an artist who was more fittingly, and without qualification, described as a genius. Like Shakespeare, Leonardo were from an insignificant background and rose to universal acclaim. Leonardo was the illegitimate son of a local lawyer in the small town of Vinci in the Tuscan region…. Mona Lisa (1503)

The Dreyfus Madonna: da Vinci 1469

The Santa Trinita Madonna: Cimabue (1260/80)

Linear perspective Things get smaller when they are away.

Occlusion

Relative height

Relative size

Atmospheric perspective

Texture gradient

Two types of perspectives Linear perspective Atmospheric perspective

Belvedere: Escher

Binocular disparity The binocular disparity arises when a given point in the external world does not project to the corresponding points on the left and right retinae (Palmer, 1999, “Vision Science”)

Binocular disparity Your two eyes are getting different images  disparity  double images Corresponding points Corresponding points No disparity disparity

Small disparity Large disparity No disparity

Horopter (Vieth-Muller circle)

small distance large distance Small disparity large disparity

Sensation & Perception Ch. 11: Sound, The Auditory System, and Pitch Perception © Takashi Yamauchi (Dept. of Psychology, Texas A&M University) Main topics Sound stimuli, amplitude and frequency Sound quality (timbre) Ear (structure) Place theory Central auditory processing

Sound wave Figure 1: Graphic representations of a sound wave. (A) Air at equilibrium, in the absence of a sound wave; (B) compressions and rarefactions that constitute a sound wave; (C) transverse representation of the wave, showing amplitude (A) and wavelength (taken from Britannica Online)

A sound wave is determined by two factors Magnitude Y axis Frequency X axis

Specifying a sound stimulus Amplitude Y axis Decibel (dB) Number of dB = 20 x log(P/P0) (P: the sound pressure of the stimulus, P0: a standard pressure) P0:=the pressure of a 1000Hz tone at threshold. Frequency X axis Hertz (Hz)  one cycle per second

dB: Decibel dB? With p=200, p0(standard pressure level)=20 dB=20 x log (200/20)= 20 x log (10) = 20 x 1= 20 with p = 2000 dB = 20 x log (2000/20)= 20 x log (100) = 20x 2 = 40 With p = 20000 dB = 20 x log (20000/20)= 20 x log (1000) = 20x 3 = 60

create a complex sound by combining simple sound waves  Additive synthesis reduce a complex sound wave into a collection of simple sound waves.  Fourier analysis

A sound wave from clarinet. Simple sound waves that make a sound of clarinet

Figure 11. 9 The frequency spectrum for the tone in Figure 11. 8d Figure 11.9 The frequency spectrum for the tone in Figure 11.8d. The heights of the lines indicate the amplitude of each of the frequencies that make up the tone.

Ear

Ear

Bekesy’s place theory

The wave reached the peak at a particular location. A wave spreads. The wave reached the peak at a particular location. The height of the wave reaches the peak at P, and then gradually subsides.

Different locations of vibration peak are produced by different spatial frequencies.

Different frequencies of sound waves activate hair cells in different locations

A complex tone (440Hz, 880Hz, and 1320Hz).. The auditory system basically carry out a “Fourier analysis”  treat a complex sound as a composite of simple waves.

Sensation & Perception Ch. 12: Sound Localization © Takashi Yamauchi (Dept. of Psychology, Texas A&M University) Main topics Auditory localization Perceptual grouping ch 12

Identifying the sound source in the horizontal coordinate Interaural differences Interaural time differences Capture the difference in the time that a sound reaches the left and right ears Interaural level differences Capture the difference in the level of the sound intensity (sound pressure level) that a sound reaches the left and right ears ch 12

ch 12

Sensation & Perception Ch. 13: Speech Perception © Takashi Yamauchi (Dept. of Psychology, Texas A&M University) Main topics Phoneme Speech spectrogram The segmentation and variability problems Categorical perception Multimodal speech perception, Top-down process Speech perception and the brain

Some major problems in speech perception The segmentation problem How do we segment the individual words? The variability problem Variability from different speakers Variability from a phoneme’s context E.g., /d/ differs in “dig” “dug”

Stimuli Phonemes: The smallest unit of the sound system in a language consonants and vowels If you change a phoneme in a word, the meaning of the word is also changed. (e.g., hip  tip; cat  hat; dog  dig) English has about 40 different phonemes Every English word is produced by a combination of phonemes. Different languages have different phonemes E.g., Japanese does not distinguish r and l. E.g., rice / lice; election / erection

Some major problems in speech perception The segmentation problem How do we segment the individual words? The variability problem Variability from different speakers Variability from a phoneme’s context E.g., /d/ differs in “dig” “dug”

Question: How does the perceptual system handle these problems, and give us a coherent speech perception?

Using visual cues in speech perception The McGurk effect (demonstration) http://www.media.uio.no/personer/arntm/McGurk_english.html What am I saying? Play the clip several times, alternating between looking at the talking head while listening, and listening with your eyes shut. Most adults (98%) think they are hearing "DA" - a so called "fused respons" - where the "D" is a result of an audio-visual illusion. In reality you are hearing the sound "BA", while you are seing the lip movements "GA". The "McGurk effect" was first described by Harry McGurk and John MacDonald in "Hearing lips and seeing voices", Nature 264, 746-748 (1976).

Using visual cues in speech perception The McGurk effect (demonstration) http://www.youtube.com/watch?v=I1XWDOwH47Y http://www.youtube.com/watch?v=jtsfidRq2tw&NR=1 The "McGurk effect" was first described by Harry McGurk and John MacDonald in "Hearing lips and seeing voices", Nature 264, 746-748 (1976).

Speech perception and the brain Broca’s area Wernicke’s area

Speech Perception and the Brain Broca’s aphasia - individuals have damage in Broca’s area (in frontal lobe) Labored and stilted speech and short sentences but they understand others http://video.google.com/videoplay?docid=9178936581276081395&q=Broca%27s+aphasia&total=6&start=0&num=10&so=0&type=search&plindex=0 Wernicke’s aphasia - individuals have damage in Wernicke’s area (in temporal lobe) Speak fluently but the content is disorganized and not meaningful They also have difficulty understanding others http://video.google.com/videoplay?docid=7590914168187986085&q=Wernicke%27s+aphasia&total=3&start=0&num=10&so=0&type=search&plindex=0