13. What made you respond face (or word)?
Something in your brain made you decide face or word.
Can we determine where this decision is made?
Related domain: Motion Direction Discrimination and area MT

14. What determines the percept and the response?
Observe a correlation between motion direction judgements and activity of cells in area MT. (Britten et al. 1996, Gold & Shadlen, 2000)
If neurons that responded to leftward motion were highly active, the monkey chose 'left' as the decision.
From Schall, 1999

15. N170 - distinct negative potential at 170 ms post-stimulus onset
fairly early in processing
This occurs at T6; smaller effect at T5
T = temporal lobe
5 = left
6 = right
right hemisphere specialization

16. Evoked Response Potential (ERP) and Face Stimuli
N170: negative-going potential at 170 ms
Largest over the right parietal lobe, also on the left parietal lobe.
N170 - distinct negative potential at 170 ms post-stimulus onset
fairly early in processing
This occurs at T6; smaller effect at T5
T = temporal lobe
5 = left
6 = right
right hemisphere specialization
From Tanaka and Curran (2001)

17. N170 Properties: Faces produce the largest amplitude.
Strong evidence of expertise: Bird experts have larger N170's to pictures of birds than pictures of dogs. Dog experts show the reverse. (Tanaka & Curran 2001).
Mainly perceptually based: prior exposure of a face does not produce large changes in the N170 for subsequent presentations (Rik Henson, AIC 2003).
Scalp distribution and latency suggest that the N170 component reflects the perceptual processing of complex visual stimuli.
fMRI studies show IT is active whether attending to faces or not (Tarr)
No effect of familiarity (Bentin & Deouell, 2000)
target status: Is the face-sensitive N170 the only ERP not affected by selective attention? (Caquil, Edmonds, & Taylor, 2000)
Appears to be feed-forward perceptual processing of faces or other face-like stimuli

18. A Thought Question:
What was going on in your perceptual regions when you thought you saw a face or a word?
Could we capture the current state (at least indirectly) with the N170 component?
Would the N170 be larger when you thought you saw a face?
fMRI studies show IT is active whether attending to faces or not (Tarr)
No effect of familiarity (Bentin & Deouell, 2000)
target status: Is the face-sensitive N170 the only ERP not affected by selective attention? (Caquil, Edmonds, & Taylor, 2000)
Appears to be feed-forward perceptual processing of faces or other face-like stimuli

19. Central Question:
Can we relate the size of the N170 to the response in the noise-alone condition?
Will it be larger when subjects think they see a face?
fMRI studies show IT is active whether attending to faces or not (Tarr)
No effect of familiarity (Bentin & Deouell, 2000)
target status: Is the face-sensitive N170 the only ERP not affected by selective attention? (Caquil, Edmonds, & Taylor, 2000)
Appears to be feed-forward perceptual processing of faces or other face-like stimuli

20. An Experiment Show Faces and Words Embedded in Noise: High Contrast
Low Contrast
Faces
Low Contrast
Words
High Contrast Words
Noise Alone

21. Methods
Face, word and noise-alone trials were presented in random order.
Ten naïve participants
120 trials per condition per subject
On each trial: Did you see a face or a word?
Subjects were told that a stimulus appeared on each trial, and that they should guess if they were unsure.
Participants had to indicate whether there was a face or a word present
Participants were told that a very low contrast stimulus was present in zero-contrast (i.e., noise alone) condition

22. EEG Recording Sites
T5 - Left
T6 - Right*

23. Methods continued
Analyze the data according to the subject’s responses on the noise-alone trials.
Incredibly important point: The noise was the same across ALL trials and stimuli. Physically the same. Not just identically distributed, but identical. There was only one noise field for the entire experiment.
Together, these procedures hold the physical stimulus constant on noise-alone trials.
Participants had to indicate whether there was a face or a word present
Participants were told that a very low contrast stimulus was present in zero-contrast (i.e., noise alone) condition

24. Amplitude (V) ***RIGHT***
Two-tailed t-tests - average amplitude for each observer in window N ms
T6: t(9) = 2.74, p = .023
no significant results for P100 ( ms) or
P300 ( ms)
no significant results for other sites

25. Central Question:
Will we see a larger N170 to the noise-alone stimulus when subjects think they see a face as opposed to a word?
T5 - Left
T6 - Right*

26. * Amplitude (V) ***RIGHT*** * t(9) = 2.74, p = .023, two-tailed
Two-tailed t-tests - average amplitude for each observer in window N ms
T6: t(9) = 2.74, p = .023
no significant results for P100 ( ms) or
P300 ( ms)
no significant results for other sites

27. Main Result:
On noise-alone trials: Larger N170 when observers report seeing a face than when report seeing a word.
Occurs in 9 of the 10 subjects.
No other differences in any other channel at the P100, N170 or P300 components.
Unlikely to just reflect activity for an already-made decision.
Relates activity in the perceptual processing areas to the behavioral response. Greater activity in the N170 neurons is associated with ‘face’ responses to the noise-alone stimulus.
One interpretation: Greater activity in the face processing region biases the response towards a ‘face’ response.
Inside-out gets at the idea that the internal response might be higher for whatever reason on that trial, and this leads observer to think saw a face, and to give an overt 'face' response.
Note that our data is just correlational; it could be that the prior trial could have primed our N170 neurons, and that we were thinking about faces for other reasons (maybe frontal cortex was primed as well), and this leads to the correlation in the absence of causality.
It might not be that the greater N170 is b/c they saw a face in the noise, but rather seeing a face on the previous trial primed them.
In other words, do observers have larger N170 on present trial if a face was shown on the previous trial?

28. Alternative Explanations for this Greater Activity
Attention to different spatial frequencies or face-like features in the noise
Unlikely to see rapid changes in spatial frequency tuning in a mixed design.
No P100 differences that might be associated with changes in activity in different spatial frequency channels.
One obvious exception: Prior trial priming
Seeing a face on the previous trial may leave residual activity in the face neurons or make subjects look for face-like features in the noise.
Inside-out gets at the idea that the internal response might be higher for whatever reason on that trial, and this leads observer to think saw a face, and to give an overt 'face' response.
Note that our data is just correlational; it could be that the prior trial could have primed our N170 neurons, and that we were thinking about faces for other reasons (maybe frontal cortex was primed as well), and this leads to the correlation in the absence of causality.
It might not be that the greater N170 is b/c they saw a face in the noise, but rather seeing a face on the previous trial primed them.
In other words, do observers have larger N170 on present trial if a face was shown on the previous trial?

29. Prior Trial Priming Amplitude (V)
Face response not influenced by previous trial
Word response IS influenced by previous trial, but in wrong direction for positive priming
Point out how to recover data reported earlier - priming effect is only small part of what we found

30. Third Possibility: Stochastic Activity
In the domain of binocular rivalry, Blake and Logothetis introduced the idea of a process that involved stochastic activity in perceptual regions, which could bias the response toward one percept or the other at different points in time. (Blake & Logothetis 2002).
A similar process could be at work in the face processing neurons:
When activity is high in the N170 neurons due to random stochastic fluctuations, the observer may be biased to respond ‘face’ on that trial.
Inside-out gets at the idea that the internal response might be higher for whatever reason on that trial, and this leads observer to think saw a face, and to give an overt 'face' response.
Note that our data is just correlational; it could be that the prior trial could have primed our N170 neurons, and that we were thinking about faces for other reasons (maybe frontal cortex was primed as well), and this leads to the correlation in the absence of causality.
It might not be that the greater N170 is b/c they saw a face in the noise, but rather seeing a face on the previous trial primed them.
In other words, do observers have larger N170 on present trial if a face was shown on the previous trial?

31. Implications for Internal Noise
Internal noise does not just limit performance or decrease calculation efficiency, but also operates in feature space to bias the response toward one alternative or another.
Manipulations such as varying the power of the noise or the stimulus pairs being compared may help constrain models of internal noise.
Supports trial-by-trial variability in parameters:
Starting point and drift-rate variability in Ratcliff’s Diffusion model.
These sources of variability take on a perceptually-based interpretation. Link with perceptual brain areas.
Inside-out gets at the idea that the internal response might be higher for whatever reason on that trial, and this leads observer to think saw a face, and to give an overt 'face' response.
Note that our data is just correlational; it could be that the prior trial could have primed our N170 neurons, and that we were thinking about faces for other reasons (maybe frontal cortex was primed as well), and this leads to the correlation in the absence of causality.
It might not be that the greater N170 is b/c they saw a face in the noise, but rather seeing a face on the previous trial primed them.
In other words, do observers have larger N170 on present trial if a face was shown on the previous trial?

32. For Noise-Alone Trials:
The Important Stuff:
For Noise-Alone Trials:
* t(9) = 2.74,
p = .023
High Contrast
Faces
High Contrast Words
Low Contrast
Words
Noise Alone
Amplitude (V)

33. ERP and Faces
Intracranial recordings reveal N200 at sites in IT/fusiform gyrus
From Allison, Puce, Spencer, and McCarthy (1999)

34. Which Stimuli Evoke an N170/N200?
Any face or face-like
visual stimulus
Face-specific N170/N200:
Any face, regardless of sex, age, emotional expression, viewpoint
High- and low-pass filtered faces
Parts of faces; internal and external features
Cartoon faces
No face-specific N170/N200:
butterflies, flowers, etc.
words*
scrambled faces
hands

35. N170 Properties: Top-down Influences
Could be just a face-detection system (Bentin, et. al. 1996)
No effect of task demands such as selective attention to faces vs. objects. (Caquil, Edmonds, & Taylor, 2000)
No effect of familiarity of the face (Bentin & Deouell, 2000)
fMRI - IT active anytime there’s a face. (Gauthier et al. 1999)
However, IT also active while imagining a face. (O’Craven & Kanwisher 2000)
fMRI studies show IT is active whether attending to faces or not (Tarr)
No effect of familiarity (Bentin & Deouell, 2000)
target status: Is the face-sensitive N170 the only ERP not affected by selective attention? (Caquil, Edmonds, & Taylor, 2000)
Appears to be feed-forward perceptual processing of faces or other face-like stimuli

36. Contextual Influences On N170
Bentin et al (2002)
Block 1:
Stimulus Set A
Block 2:
Set B (experimental)
Set C (control)
Set D (all; targets)
Block 3:
Set A again (all)
Task: count schematic flowers

37. Contextual Influences On N170
Bentin et al (2002)
Face context elicits N170 to schematic eyes
Once the stimulus has been interpreted as containing face-like features, a stronger N170 is produced.
Experimental and control group showed no N170 response to schematic eyes alone before priming
Experimental showed N170 response to schematic eyes alone in Block 3 (after priming)
Control group showed no N170 response to schematic eyes alone in Block 3
*We wanted to further explore non-perceptual influences

38. N170 Summary
Represents activity of face-selective neurons most likely in area IT.
Magnitude varies with the degree to which physiognomic information is perceived in the image.
Is the N170 related to the response in the noise-alone condition?
If so, tie the activity in face-selective cells to the percept and the response.

39. Would we get an N170 to the noise-alone trials at all?
Previous work showed only a weak N170 to random noise stimuli.
Pilot study: Blocked presentations of faces and words.
Result: when looking for faces, get a larger N170 to noise-alone trials than when looking for words.
Replicates Bentin et al., extends to images that contain no facial features. We see top-down effects on the N170 even with no face-like features.

40. Face Condition : Male or female?
Task : Is the face male or female?
Have hi-, low-, and no-contrast (i.e., noise alone) condition
Told subjects there was actually a very low-contrast stimulus in the no-contrast condition

41. Word condition: Honesty or Trust?
IMPORTANT POINT:
NOISE STIMULI IDENTICAL FOR BOTH FACE AND WORD TRIALS

42. Experiment 1 Results - T5 Grand Average for T5 High-Contrast Face8 6 4
High-Contrast Face
Low-Contrast Face
Zero-Contrast Face
High-Contrast Word
Low-Contrast Word
Zero-Contrast Word 2
Voltage (microvolts) -2
Big mess - just look at crucial condition -4 -6 -8
-10
-12
-100
100
200
300
400
500
600
Time (ms)

43. Experiment 1 Results - T5 Zero-Contrast Word Zero-Contrast Face LEFT
Grand Average for T5
Zero-Contrast Word
Zero-Contrast Face 6 * 5 4
*t(8) = 2.62, p = .031 3
Voltage (microvolts) 2 1
LEFT
Two-tailed t-tests - average amplitude for each observer in window from ms
T5: t(8) = 2.62, p = .031
no significant results for P100 ( ms) or
P300 ( ms)
N170 significant for all sites, but difficult to interpret -1 -2 -3
-100
100
200
300
400
500
600
Time (ms)

44. Neural Processing of Face Stimuli
Cells in inferotemporal cortex (IT) are known to respond selectively to faces
(single cell recording, fMRI)
Slight right hemisphere dominance

45. Experiment 1 Results - T6 Zero-Contrast Word Zero-Contrast Face RIGHT
Grand Average for T6
Zero-Contrast Word
Zero-Contrast Face 5 * 4 3
* t(8) = 2.35, p = .047 2
Voltage (microvolts) 1
RIGHT
Two-tailed t-tests - average amplitude for each observer in window from ms
T6: t(8) = 2.35, p = .047
no significant results for P100 ( ms) or
P300 ( ms)
N170 significant for all sites, but difficult to interpret -1 -2 -3
-100
100
200
300
400
500
600
Time (ms)

46. Experiment 1 Conclusion
When an observer expects a face (versus a word)
there is a greater N170
When an observer expects, or is looking for, a face (versus a word) there is a greater N170