1. Neural Correlates of Shape from Shading
Mamassian, Jentzsch, Bacon, Schweinberger
NeuroReport, 2003 Or
Hou, Pettet, Vildavski, Norcia
Journal of Vision, 2006
Brian Potetz 2/15/06

2. Neural Correlates of Shape from Shading Mamassian, Jentzsch, Bacon, Schweinberger NeuroReport, 2003
Human observers are biased towards perceiving light coming from above-left.
Where in the brain is this prior represented?
How quickly can this prior be observed in neural signals?

3. Stimulus

4. Thin stripes lit from above, thick from below

5. Thin stripes lit from the left, thick from the right

6. Thin stripes lit from below, thick from above

7. Thin stripes lit from the right, thick from the left

8. Stimulus Repeated for 16 orientations
2 phases (responses averaged together due to similarity)

9. Results
Subjects prefer above-left lighting percepts (13.8 bias)

10. Results Subjects prefer above-left lighting percepts (13.8 bias)
N2 ( ms) VEP signal resembled “narrow score” (~26 bias)

11. No Controls for Low-Level Cues
Perceived shape is not the only property that changes with stimulus orientation.
Many low-level image cues could have similar response profiles.

12. VEP does not vary according to stimulus orientation until ~232ms.

13. Behavioral Response vs Early VEP
Authors select ambiguous orientations (90, 105 ), (270, 285 )
Divide trials according to perceived 3D shape (narrow or thick strips)
Using ANOVA, they find an interaction between behavioral response and early-response (96-104ms) VEP signal.
This interaction is found for “all lateral electrode sites”

14. Some missing statistics
The authors use ANOVA to find an interaction between behavioral response and early-response VEP signal at some recording sites.
The authors claim: The interaction of the P1 amplitude with the participants’ response indicates that the shape was disambiguated within the first 100ms of the stimulus presentation.
Was this “interaction” in the same direction as the interaction between VEP and orientation?
Based on Fig. 2, we would expect VEP to be higher when narrow strips are perceived. Are they? (Not always, as we will see)
How strong was this effect? Could I accurately predict participants’ responses based on early VEP?
Even if VEP were strongly2 correlated1 with behavioral response, there are multiple possible conclusions. Differences in VEP might be due to variations in a neural signal that encodes the perceived lighting direction (as the authors suggest), or priors on lighting direction. Or, the VEP may merely reflect a signal that is largely unrelated to SFS unless the stim is completely bistable, so that the perceived shape is effectively a toss-up. Could VEP have been correlated with response even before the stim was presented?

15. Hemispheric Difference
Relationship between VEP and behavioral response depends on hemisphere.
Extent of that deviation is correlated with subject’s preferred lighting direction (R = 0.83).

16. A Bottom-Up Mechanism for Shape from Shading?
Authors argue that these results are evidence that “shape from shading is mostly a bottom-up mechanism”.
However, static priors like p(L) are not the only source of contextual information.
It makes sense for static priors to be encoded in low-level visual areas.
But dynamic, contextual priors change frequently, and may need to be represented in higher cortical areas.
Examples: I can see where the light-source is. I was told where it is, I’ve been here before, etc. P(L|context)

17. A Bottom-Up Mechanism for Shape from Shading?

18. A Bottom-Up Mechanism for Shape from Shading?

19. A Bottom-Up Mechanism for Shape from Shading?
Also, ambiguity in lighting direction is not the only potential source of ambiguity.
Even under known illumination conditions, solving shape from shading in natural images is a difficult, unsolved problem.
Harder problems may require more top-down cues.
Ambiguous, but computationally simple once a light source direction is chosen.
A harder problem, may benefit from contextual information (recognizing the material as fabric, etc)

20. A Bottom-Up Mechanism for Shape from Shading?
Also, ambiguity in lighting direction is not the only potential source of ambiguity.
Even under known illumination conditions, solving shape from shading in natural images is a difficult, unsolved problem.
Harder problems may require more top-down cues.
Disambiguating surface markings and shadow from shading variations is even more difficult, and can benefit strongly from contextual cues.
In this image, our perception is aided by mid-level context, like the recognition that the object is dirty, tarnished metal
and also high-level contextual cues, like the recognition of the object as a coin, and the figure as a face.

21. Neural Correlates of Shape from Shading Hou, Pettet, Vildavski, Norcia Journal of Vision, 2006

22. Switching from 3D!2D stim results in a more flat response than switching from 2D!3D

23. Controlling for low-level cues

24. Controlling for low-level cues
Grey line: 3D on/off percept
Black line: 3D lateral motion