Neural Correlates of Shape from Shading Mamassian, Jentzsch, Bacon, Schweinberger NeuroReport, 2003 Or Hou, Pettet, Vildavski, Norcia Journal of Vision, 2006 Brian Potetz 2/15/06 http://www.cnbc.cmu.edu/cns

Neural Correlates of Shape from Shading Mamassian, Jentzsch, Bacon, Schweinberger NeuroReport, 2003 Human observers are biased towards perceiving light coming from above-left. Where in the brain is this prior represented? How quickly can this prior be observed in neural signals?

Stimulus

Thin stripes lit from above, thick from below

Thin stripes lit from the left, thick from the right

Thin stripes lit from below, thick from above

Thin stripes lit from the right, thick from the left

Stimulus Repeated for 16 orientations 2 phases (responses averaged together due to similarity)

Results Subjects prefer above-left lighting percepts (13.8 bias)

Results Subjects prefer above-left lighting percepts (13.8 bias) N2 (280-300ms) VEP signal resembled “narrow score” (~26 bias)

No Controls for Low-Level Cues Perceived shape is not the only property that changes with stimulus orientation. Many low-level image cues could have similar response profiles.

VEP does not vary according to stimulus orientation until ~232ms.

Behavioral Response vs Early VEP Authors select ambiguous orientations (90, 105 ), (270, 285 ) Divide trials according to perceived 3D shape (narrow or thick strips) Using ANOVA, they find an interaction between behavioral response and early-response (96-104ms) VEP signal. This interaction is found for “all lateral electrode sites”

Some missing statistics The authors use ANOVA to find an interaction between behavioral response and early-response VEP signal at some recording sites. The authors claim: The interaction of the P1 amplitude with the participants’ response indicates that the shape was disambiguated within the first 100ms of the stimulus presentation. Was this “interaction” in the same direction as the interaction between VEP and orientation? Based on Fig. 2, we would expect VEP to be higher when narrow strips are perceived. Are they? (Not always, as we will see) How strong was this effect? Could I accurately predict participants’ responses based on early VEP? Even if VEP were strongly2 correlated1 with behavioral response, there are multiple possible conclusions. Differences in VEP might be due to variations in a neural signal that encodes the perceived lighting direction (as the authors suggest), or priors on lighting direction. Or, the VEP may merely reflect a signal that is largely unrelated to SFS unless the stim is completely bistable, so that the perceived shape is effectively a toss-up. Could VEP have been correlated with response even before the stim was presented?

Hemispheric Difference Relationship between VEP and behavioral response depends on hemisphere. Extent of that deviation is correlated with subject’s preferred lighting direction (R = 0.83).

A Bottom-Up Mechanism for Shape from Shading? Authors argue that these results are evidence that “shape from shading is mostly a bottom-up mechanism”. However, static priors like p(L) are not the only source of contextual information. It makes sense for static priors to be encoded in low-level visual areas. But dynamic, contextual priors change frequently, and may need to be represented in higher cortical areas. Examples: I can see where the light-source is. I was told where it is, I’ve been here before, etc. P(L|context)

A Bottom-Up Mechanism for Shape from Shading?

A Bottom-Up Mechanism for Shape from Shading?

A Bottom-Up Mechanism for Shape from Shading? Also, ambiguity in lighting direction is not the only potential source of ambiguity. Even under known illumination conditions, solving shape from shading in natural images is a difficult, unsolved problem. Harder problems may require more top-down cues. Ambiguous, but computationally simple once a light source direction is chosen. A harder problem, may benefit from contextual information (recognizing the material as fabric, etc)

A Bottom-Up Mechanism for Shape from Shading? Also, ambiguity in lighting direction is not the only potential source of ambiguity. Even under known illumination conditions, solving shape from shading in natural images is a difficult, unsolved problem. Harder problems may require more top-down cues. Disambiguating surface markings and shadow from shading variations is even more difficult, and can benefit strongly from contextual cues. In this image, our perception is aided by mid-level context, like the recognition that the object is dirty, tarnished metal and also high-level contextual cues, like the recognition of the object as a coin, and the figure as a face.

Neural Correlates of Shape from Shading Hou, Pettet, Vildavski, Norcia Journal of Vision, 2006

Switching from 3D!2D stim results in a more flat response than switching from 2D!3D

Controlling for low-level cues

Controlling for low-level cues Grey line: 3D on/off percept Black line: 3D lateral motion