Brain Mechanisms of Unconscious Inference J. McClelland Symsys 100 April 7, 2011

Last time… We considered many examples of unconscious inference –Size illusions –Illusory contours –Perception of objects from vague cues –Unconsious associative priming –Lexical effects on speech perception –Effect of visual speech on speech perception.

Today… We ask about the mechanisms through which this occurs

Three Problems for Unconscious Inference Theory GARY HATFIELD, Philosopher, Univ. of Penna. The cognitive machinery problem: Are the unconscious inferences posited to explain size perception and other phenomena carried out by the same cognitive mechanisms that account for conscious and deliberate inferences, or does the visual system have its own inferential machinery? In either case, what is the structure of the posited mechanisms? The sophisticated content problem: how shall we describe the content of the premises and conclusions? For instance, in size perception it might be that the premises include values for visual angle and perceived distance […]. But shall we literally attribute concepts of visual angle […] to the visual system? The phenomenal experience problem: Third, to be fully explanatory, unconscious inference theories of perception must explain how the conclusion of an inference about size and distance leads to the experience of an object as having a certain size and being at a certain distance. In other words, the theories need to explain how the conclusion to an inference […] can be or can cause perceptual experience.

Proposed answers to these questions The cognitive machinery problem. The machinery of unconscious inference is the propagation of activation among neurons. Neurons embedded in the perceptual system can carry out such inferences without engaging the mechanisms used in conscious and deliberative inference. The sophisticated content problem. Activation of particular neurons or groups of neurons codes for particular content. Connections among neurons code the conditional relationships between items of content. The phenomenal experience problem. Activity of certain populations of neurons is a necessary condition for conscious experience. Anything that affects the activation of the neurons will affect conscious experience. Is this activity the actual substrate of experience itself?

Outline of Lecture Neurons: Structure and Physiology Neurons and The Content of Experience How Neurons Make Inferences –And how these capture features of Bayes Rule Integration of Information in Neurons and in Perception

Neuronal Structure and Function Neurons combine excitatory and inhibitory signals obtained from other neurons. They signal to other neurons primarily via ‘spikes’ or action potentials.

Neurons and the Content of Experience The doctrine of specific nerve energies: –Activity of specific neurons corresponds to specific sensory experiences Touch at a certain point on the skin Light at a certain point in the visual field The brain contains many ‘maps’ in which neurons correspond to specific points On the skin In the visual world In non-spatial dimensions such as auditory frequency If one stimulates these neurons in a conscious individual, an appropriate sensation is aroused. If these neurons are destroyed, a corresponding void in experience occurs. –Visual ‘scotomas’ arise from lesions to the maps in primary visual cortex.

Feature Detectors in Visual Cortex Line and edge detectors in primary visual cortex (classic figure at left). Representation of motion in area MT. –Neurons in MT fire when the visual input in a region of space is moving in a certain direction. –Destroy MT on one side of the brain, and perception of motion in the opposite side of space is greatly impaired. –Stimulate neurons in MT, and a monkey will report motion in the neurons’ preferred direction.

Neural Representations of Objects and their Identify The ‘Grandmother Cell’ controversy –Is there a dedicated neuron, or set of dedicated neurons, for each cognized object, such as my Grandmother? –Most researchers argue that objects are represented by patterns of activation over a set of neurons. –Some cells have surprisingly specific responses, and some authors treat them as dedicated to specific concepts. –We will adopt the grandmother neuron hypothesis today, for simplicity, and proceed as though single neurons stand for hypotheses at all levels of generality. –Next time we will think in terms of the pattern-based approach.

Stimuli used by Baylis, Rolls, and Leonard (1991)

Responses of Four Neurons to Face and Non-Face Stimuli in Previous Slide

Responses to various stimuli by a neuron responding to a Tabby Cat (Tanaka et al, 1991)

The Infamous ‘Jennifer Aniston’ Neuron

A ‘Halle Barry’ Neuron

Outline of Lecture Neurons: Structure and Physiology Neurons and The Content of Experience  How Neurons Make Inferences  And how these capture features of Bayes Rule Integration of Information in Neurons and in Perception

The Key Idea Think of a neuron as corresponding to a hypothesis. Furthermore, suppose: The excitatory inputs to a neuron correspond to evidence that supports the hypothesis. The inhibitory inputs correspond to evidence that goes against the hypothesis. There is a background input or ‘bias’ that reflects the prior probability of the hypothesis for which the neuron stands. All elements of the evidence are conditionally independent given H. THEN the resulting activation of the neuron can represent the posterior probability of the hypothesis given the evidence. Neuron i Input from neuron j w ij

Unpacking this idea It is common to consider a neuron to have an activation value corresponding to its instantaneous firing rate or p(spike) per unit time. The baseline firing rate of the neuron is thought to depend on a constant background input called its ‘bias’. When other neurons are active, their influences are combined with the bias to yield a quantity called the ‘net input’. The influence of a neuron j on another neuron i depends on the activation of j and the weight or strength of the connection to i from j. Note that connection weights can be positive (excitatory) or negative (inhibitory). These influences are summed to determine the net input to neuron i: net i = bias i +  j a j w ij where a j is the activation of neuron j, and w ij is the strength of the connection to unit i from unit j. Note that j ranges over all of the units that have connections to neuron i. Neuron i Input from neuron j w ij

How an Idealized Neuron’s Activation can Reflect P(H|E) Let us suppose that the activation of neuron i given its net input net i is assumed to be given by: a i = exp(net i ) 1 + exp(net i ) This function is called the ‘logistic function’ (graphed at right) Under this activation function: a i = P(H i |E) if a j = 1 when E j is present, 0 when E j is absent; w ij = log(P(E j |H)/P(E j |~H))= j bias i = log(P(H)/P(~H))= In short, idealized neurons using the logistic activation function can compute the probability of the hypothesis they stand for, given the evidence represented in their inputs, if their weights and biases have the appropriate values, and the elements of the evidence are conditionally independent given H. aiai net i

Math Supporting Above Statements Bayes Rule with two conditionally independent sources of information: Divide through by: And let: We obtain: This is equivalent to: And more generally, when {E} consists of multiple conditionally independent elements E j : Logarithmatic:

Modeling the McGurk Effect Visual input is one source of input to a ‘da’ detector neuron. Auditory input is another source of input. Simulation of the pattern of data is shown next to actual data. /BA/ /DA/ Probability /DA/ Identification Auditory

Choosing between N alternatives Often we are interested in cases where there are several alternative hypotheses (e.g., different directions of motion of a field of dots). Here we have a situation in which the alternatives to a given H, say H 1, are the other hypotheses, H 2, H 3, etc. In this case, the probability of a particular hypothesis given the evidence becomes: P(H i |E) = p(E|H i )p(H i )  i’ p(E|H i’ )p(H i’ ) The normalization implied here can be performed by an idealized neural circuit that computes net inputs as before but now setting each unit’s activation according to: a i = exp(net i )  i’ exp(net i’ ) This normalization effect is approximated by lateral inhibition mediated by inhibitory interneurons (shaded unit in illustration). H E

Outline of Lecture Neurons: Structure and Physiology Neurons and The Content of Experience How Neurons Make Inferences And how these capture features of Bayes Rule  Integration of Information in Neurons and in Perception

‘Cue’ Integration in Monkeys Saltzman and Newsome (1994) combined two cues to the perception of motion: –Partially coherent motion in a specific direction –Direct electrical stimulation They measured the probability of choosing each direction with and without stimulation at different levels of coherence. Effect of electrical stimulation is absent if visual motion is very strong, but is considerable if the motion stimulation is weak or moderate.

Model used by S&N: S&N applied the model we have been discussing: P i = exp(net i )/  i’ exp(net i’ ) Where P i represents probability of responding in direction i net i = bias i + w ie e +w ij v j w ie = effect of microstimulation on neurons representing percept of motion in direction i e = 1 if stimulation was applied, 0 otherwise W ij = effect of visual stimulation in direction j v j = strength of motion in direction j Visual Input Electrical Input

Evidence for the Model When the electrical stimulation favors motion in a different direction from the stimulus, the perceived direction of motion is not a weighted average of the two directions. Instead it correctly reflects how different sources of evidence should combine according to the model (right). Open circles above show effect of presenting visual stimulation at 90 o (using an intermediate coherence level) together with electrical stimulation favoring the 225 o position. Dip between peaks rules out simple averaging of the directions cued by visual and electrical stimulation but is consistent with model predictions (filled circles).

Summary: The Mechanism of Unconscious Perceptual Inference Neurons (or populations of neurons) can represent perceptual hypotheses at different levels of abstraction and specificity Connections among neurons can code conditional relations among hypotheses. –Excitation and Inhibition code p(E|H)/p(E|~H) –Lateral inhibition codes mutual exclusivity Propagation of activation produces results corresponding approximately to Bayesian inference. The resulting activity incorporates inferential processes that may alter our phenomenal experience.