1. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector 1 Laurent Itti: CS564 - Brain Theory and Artificial Intelligence Lecture 7. Didday Model of Winner-Take-All Reading Assignments: TMB2: 4.3, pp. 194-197. Prey Selection - or Winner Takes All 4.4. A Mathematical Analysis of Neural Competition

2. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector 2 The Prey-Selector Model of Didday Consider how the frog's brain might select one of several visually presented prey objects. The task: to design a distributed network (not a serial scan strategy) that could take a position-tagged "foodness array" and ensure that usually the strongest region of activity would influence the motor control system. The prey-selector: an early, biological example of what connectionists call a winner-take-all network. Didday’s model adds a relative foodness layer of cells in topographic correspondence to the retinotopic foodness layer. The new layer yields the input to the motor circuitry.

3. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector 3 Winner-take-all Networks Goal: given an array of inputs, enhance the strongest (or strongest few) and suppress the others No clear strong input yields global suppression Strongest input is enhanced and suppresses other inputs

4. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector 4 Winner-Take-All Networks Basic architecture: - topographically organized layer of neurons - each neuron receives excitatory input (e.g., sensory) - each neuron also sends inhibition to neighboring neurons, possibly via an inhibitory inter-neuron a b Example of 2-neuron WTA with spiking neurons strong stimulus weak stimulus

5. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector 5 Didday’s Model = copy of input = inhibitory inter-neurons = receives excitation from foodness layer and inhibition from S-cells retinotopic input

6. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector 6 Didday's transformation scheme from foodness to relative-foodness It uses a population of S “sameness“ cells in topographic correspondence with the other layers. Each S-cell inhibits the activity that cells in its region of the relative- foodness layer receive from the corresponding cells in the foodness layer by an amount that augments with increasing activity outside its particular region. This ensures that high activity in a region of the foodness layer penetrates only if the surrounding areas do not contain sufficiently high activity to block it.

7. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector 7 Hysteresis How does the network respond to a change in the maintained stimulus pattern? Once in equilibrium, one may increase a non-maximal stimulus s 2 so that it becomes larger than the previously largest stimulus s 1, yet not switch activity to the corresponding element. In neural networks with loops - an internal state resists dependence on input: buildup of excitation and inhibition precludes the system's quick response to new stimuli. For example, if one of two very active regions were to suddenly become more active, then the deadlock should be broken quickly. In the network so far described, however, the new activity cannot easily break through the excitation of its competitor and the global inhibition: there is hysteresis.

8. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector 8 Hysteresis in a Ferromagnet Not a single function magnetic field  direction but a history-dependent relation. Direction of External Magnetic Field In a ferromagnetic material, the atoms act like magnets which tend to align themselves with their neighbors -- this cooperation is opposed by thermal noise. A ferromagnet becomes magnetized when the atomic magnets have high probability of pointing in the same direction.

9. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector 9 Didday's Transformation Scheme From Foodness to Relative-Foodness Didday uses a local mechanism, to combat hysteresis, introducing a Newness cell - "N-cell" for each S-cell to monitor temporal changes in the activity of its region. For a dramatic increase in the region's activity, it overrides the inhibition on the S-cell and permits this new level of activity to enter the relative foodness layer.

10. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector 10 Formalizing: An "N-cell" for each S-cell Introducing an "N-cell" for each S-cell Formally: neurons N i have activity k i proportional to dS i /dt. We then postulate that the input to the i th element to our array is no longer s i but s i + k i. If k i is large enough for long enough, the relative foodness E i can gain ascendancy. SiSi dS i /dt

11. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector 11 Formalizing Didday's Model The rest of this lecture provides a mathematical analysis from TMB2, Section 4.4, and is based on the paper: Arbib, M.A., and Amari, S.I., (1977) Competition and Cooperation in Neural Nets, in Systems Neuroscience (J. Metzler, Ed.), New York: Academic Press, pp. 119-165. To see the mathematics, switch to the Word file: 7W. Amari-Arbib Mathematics