PSY105 Neural Networks 5/5 5. “Function – Computation - Mechanism”

Lecture 1 recap We can describe patterns at one level of description that emerge due to rules followed at a lower level of description. Neural network modellers hope that we can understand behaviour by creating models of networks of artificial neurons.

Lecture 2 recap Simple model neurons – Transmit a signal of (or between) 0 and 1 – Receive information from other neurons – Weight this information Can be used to perform any computation

Lecture 3 recap Classical conditioning is a simple form of learning which can be understood as an increase in the weight (‘associative strength’) between two stimuli (one of which is associated with an ‘unconditioned response’)

Lecture 4 recap A Hebb Rule for weight change between two neurons is: – Δ weight = activity 1 x activity 2 x learning rate constant In order to use this rule to associate two stimuli which are separated in time we need neuron activity associated with stimuli to persist in time. – This can be implemented as an ‘eligibility trace’

I present you with a robot which uses a simple neural network to acquire classically conditioned responses. It can, for example, learn to associate a warning stimulus with an upcoming wall and hence turn around before it reaches the wall. Describe an experiment which you would do to test the details of how the robot learns. Say what you would do, and what aspect(s) of the robot's learning the results would inform you of, and why.

The problem of continuous time Stimulus 1 Stimulus 2

Traces Stimulus 1 Stimulus 2

Consequences of this implementation Intensity of CS stimulus Duration of CS stimulus Intensity of UCS stimulus Duration of UCS stimulus Separation in time of CS and UCS The order in which the CS and UCS occur – (cf. Rescola-Wagner discrete time model)

We have designed an information processing system that learns associations Trials Association Sutton, R.S., Barto, A.G. (1990). Time- derivative models of pavlovian reinforcement. In Learning and Computational Neuroscience: Foundations of Adaptive Networks, M. Gabriel and J. Moore, Eds., pp MIT Press.Time- derivative models of pavlovian reinforcement

Without continued pairing Trials Association Stop pairing

Without continued pairing -> extinction Trials Association Stop pairing

Analysis of information processing systems Function (‘computational level’) Computation (‘algorithmic level’) Mechanism (‘implementational level’) Marr, David (1982) Vision. San Francisco: W.H. Freeman

Marrian analysis of classical condtioning Function: learn to predict events based on past experience Computation: Stimuli evoke ‘eligibility traces’. Hebb Rule governs changes in weights [+ other additional assumptions which are always needed when you try and make a computational recipe] Mechanism: At least one response neuron, one unconditioned stimulus neuron and one neuron for each conditioned stimulus

Kim, J. J., & Thompson, R. F. (1997). Cerebellar circuits and synaptic mechanisms involved in classical eyeblink conditioning. Trends in Neurosciences, 20(4),

Marrian analysis: a simple example Function Computation Mechanism

Theory Experiments Synthesis Analysis

Our classical conditioning networks Stimuli Responses CS 1 UCSCS 2 S-S link

Internal representation of the conditioned stimulus

The lab : using Marrian analysis to make predictions

Function What is the purpose of learning for an animal? – Does our model behave in a sensible (‘adaptive’) way when it follows our rule? – Is the rule sufficient to explain animal learning? Test: think of a way you would want the model/robot to behave, test if it does

Computation Intensity of CS stimulus Duration of CS stimulus Intensity of UCS stimulus Duration of UCS stimulus Separation in time of CS and UCS The order in which the CS and UCS occur – (cf. Rescola-Wagner discrete time model) The learning rate The rate of decay of the trace The frequency of pairing

Mechanism Stimuli Responses CS 1 UCSCS 2 S-S link

What is your prediction? What will you do to the rule or the environment? How will you know if it has been confirmed or falsified?

ψ π Ω