PSY105 Neural Networks 4/5 4. “Traces in time” Assignment note: you don't need to read the full book to answer the first half of the question. You should be able to answer it based on chapter 1 and the lecture notes.
Lecture 1 recap We can describe patterns at one level of description that emerge due to rules followed at a lower level of description. Neural network modellers hope that we can understand behaviour by creating models of networks of artificial neurons.
Lecture 2 recap Simple model neurons – Transmit a signal of (or between) 0 and 1 – Receive information from other neurons – Weight this information Can be used to perform any computation
Lecture 3 recap Classical conditioning is a simple form of learning which can be understood as an increase in the weight (‘associative strength’) between two stimuli (one of which is associated with an ‘unconditioned response’)
Nota Bene Our discussion of classical conditioning has involved – A behaviour: learning to associate a response with a stimulus – A mechanism: neurons which transmit signals – These are related by…a rule or algorithm
Learning Rules “When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased.” Hebb, D.O. (1949), The organization of behavior, New York: Wiley
Operationalising the Hebb Rule Turn ….“When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased.” ….Into a simple equation which is a rule for changing weights according to inputs and outputs
A Hebb Rule Δ weight = activity A x activity B x learning rate constant In words: increase the weight in proportion to the activity of neuron A multiplied by the activity of neuron B
Stimulus On Stimulus Off Time
Implications of this rule Stimulus 1 Stimulus 2 CS 1 UCS ?
Implications of this rule Stimulus 1 Stimulus 2 ? CS 1 UCS
Implications of this rule Stimulus 1 Stimulus 2 ? CS 1 UCS weight
Implications of this rule Stimulus 1 Stimulus 2 ? CS 1 UCS Activity A Activity B weight
Implications of this rule Stimulus 1 Stimulus 2 ? CS 1 UCS Activity A Activity B weight = x x 0.1
The most successful model of Classical Conditioning is the Rescorla Wagnar model Accounts for the effects of combinations of stimuli in learning S-S links Based on the discrepancy between what is expected to happen and what happens But… Deals with discrete trials…ie has no model of time Rescorla RA, Wagner AR. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In: Classical Conditioning II: Current Research and Theory (Eds Black AH, Prokasy WF) New York: Appleton Century Crofts, pp , 1972 Robert Rescorla (2008) Rescorla-Wagner model. Scholarpedia, 3(3):2237.
The problem of continuous time Stimulus 1 Stimulus 2
The problem of continuous time Stimulus 1 Stimulus 2
The problem of continuous time Stimulus 1 Stimulus 2
The problem of continuous time Stimulus 1 Stimulus 2 Activity A Activity B = 0
The problem of continuous time Stimulus 1 Stimulus 2 Activity A Activity B = 0
The problem of continuous time Stimulus 1 Stimulus 2 Activity A = 0 Activity B
We need to add something to our model to deal with a learning mechanism that is always “on”
Traces Stimulus 1
Traces Stimulus 1
Traces Stimulus 1 Stimulus 2
Traces Stimulus 1 Stimulus 2 Activity B Activity A
Traces Stimulus 1 Stimulus 2
Consequences of this implementation Size of CS stimulus Duration of CS stimulus Size of UCS stimulus Duration of UCS stimulus Separation in time of CS and UCS The order in which the CS and UCS occur – (cf. Rescola-Wagner discrete time model)