Learning about time: Lecture 17 Distinguish periodic (learning to respond at a particular time of day) and interval timing (learning to respond after a particular interval of time). PERIODIC TIMING e.g. Circadian rhythms. Question: is the cyclical behaviour really controlled by time per se? Or is it controlled by stimuli always present at that particular time? Wheel running in the rat (from Carlson): 4am 8am Midday 4pm 8pm Midnight ACTIVITY Light off Light on

When no light cues are available -- for example under continuous dim light -- behaviour maintained on an approximately 25-hour cycle. (Roberts, 1965). Cockroaches show increased activity at dusk. When visual cues removed cycle drifted until increased activity started 15 hours before dusk (cycle slightly less than 24 hours). Restoring visual cues produced a gradual shift back to correct time. Entrainment : light acts as a zeitgeber synchronising the internal clock 4am 8am Midday 4pm 8pm Midnight Constant dim light

Question: Is the apparent internal 24-hour clock the result of environmental experience? Bolles & Stokes (1965) Subjects born and reared under either 19, 24 or 29 hour light/dark cycles. Then fed at a regular point in their own particular cycle. Only animals on the 24-hour cycle learned to anticipate food. FOOD

Evidence for a physiological system that could provide this 24-hour clock? The suprachiasmatic nucleus (SCN) of the hypothalamus may be a candidate. - The metabolic rate in the SCN appears to vary with the day-night cycle. - Lesions abolish circadian regularity of foraging and sleeping in the rat. - It also receives direct and indirect inputs from the visual system -- could keep circadian rhythms entrained with the real day-night cycle. INTERVAL TIMING Consider a normal classical conditioning procedure Tone (20 sec) --> food.....so what happens if the stimulus keeps on going? The peak procedure

Church & Gibbon, 1982 Rats in lit chamber. Stimulus presented for 0.8, 4.0 or 7.2 sec. At stimulus offset a lever presented. If the rat pressed the lever after a 4-sec stimulus it got food; after the other stimuli it did not. Then the rats were tested with a range of stimuli, with durations between 0.8 and 7.2 seconds Reinforced after 2 seconds Reinforced after 4 seconds Reinforced after 8 seconds

Webers Law The just noticeable difference is proportional to the magnitude of the stimulus. Hence small amounts judged more accurately than large amounts May be called the scalar property of timing (applies to other judgements too). One versus two Nineteen versus twenty One versus two Difference = 2-1 = 1 Ratio = (2-1)/2 = 0.5 Nineteen versus twenty Difference = = 1 Ratio = (20-19)/20 = 0.05

comparator pacemaker t pulses per sec Working memory N * t Reference memory K * N * t response? Scalar timing theory e.g. Gibbon, Church & Meck, (1984) Pacemaker emits pulses at a constant rate t (but is some random variation). e.g. t=1 per second When a stimulus is presented, a switch is operated, and the pulses are allowed to accumulate in working memory. This will equal t multiplied by the number of seconds that have passed (N). e.g., after five seconds, 5 pulses stored.

When the reinforcement occurs, pulses stop accumulating; the number of pulses in working memory (N * t) is now stored in reference memory; this storage is not always completely accurate -- there is some memory distortion represented by K: If K=1 then the memory is accurate; if K<1 then a smaller number of pulses is stored; if K>1 then a greater number is stored e.g. store 4.9 After several trials, several numbers stored in reference memory Nm1, Nm2, Nm3, etc e.g. store 4.7, 5.1, 4.9, 5.0, etc. On each trial the animal compares pulses in working memory (N * t) with random value from reference memory Nmx, using the comparator. If values close, animal responds. Comparator uses a ratio rule -- NOT a difference rule i.e. NOT N * t - NMx but N * t - NMx / NMx This is one of the reasons that accuracy is better with short intervals. e.g. draw 4.7 randomly; after 4 seconds compare 4&4.7: / 4.7 = 0.15 small number therefore respond

Potential problems with scalar timing theory 1) There is as yet no physiological evidence for a pacemaker. Alternatives have been proposed: (i) Instead of a pacemaker, timing could be achieved by a series of oscillators, each of which has two states, on or off. If each oscillator switches after a different period of time, then the entire pattern of activation could be used to determine the exact time (e.g., Gallistel, 1990; Church & Broadbent, 1991): RRR RRG RGR RGG GRR GRG GGR GGG (ii) Another alternative is the Behavioural theory of timing (e.g., Killeen & Fetterman, 1988). When animal gets a reward, this stimulates behaviour. The animal moves across an invariant series of behaviours in between reinforcements. A pulse from an internal pacemaker will change the behaviour from one class to another. The behaviour that is occurring when the next reinforcer occurs becomes a signal for that reinforcer.

Food Wipe whiskersWash faceEat the foodExplore corner Food 2) Conditioning and timing supposedly occur at the same time, and yet are controlled by completely different learning mechanisms. Some theories of timing try and explain conditioning; e.g., Gibbon & Balsam (1977). Calculate rate of reinforcement during stimulus, and rate of reinforcement during background. If first is higher than second, get conditioning. Six reinforcers in 60 minutes = 1/10 = 0.1 Four reinforcers in 15 minutes = 4/15 = > 0.1 This theory cannot explain basic phenomena, like blocking. Potential problems with scalar timing theory continued

Some conditioning models try to explain timing -- e.g. Real time models (e.g., Sutton & Barto, 1981). They work with the Rescorla- Wagner model, just like regular conditioning theories. However, the stimulus is assumed to change over the course of its presentation, and this allows the animal to learn about when a reinforcer occurs. Stimulus trace: Food General references Bouton, M.E. (2007). Learning and Behavior. Sinauer Associates. Carlson, N.R. (2001) Physiology of Behaviour. Allyn & Bacon. Chapter 9. Domjan, M. (1988). The principles of learning and behavior. Brooks/Cole Publishing Company. Chapter 12. Pearce, J.M. (1997). Animal Learning and Cognition. Lawrence Erlbaum Associates. Chapter 7. Shettleworth, S.J. (1998). Cognition, Evolution and Behaviour. Oxford University Press. Chapter 8. Wynne, C.D.L. (2000). Animal Cognition. Macmillan. Chapter 5 pp

Bolles, R.C., & Stokes, L.W. (1965). Rats anticipation of diurnal and a-diurnal feeding. Journal of Comparative and Physiological Psychology, 60, Church, R.M., & Broadbent, H.A. (1991). A connectionist model of timing. In M.L.Commons, S Grossberg, & J.E.R. Staddon (Eds.) Neural network models of conditioning and timing (pp ). Hillsdale, N.J.: Lawrence Erlbaum. Church, R.M., & Gibbon, J. (1982). Temporal generalization. Journal of Experimental Psychology: Animal Behaviour Processes, 8, Gallistel, C.R. (1990). The organisation of learning. Cambridge, MA: MIT Press. Gibbon, J., Church, R.M., & Meck, W.H. (1984). Scalar timing in memory. In J. Gibbon & L. Allen (Eds.) Time and time perception, Annals of the New York Academy of Sciences (Vol. 423, pp.52-77). New York: New York Academy of Sciences. Gibbon, J., & Balsam, P. (1981). Spreading association in time. In C.M. Locurto, H.S. Terrace & J. Gibbon (Eds.) Autoshaping and conditioning theory (pp ). New York: Academic Press. Killeen, P.R., & Fetterman, J.G. (1988). A behavioral theory of timing. Psychological Review, 95, Roberts, S.K. (1965). Photoreception and entrainment of cockroach activity rhythms. Science, 148, Sutton, R.S., & Barto, A.G. (1981). Toward a modern theory of adaptive networks: Expectation and prediction. Psychological Review, 88,