Extinction of Conditioned Behavior Chapter 9 Effects of Extinction Extinction and Original Learning What is learned during Extinction

Effects of Extinction Extinction involves omitting the US or reinforcer  In Pavlovian conditioning, this means repeated presentations of the CS alone  In operant conditioning, this means not giving the reinforcer after the response Two main effects of extinction procedures on behavior  the rate of responding decreases  response variability increases

Extinction Increases Variability in Behavior Three levers present and Var group had to vary sequence Neuringer, Kornell, & Olufs (2001) All groups increased variability during extinction

Extinction and Original Learning Spontaneous Recovery Renewal Reinstatement US-devaluation

Spontaneous Recovery Rescorla (1996) Rats trained to make 2 different responses for food Lever-press – food Nose-poke – food At end of training phase, rats were making about 17 resp/min After acquisition, the rats were given 2 20-min extinction sessions with each response For one of the responses, the extinction phase occurred shortly before the recovery test (R-No rest) For the other response, the extinction phase occurred 7 days before the recovery test (R-Rest) At end of extinction phase, rats were making about 2 resp/min

Spontaneous Recovery Shows importance of passage of time

Renewal Bouton & King (1983) Rats were first trained to bar-press for food Then,Context A T – Shock Conditioning resulted in T suppressing BP response During extinction, the rats were divided into 3 groups Group A: CS alone in Context A Group B: CS alone in Context B Group NE: did not receive extinction After extinction, all rats received test trials in Context A

Renewal

Reinstatement Could be due to context conditioning In initial studies, the excitatory conditioning, extinction and subsequent US exposures were all given in same context Recent research indicates that context is important but not because it permits summation The role of context appears to be to disambiguate the significance of a stimulus that has a mixed history of conditioning and excitation Context has little effect on stimuli that have a history of only conditioning

Reinstatement Bouton (1984) Conditioned suppression with rats Phase 1 CS – weak shock CS – strong shock Phase 2 No treatment Extinction Reinstatement Shock-same Shock-different Shock-same Shock-different Test CS Reinstatement was introduced when the level of responding was similar in all subjects However, some rats got to that point by receiving only conditioning, whereas for others the CS was both conditioned and extinguished

Reinstatement

For rats that did not receive extinction, it did not matter whether the reinstatement shocks were given in the same context or a different context - shows that contextual conditioning did not summate with the suppression elicited by the CS For rats that did receive extinction, reinstatement shocks given in the same context produced more suppression than shocks given in a different context - shows that contextual conditioning facilitates the reinstatement effect According to Bouton, reinstatement US presentations given in the test context serve to restore the excitatory properties of the contextual cues and enable those cues to be more effective in reactivating the memory of excitatory conditioning of the CS

US-Devaluation Rescorla (1993) Used devaluation procedure in operant experiment to show that the original R-O association was not abolished by extinction Phase 1 R 1 – O 1 R 2 – O 2 R 3 – O 1 R 4 – O 2 Phase 2 R 1 extinguished R 2 extinguished US-devaluation O 1 – LiCl Test R 1 vs R 2 and R 3 vs R 4 The extinguished responses were re-trained with a third reinforcer before the devaluation procedure to make sure there would be some responding during the final test

Effect of Devaluation - Operant Shows that extinction does not eliminate the R-O association

What is learned in extinction? The experiments on spontaneous recovery, renewal, reinstatement and US-devaluation all suggest that extinction does not abolish the original learning So why does extinction produce a decrease in responding? Extinction may produce an inhibitory S-R association The non-reinforcement of a response in the presence of a specific stimulus produces an inhibitory S-R association that suppresses that response whenever S is present

Inhibitory S-R Association

Paradoxical Reward Effects Overtraining Extinction Effect Magnitude of Reinforcement Extinction Effect Partial Reinforcement Extinction Effect

In addition to behavioral effects of extinction, there are also emotional effects (i.e. frustration response) The most important variable that determines the magnitude of both the behavioral and emotional effects of an extinction procedure is the schedule of reinforcement that was in effect before extinction was introduced The dominant schedule characteristic that determines extinction effects is whether the response was reinforced every time it occurred (CRF) or only some of the time it occurred (partial or intermittent reinforcement)

The general finding is that extinction is slower, and involves fewer frustration reactions, if a partial reinforcement schedule, rather than CRF, was in effect before extinction — PREE The PREE was previously thought to be counterintuitive - if a reward strengthens a response, then CRF should produce a stronger response and slower extinction Partial Reinforcement Extinction Effect

Mechanisms of the Partial Reinforcement Extinction Effect Discrimination hypothesis Frustration hypothesis Sequential memory hypothesis

Discrimination Hypothesis introduction of extinction is easier to detect after CRF than partial reinforcement with CRF, easier to notice when reinforcement is absent during extinction with PRF, not as easy because reinforcement was sometimes absent during training

2 groups:Phase 1Phase 2 Group 1: CRFCRFExtinction Group 2: partialCRFExtinction Discrimination Hypothesis While this idea makes intuitive sense, it was discarded a long time ago. Jenkins (1962) Extinction procedure should be equally discriminable in the 2 groups However, group 2 was slower to extinguish These results suggest that the PREE is not just a discrimination problem but that subjects actually learned something during partial reinforcement that affects the rate of extinction

Frustration Theory SF:SF:R S Appetitive With F = frustration Early in training, after nonreward, the organism is frustrated and expects not to be rewarded on the next trial. But on some trials following nonreward, the organism is rewarded Thus the stimuli associated with nonreward (frustration) become associated with reward, and the organism learns to respond in their presence – they become S D s for responding. S F is present during extinction, thus the organism makes many responses.

SN:SN: RS Appetitive With N = memory of nonreward Thus the animal learns to make the response when the memory of nonreward, including sequences of nonreward, is present. Sequential Theory Early in training, when the organism is rewarded on a trial after one or more nonreward trials, the organism remembers not being rewarded on those previous trials. That memory then becomes associated with reward on the present trial.

During extinction, the stimulus conditions are always those of non-reward. Animals that were on PR schedules should make many responses in extinction. They have learned to respond in the presence of stimuli associated with nonreward (either frustration or memory of nonreward). In contrast, animals that were on CRF, should make few responses during extinction. They have never experienced nonreward, and never learned to respond in its presence. They quickly become frustrated and give up responding.