Mechanisms of Extinction

What is Learned in Extinction? The experiments on spontaneous recovery, renewal, reinstatement and resurgence all suggest that extinction does not abolish the original learning. There also is evidence that R-O and S-O associations are intact following extinction S – [ R – O ] conditioning procedure followed by S only extinction p R – O stays intact S --- O stays intact So why does the responding go down during extinction? What type of new learning (new association) occurs during extinction?

Extinction Produces An Inhibitory S-R Association Extinction is a special type of nonreinforcement it occurs after an experience with that reinforcer Similar to producing a conditioned inhibitor must have excitatory conditioning before inhibitory conditioning can occur behavior is being withheld, inhibited an inhibitory S-R association that suppresses that response whenever S is present. Predicts that extinction should be highly context-specific Predicts that ‘frustration’ brought about by the omission of an expected reinforcer discourages responding due to establishment of inhibitory S-R association Also produces negative affect when the expected outcome does not occur

Extinction Produces An Inhibitory S-R Association Rescorla (1993) Procedures: see Table9.2 Phase 1: with noise(N) nose poking (Rc) reinforced with food(P) with light(L) nose poking(Rc) reinforced with food(P) noise and light act as modulators (occasion setters) L & N presentations are supposed to create expectancy of reward Phase 2: no light or noise lever press(R1) reinforced with food(P) chain pull(R2) reinforced with food(P) No S-R associations possible between N or L and lever press or chain pull Extinction: Noise(N) lever press(R1) not reinforced Light(L) chain pull(R2) not reinforced Dose this establish negative S-R associations? N - R1 L - R2

Extinction Produces An Inhibitory S-R Association Rescorla (1993) Test (N)with lever press(R1) or (N) with chain pull(R2) (L)with lever press(R1) or (L) with chain pull(R2) Testing for presence of inhibitory S-R associations as driving force behind extinction If inhibitory N-R1 association forms during extinction then make fewer R1 than R2 responses with noise occasion setter If inhibitory L-R2 association forms during extinction then make fewer R2 than R1 responses with light occasion setter Results: Figure 9.11 The graph labeling is a little confusing: ITI: is the baseline responding without any stimuli presented Ext: responding during the test for either the N:R1 or L:R2 combinations Not Ext: when either the N:R2 or L:R1 combinations where presented There is decreased responding during combinations that had been extinguished Suggests that extinction procedure produced inhibitory S-R association specific to particular S-R combinations

The Principles of Learning and Behavior, 7e by Michael Domjan Copyright © 2015 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.

N:R2 or L:R1 N:R1 or L:R2 The Principles of Learning and Behavior, 7e by Michael Domjan Copyright © 2015 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.

Paradoxical Reward Effects Overtraining Extinction Effect with more acquisition training there will be faster extinction More training produces greater expectation of reward Because extinction effects is due to frustration more expectation will produce more frustration during extinction. Magnitude of Reinforcement Extinction Effect Larger reinforcers produce faster extinction Larger reinforcers produces greater expectation of reward

Paradoxical Reward Effects Partial Reinforcement Extinction Effect (PREE) Partial reinforcement (PRF) schedules produces slower extinction then do continuous reinforcement schedules Continuous reinforcement schedule (CRF) when one response results in one reinforcer Partial reinforcer schedules are anything else such as ten response results in one reinforcer PREE can explain habitual gambling, Horsley 2012 Figure 9.12 Persistence even when not winning is stronger in individuals familiar with gambling Compare individuals with little gambling experience to those with frequent gambling Playing a computer game to open a pirate’s treasure chest with verbal feedback Conditioning One group received CRF Another group received PRF at 50% Extinction: can not open the pirate’s treasure chest PREE produced intermittent reinforcement is common because reinforcers are inconsistently located and maintained

FIGURE 9.12 Number of responses in extinction for low and high gamblers following continuous reinforcement or partial reinforcement. (Based on Horsley et al., 2012.) The Principles of Learning and Behavior, 7e by Michael Domjan Copyright © 2015 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.

Mechanisms of the Partial Reinforcement Extinction Effect Discrimination Hypothesis introduction of extinction is easier to detect after CRF than partial reinforcement With CRF, easier to notice when reinforcement is absent during extinction with PRF, not as easy because reinforcement was sometimes absent during training While this idea makes intuitive sense, it was discarded a long time ago. Jenkins (1962) Group 1: trained with CRF - CRF - Extinction Group 2: trained with PRF - CRF - Extinction Extinction procedure should be equally discriminable However, group 2 was slower to extinguish PREE is not just a discrimination problem learn something during partial reinforcement that affects the rate of extinction

Mechanisms of the Partial Reinforcement Extinction Effect Frustration Theory PREE is maintained by emotional effects of nonreward During PRF Early in training nonreward produces frustration But on some subsequent trials there is reward There is a sequence of nonreward -- frustration – reward Continuing to respond during frustration is rewarded Can learn to respond in the presence of frustration During extinction there is a long sequence of nonreward -- frustration which produces many responses Similar to the ideas about resilience training in children

Mechanisms of the Partial Reinforcement Extinction Effect Sequential Theory PREE is maintained by memory of nonreward During PRF There is a sequence of reward (R) -- nonreward (N) – reward (R) remember not being rewarded on the (N) trials Memory - backward looking - remembers nonreward on trial N-1 With a sequence of R and N trials: R N N R R N R Trial 4 - memory of 2 N trials Trial 7 - memory of 1 N trials Memory of nonreward then becomes associated with reward on the present trial learn to make the responses after nonreward During extinction, always non-reward

Resistance to Change and Behavioral Momentum PREE is resistance to change even when reinforcement ends Behavioral momentum- as in “a body in motion tends to stay in motion” tendency to continue with the behavior even without reinforcement, something like a habit directly related to the rate of reinforcement but is not related to the response rate Dube (2003) using video game with developmentally disabled children responding to computer icon "a sprite" by touching the screen getting reinforcement on a VI 12 sec schedule or VI 12 sec plus extra reinforcers more behavioral momentum with the increased reinforcer rate Figure 9.13

FIGURE 9.13 Relative rate of responding during two components of a multiple schedule that involved either a low or high rate of reinforcement during a test for behavioral momentum for 10 students identified by the letters on the horizontal axis. (From Dube et al, 2003. Figure 1, page 139.) The Principles of Learning and Behavior, 7e by Michael Domjan Copyright © 2015 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.