1. Mechanisms of Extinction

2. What is Learned in Extinction?
The experiments on spontaneous recovery, renewal, reinstatement and resurgence all suggest that extinction does not abolish the original learning.
There also is evidence that R-O and S-O associations are intact following extinction
S – [ R – O ] conditioning procedure followed by S only extinction p
R – O stays intact
S --- O stays intact
So why does the responding go down during extinction?
What type of new learning (new association) occurs during extinction?

3. Extinction Produces An Inhibitory S-R Association
Extinction is a special type of nonreinforcement
it occurs after an experience with that reinforcer
Similar to producing a conditioned inhibitor
must have excitatory conditioning before inhibitory conditioning can occur
behavior is being withheld, inhibited
an inhibitory S-R association that suppresses that response whenever S is present.
Predicts that extinction should be highly context-specific
Predicts that ‘frustration’ brought about by the omission of an expected reinforcer discourages responding due to establishment of inhibitory S-R association
Also produces negative affect when the expected outcome does not occur

4. Extinction Produces An Inhibitory S-R Association
Rescorla (1993) Procedures: see Table9.2
Phase 1:
with noise(N) nose poking (Rc) reinforced with food(P)
with light(L) nose poking(Rc) reinforced with food(P)
noise and light act as modulators (occasion setters)
L & N presentations are supposed to create expectancy of reward
Phase 2: no light or noise
lever press(R1) reinforced with food(P)
chain pull(R2) reinforced with food(P)
No S-R associations possible between N or L and lever press or chain pull
Extinction:
Noise(N) lever press(R1) not reinforced
Light(L) chain pull(R2) not reinforced
Dose this establish negative S-R associations?
N - R1
L - R2

5. Extinction Produces An Inhibitory S-R Association
Rescorla (1993)
Test
(N)with lever press(R1) or (N) with chain pull(R2)
(L)with lever press(R1) or (L) with chain pull(R2)
Testing for presence of inhibitory S-R associations as driving force behind extinction
If inhibitory N-R1 association forms during extinction then make fewer R1 than R2 responses with noise occasion setter
If inhibitory L-R2 association forms during extinction then make fewer R2 than R1 responses with light occasion setter
Results: Figure 9.11
The graph labeling is a little confusing:
ITI: is the baseline responding without any stimuli presented
Ext: responding during the test for either the N:R1 or L:R2 combinations
Not Ext: when either the N:R2 or L:R1 combinations where presented
There is decreased responding during combinations that had been extinguished
Suggests that extinction procedure produced inhibitory S-R association specific to particular S-R combinations

6. The Principles of Learning and Behavior, 7e by Michael Domjan
Copyright © 2015 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.

7. N:R2 or L:R1
N:R1 or L:R2
The Principles of Learning and Behavior, 7e by Michael Domjan
Copyright © 2015 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.

8. Paradoxical Reward Effects
Overtraining Extinction Effect
with more acquisition training there will be faster extinction
More training produces greater expectation of reward
Because extinction effects is due to frustration
more expectation will produce more frustration during extinction.
Magnitude of Reinforcement Extinction Effect
Larger reinforcers produce faster extinction
Larger reinforcers produces greater expectation of reward

9. Paradoxical Reward Effects
Partial Reinforcement Extinction Effect (PREE)
Partial reinforcement (PRF) schedules produces slower extinction then do continuous reinforcement schedules
Continuous reinforcement schedule (CRF) when one response results in one reinforcer
Partial reinforcer schedules are anything else such as ten response results in one reinforcer
PREE can explain habitual gambling, Horsley Figure 9.12
Persistence even when not winning is stronger in individuals familiar with gambling
Compare individuals with little gambling experience to those with frequent gambling
Playing a computer game to open a pirate’s treasure chest with verbal feedback
Conditioning
One group received CRF
Another group received PRF at 50%
Extinction: can not open the pirate’s treasure chest
PREE produced intermittent reinforcement is common because reinforcers are inconsistently located and maintained

10. FIGURE 9.12
Number of responses in extinction for low and high gamblers following continuous reinforcement or partial reinforcement. (Based on Horsley et al., 2012.)
The Principles of Learning and Behavior, 7e by Michael Domjan
Copyright © 2015 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.

11. Mechanisms of the Partial Reinforcement Extinction Effect
Discrimination Hypothesis
introduction of extinction is easier to detect after CRF than partial reinforcement
With CRF, easier to notice when reinforcement is absent during extinction
with PRF, not as easy because reinforcement was sometimes absent during training
While this idea makes intuitive sense, it was discarded a long time ago.
Jenkins (1962)
Group 1: trained with CRF - CRF - Extinction
Group 2: trained with PRF - CRF - Extinction
Extinction procedure should be equally discriminable
However, group 2 was slower to extinguish
PREE is not just a discrimination problem
learn something during partial reinforcement that affects the rate of extinction

12. Mechanisms of the Partial Reinforcement Extinction Effect
Frustration Theory
PREE is maintained by emotional effects of nonreward
During PRF
Early in training nonreward produces frustration
But on some subsequent trials there is reward
There is a sequence of nonreward -- frustration – reward
Continuing to respond during frustration is rewarded
Can learn to respond in the presence of frustration
During extinction there is a long sequence of nonreward -- frustration which produces many responses
Similar to the ideas about resilience training in children

13. Mechanisms of the Partial Reinforcement Extinction Effect
Sequential Theory
PREE is maintained by memory of nonreward
During PRF
There is a sequence of reward (R) -- nonreward (N) – reward (R)
remember not being rewarded on the (N) trials
Memory - backward looking - remembers nonreward on trial N-1
With a sequence of R and N trials: R N N R R N R
Trial 4 - memory of 2 N trials
Trial 7 - memory of 1 N trials
Memory of nonreward then becomes associated with reward on the present trial
learn to make the responses after nonreward
During extinction, always non-reward

14. Resistance to Change and Behavioral Momentum
PREE is resistance to change even when reinforcement ends
Behavioral momentum-
as in “a body in motion tends to stay in motion”
tendency to continue with the behavior even without reinforcement, something like a habit
directly related to the rate of reinforcement
but is not related to the response rate
Dube (2003)
using video game with developmentally disabled children
responding to computer icon "a sprite" by touching the screen
getting reinforcement on a VI 12 sec schedule or VI 12 sec plus extra reinforcers
more behavioral momentum with the increased reinforcer rate
Figure 9.13

15. FIGURE 9.13
Relative rate of responding during two components of a multiple schedule that involved either a low or high rate of reinforcement during a test for behavioral momentum for 10 students identified by the letters on the horizontal axis. (From Dube et al, Figure 1, page 139.)
The Principles of Learning and Behavior, 7e by Michael Domjan
Copyright © 2015 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.