Chapter 9 – Extinction of Conditioned Behavior Outline –Effects of Extinction Procedures Decreased responding Increased variability in responding –Extinction of Original Learning Spontaneous Recovery Renewal of Original Excitatory Conditioning Reinstatement of Conditioned excitation –Enhancing Extinction Number and Spacing of Extinction Trials Reducing Spontaneous Recovery Reducing Renewal Compounding Extinction Stimuli –What is learned in Extinction Paradoxical Reward Effects Mechanisms of the Partial-Reinforcement Extinction Effect

So far we have focused on acquisition effects. –What happens when a stimulus predicts the presence of some outcome This chapter focuses on what happens when that outcome is later omitted Pavlovian –Acquisition CS-US –Extinction CS alone Instrumental –Acquisition S+ –Barpress/keypeck  RF –Extinction Barpress/keypeck  no longer RF

Extinction is a hot area of research –Particularly relevant to therapy Exposure therapies for phobias –Extinguish fear Also for drug addictions –Extinguish cues for drug taking behavior Effects of Extinction Procedures –Domjan’s key example Key no longer works for door –What do you do? »Try several times »Try it in a variety of ways »Jiggle it »Eventually quit

Effects of Extinction –Decrease in responding –Increase in variability of responding Empirical Evidence –Neuringer, Kornell, and Olufs (2001) –Right lever(R), left lever (L), key (K) Three responses required Group Var –Not allowed to repeat Yoked control –No variability requirement but RF was yoked to Group Var

Variability Responding TrainExtinction Var control Var

Extinction also can cause a strong emotional response –Frustration (possible aggression) Car won’t start Vending machine doesn’t work Pigeon’s out of food Pigeon’s key pecking with restrained partner –Extinction = attack

Extinction and Original Learning –Does extinction erase original learning Evidence says no –From several lines »Spontaneous Recovery »Renewal »Reinstatement »Retention of knowledge of the Reinforcer

Spontaneous Recovery –Phase 1 acquisition –Train CS-US –Phase 2 extinction –CS alone –Phase 3 Time off –Phase 4 Extinction What does Spontaneous recovery tell us about extinction learning? –Original learning remains A little time off? –Responding returns

Renewal –A shift in context can renew extinguished learning bring back responding –Demonstrated by Bouton and King (1983) used the conditioned suppression procedure. Phase 1 –Train all rats to bar press Phase 2 –Train conditioned emotional response to a CS (tone) CS (tone)  US (shock)

Phase 3 (Extinction) –3 Groups Group Ext A) –Extinguish CER same context as phase 1 and 2 »Present CS alone in same context Group Ext B) –Extinguish CER different context »Present CS alone in different context Group NE –No extinction Test –Conditioned suppression to tone in original context Result –Ext A? No Fear –Ext B? Fear –NE? Most Fear

Which group demonstrated Renewal? What does this say about Extinction? –1) Initial learning is not forgotten –2) Extinction is at least somewhat context specific Has implications for therapists attempting to extinguish unwanted behaviors –Extinction of phobia or drug taking behavior may be specific to the therapists office –Extinction in multiple contexts?

Reinstatement –Exposure to the US serves as a reminder reinstates an extinguished response. Train –CS (tone)  US (shock) –tone elicits fear. Extinction –CS alone. Reinstatement –US alone Test –Fear tone? Yes = reinstatement

Like Renewal, Reinstatement is context specific –US exposure works best if in the same context Reinstatement is also an issue for therapists –Worry that extinguished fears/behaviors will return if exposed to certain reminder stimuli Patient has intimacy issues because of abusive parents –Treated with therapy Abusive encounter later in life –Reinstates intimacy issues?

Enhancing Extinction –Because Extinction can be so useful therapeutically,efforts have been made to enhance it Number and Spacing of Extinction of Trials –More extinction is more effective Makes sense –New learning after all –Massed trials are more effective than spaced trials This effect seems to be temporary –Within session effect »Large spontaneous recovery

Reducing Spontaneous Recovery –Repeated spontaneous recovery sessions reduces the effect –If there are cues that are specific to extinction, those cues can be effective in reducing spontaneous recovery Requires special extinction cues

Reducing Renewal –Providing extinction training in multiple contexts can reduce renewal effects –Extinction cues can reduce renewal Same as for spontaneous recovery patients asked to recall the context of extinction training (therapists office) showed reduced anxiety in novel locations Prompted some therapists to specifically train “portable” extinction cues –Memorize a specific “relaxation phrase” or carry a “relaxation card”

One view of extinction is that it is due to an increase in frustration that interferes with normal responding Paradoxical Reward Effects –support the acquired frustration view of extinction Whenever expectancy of reward is greatest following training, extinction occurs fastest The Paradox? –Better acquisition causes faster extinction we might expect that the better learning would slow extinction –Not the case

Overtraining extinction effect –More training = faster extinction animal is more sure that reward is forthcoming causes increased frustration when the reward does not come.

Magnitude of reinforcement effect –Bigger reinforcers = faster extinction –Animals are far more frustrated when they miss out on big reinforcers Leads to faster extinction –Would you behave the same? If you expect a big bonus at work and don’t get it? –Might slow you down quite a bit –May not affect you much if you miss out on a small bonus

Partial reinforcement extinction effect (PREE) –Partial reinforcement slows extinction. –Much faster extinction following continuous reinforcement (CRF) –An animal that expects a reinforcer after every response quickly becomes frustrated.

PREE has received a lot of research attention –Has real world relevance Gambling behavior –Widely dispersed pay offs make responding (gambling) very resistant to extinction »Keep playing despite a lot of loss Parenting –Giving in after repeated (annoying) requests for candy/toys ensures that future annoying requests will persist.

Mechanisms of the Partial-Reinforcement Extinction Effect –Three hypotheses Discrimination hypothesis Frustration Theory Sequential Theory

Discrimination hypothesis –Extinction is easier to detect following continuous reinforcement When reinforcement only occurs every so often maybe you don’t notice extinction for a while. –Turns out this isn’t it. –Jenkins (1962) Theios (1962) Phase 1 –Train animals »Group 1 »Partial RF »Group 2 »CRF Phase 2 –Put all animals on CRF for a while Test –Extinction »Still more responding in Partial RF group –Extinction should have been equally discriminable for both groups. –Seems animals learn something longer lasting Perhaps it teaches them not to give up?

Frustration Theory (I consider this a molar theory) –Amsel –Persistence in extinction occurs because the animals have learned to make responses even when they expect nonreward (or are frustrated). –Breaking down frustration theory –Partial RF Sometimes animals get rewarded when they don’t expect it –Early reward on a variable ratio schedule (after a few responses) = surprised They also sometimes get rewarded when they are frustrated –Late rewards on a variable ratio schedule (after many responses) –Thus animals on partial RF schedules learn to respond even when they don’t expect reward and even if frustrated –Animals on CRF never learn this

In addition frustration can be viewed as a kind of drive in Amsel’s theory –It energizes behavior Respond in order to reduce frustration Daly (1969) –Trained rats to expect food in a goal box Then stopped feeding them there –frustrated rats. –Allowed them to jump a hurdle to escape the goal box rats learned to jump the hurdle –No other RF was available –Presumably jumped to escape frustration »Negative RF –Shows that frustration reduction can motivate behavior. We previously discussed the differential outcomes procedure –Matching to sample One choice was reinforced with no food Animals still learn the correct response –Could be to avoid the frustration of repeating the trial

Keep in mind, that continued responding in the face of frustration is a human characteristic as well We often value it –Songwriter continues to work in the face of multiple rejections Sometimes it can be a negative –Gambling in the face of continued losses

Sequential Theory (I consider this a molecular theory) –Capaldi Much of Capaldi’s work is done in a straight alleyway –To examine PREE sometimes there would be food at the end (R) and sometimes not (N) Capaldi examines behavior across specific sequences of trials –RNNRRNR The underlined trials represent rewarded trials that were preceded by nonrewarded trials Thus the rat will have a recent memory of an N trial that is essentially reinforced. –Leading to persistence in extinction –The longer the strings of Ns that the animal has experienced that eventually lead to RF The more resistance to extinction

Both theories are likely correct. –Frustration theory seems to do a better job of explaining PREE when trials are spaced out Long ITIs –Sequential theory does better when trials are close to one another (easy to remember) Short ITIs