1. How do animals “know” when a schedule is on extinction?

2. Remember
Operant conditioning extinction differs from classical conditioning extinction
Responds decreases to near zero for both
Operant conditioning:
Transient increase
Extinction induced aggression

3. Partial Reinforcement Extinction Effect: PREE
Extinction occurs at different rates depending on the schedule:
Continuous reinforcement: FAST extinction
Partial reinforcement schedules: SLOWER extinction
Variable schedules show slower extinction than fixed (rate or time) schedules.
PREE used to describe greater persistence in instrumental responding during extinction after partial (or intermittent) reinforcement training
Faster extinction after continuous reinforcement training.
Partial reinforcement schedules show RESISTANCE TO EXTINCTION

4. Other Extinction Effects
magnitude reinforcement extinction effect
Less persistence of instrumental behavior in extinction following training with a large reinforcer
More persistance of responding with a small or moderate reinforcer.
Effect is most prominent with continuous reinforcement.
overtraining extinction effect
Less persistence of instrumental behavior in extinction following extensive training with reinforcement (overtraining)
Faster extinction following moderate levels of reinforcement training.
Again, effect most prominent with continuous reinforcement

5. Other Extinction Effects
reinstatement
Recovery of responding to an extinguished stimulus
produced by exposures to unconditioned stimulus or reinforcer
renewal
Recovery of excitatory responding to an extinguished stimulus
produced by shift away from the contextual cues that were present during extinction.

6. Why do we get these effects?
Behavioral momentum and Sequential theory
Trouble with discrimination
Frustration

7. Behavioral Momentum
Suggests that PREE occurs because the animal has a high momentum of responding and it is more difficult to stop this momentum
Timberlake and Lucas 1985:
Ball bearing studies
Rolled ball bearing across cage; rats had to let it go past to receive reinforcer
Played with the ball bearing, slowing reinforcement
During extinction (ball bearing but no food): played with ball bearings MORE
Does suggest that animals show strong patterns of behavior that may interfere and thus slow the extinction process
But not a complete explanation

8. Discrimination and Frustration
Discrimination hypothesis:
Mowrer and Jones 1945
In order for subjects’ behavior to change during extinction, the subject must be able to discriminate the change in reinforcement contingencies
With CRF: This is immediately noticeable
With PRF: not immediately noticeable
More discriminative on fixed schedules
Less discriminative on variable schedules
Evidence does not completely support this

9. Generalization Decrement Hypothesis
Capaldi, 1966
Generalization decrement: decreased responding observed in generalization test when test stimuli become less and less similar to training stimulus
Responding during extinction is weak if the stimuli present during extinction are different from those during the reinforcement phase
Responding during extinction is STRONG if the stimuli present during extinction are very similar to those during reinforcement phase.

10. Generalization Decrement Hypothesis
Large generalization decrement when schedule moves from CRF to EXT
Subject never experienced situation in which some of its responses are not reinforced
Not been taught to keep responding in absence of a reinforcer
Small generalization decrement when schedule moves from PRF to EXT
Subject has experience in situation where some of its responses are not reinforced
HAS been taught to keep responding in absence of a reinforcer

11. Sequential Theory sequential theory: memory of reward vs. non reward
Cognitive theory
Fast extinction after CRF
Extinction occurs quickly because the instrumental response has NOT been conditioned to the memory of nonreward
Slow extinction during PREE
extinction is slowed after partial reinforcement because the instrumental response becomes conditioned to the memory of nonreward.

12. Bottom Line Rate of extinction is affected by: Why frustration?
Discriminability of the situation
Previous experience with non-reinforced responding
Possibly, memory of non-reinforced responding
Why frustration?
Not getting what expected
Because of contingency between response and reinforcer:
Subject “controls” responding, and thus reward
During EXT: this contingency is disrupted
Subject becomes “frustrated” in that
Tries other responses
Increases magnitude of responding
Increases intensity of responding
Pushed into fear/flight/fight sequence: aggression is elicited

13. Avoidance:

14. Avoidance Tests
Negative reinforcement = removing a stimulus to INCREASE a behavior
Negative reinforcement =
escape: a response removes something
avoidance: a response prevents some event
Procedure for studying negative reinforcement and avoidance: Discriminated avoidance:
a response CANCELS a shock
Organism is responding for food reinforcers
When light comes on, must press another lever to AVOID the shock
if the response does not occur during the S+ the stimulus is followed by a shock
if the response does occur during the S+, the shock is cancelled
thus: signal or sD for shock
if this were an escape: response could also occur DURING the shock to shut off shock

15. Two Avoidance Procedures:
Sidman Avoidance:
the response POSTPONES or DELAYS the shock
thus: only temporary solution
must keep responding to keep delaying the shock
results in lots of responding
again: some signal may be used to signal when must respond
Herrnstein and Hineline Procedure:
the response reduces the rate of the shock
note: note delay or cancel, just slows down rate of delivery
the response switches the schedule of shock to a lower rate
Note: cannot entirely AVOID shock in this procedure:
once animal receives shock on lowered schedule, reverts back to original schedule
animal must respond again to switch schedule again

16. Characteristics of Avoidance Behavior
extremely variable:
from subject to subject
from session to session with SAME subject
procedure to procedure
choice of response is important
determines how quickly will learn contingency
how well learning is maintained
1-way vs 2-way shuttle avoidance tests:
1-way shuttle: run to other end w/sD
2-way shuttle: run to opposite end w/sD
rat will learn to run to other end of shuttle box when sD comes on to avoid shock
difficult time learning 2-way shuttle avoidance

17. Characteristics of Avoidance Behavior
Species specific Defense reactions:
behaviors which animal does naturally in time of danger
includes: freezing, fleeing, fighting
Why?
animal has innate behaviors does when avoiding noxious stimulus-
can't make it go against its nature

18. Characteristics of Avoidance Behavior
Negatively reinforced behavior is difficult to extinguish:
escape behaviors take long time to go away
e.g.: rat in 1-way shuttle still runs when light comes on-even after hundreds of EXT trials
BUT: will extinguish quickly if animal can detect change from conditioning to EXT situation

19. Negative Reinforcement in Humans
most often "reinforcement" technique used in real world
often used because is cheaper, easier, more natural
produces "bad" side effects: avoidance responses to sD = boss, principal, spouse, etc.

20. Theories of Avoidance: Two Factor theory
Two things happen during avoidance conditioning:
animal learns to fear S+ via class. cond'ing
CS (light)---> US (shock): UR (fear)
animal learns to fear light via pairing with shock
animal will then learn a response to AVOID shock and thus remove/lessen their fear
thus: not getting shocked reduces fear that was signaled by the CS
experimental evidence:
on initial training trials:
light/CS produces physiological symptoms of fear
escape response results in decrease in these physiological symptoms
on later trials:
little or no evidence of physiological fear with CS presentation
suggests fear has been reduced/replaced by the escape response
in sense: forms a negative feedback loop

21. Two Factor theory in Humans
many ineffective and/or irrational fears
often involve avoidance responses due to original fear
maintained by decrease in fear
e.g., banging two sticks to keep the tigers away
Symptoms of obsessive/compulsive disorders:
compulsions = repeated, stereotyped, ritualized actions
individual feels compelled to engage in them
obsessions = compulsive thoughts (no actual actions)
many, many examples of this
can begin to interfere in life

22. Problems with 2-factor theory:
Signs of fear dissipate w/time:
as animal gets "better" at avoidance response
thus: no fear to be avoided
the CS is not as important in avoidance learning as 2-factor theory states:
animals can learn to avoid in a discriminated avoidance situation long before there is any sign that they are responding to/detecting the CS

23. Herrnstein and Hineline: Test of 2-factor theory
Herrnstein and Hineline procedure:
not use any CS, but the animal still learns to lessen/avoid the shock
Test of the theory:
two groups of rats used
Group 1: can turn off light, but still shock
Group 2: can turn off shock, light still on
2-factor theory would predict that Group 1 should respond more, because this would be cancelling the CS that produces fear
results: group 2 responds much more accurately, faster

24. Alternative: One-Factor Theory
responses occur whenever they reduce the rate at which aversive events occur
when a CS is present: only providing information about the effectiveness of a response
fear may be a by-product of avoidance training, but not crucial to learning/ maintaining an avoidance response

25. Evidence for One-Factor theory
Almost postulating a "cognitive" theory of avoidance:
Seligman and Jonston (1973) did postulate cognitive theory:
like Rescorla Wagner theory in that deals with predictability
Basic premise:
learning occurs only when discrepancy between observation and expectation
subjects' behavior will change in avoidance task whenever there is a discrepancy between expectancy and observation

26. Evidence for One-Factor theory
Two important expectations in avoidance task:
expectation about consequences of a response
.expectation about consequences of not responding
Data support One-factor theory
on trial 1: no expectations
On trial 2 (and more): expectation about what will happen
no shock will occur if response is made
shock will occur if no response is made
animal prefers no shock to shock- so responds
Contingency is what is important in avoidance, fear is by-product!

27. Evidence, con’t. as long as animal continues to respond- no shock
not know when extinction occurs- no sampling
only stop when learn situation has changed
Thus: to EXT responding:
Must use response blocking or flooding:
present sD, but prevent R from occurring
thus animal learns that shock no longer comes
animal stops responding in presence of sD

28. Flooding
To extinguish an inappropriate response: must make contact with "changed reinforcement or punishment" situation
sometimes used as alternative to systematic desensitization
flood with presentation of fear-provoking stimulus
Again, no actual consequence occurs
Continue presentations until the response is extinguished
Problem: may "scare the patient to death"