PSY402 Theories of Learning Chapter 8, Theories of Appetitive and Aversive Conditioning

Operant Conditioning The nature of reinforcement: Premack’s probability differential theory Response deprivation theory Behavioral economics: Behavioral allocation – blisspoint Choice behavior – Herrnstein’s matching law. Momentary maximization theory Delay-reduction theory

Probability-Differential Theory Premack – a reinforcer can be any activity that is more likely to occur than the reinforced behavior. Manipulators vs eaters High probability behaviors can be used as reinforcers of low probability behaviors. Frequency of the reinforcer decreases when it is made contingent on another response.

Activities can be Reinforcers Playing with toys reinforces working math problems correctly

Response Deprivation Theory Timberlake & Allison – deprivation occurs when an activity is used as a reinforcer and is not freely emitted. The activity is reinforcing because it satisfies the deprivation created. The animal tries to return to its pre-deprivation level of responding. Activities can be reinforcing even if their initial baselines were not higher.

Behavioral Allocation Blisspoint (paired basepoint) – the free operant level of two responses. Unrestricted responding with two choices of behaviors. Blisspoint is used to figure out how much behavior an animal will engage in to obtain a reward. Animals try to get as close to the blisspoint as possible.

Finding the Blisspoint

Contingency Lines for Rewards

Problems with Contingencies Blisspoint is established by looking at behavior before a contingency is established. The established contingency must take blisspoint into account or it may not increase desired behavior.

Choice Behavior Herrnstein’s matching law – describes how animals act when they have two or more choices. Different responses have different schedules of reinforcement. Responding to each choice is proportionate to the reinforcement for each choice – after learning. This can be expressed mathematically.

Mathematical Expression The formula for the matching law is: where R1 and R2 are the rates of response for two alternative responses And r1 and r2 are rates of reinforcement for those responses

Law Predicts Pecking Behavior

Delayed Gratification Why does anyone choose a smaller reward part of the time? Animals and people typically choose a small immediate reward over a larger delayed reward. Large rewards are selected when: The choice is made in advance of reward. Reinforcers are not visible or reward is already present (pleasurable activity).

Complexities of the Matching Law Maximizing law – sometimes the aim is to obtain as many rewards as possible. Explains FR-10 vs FR-40 schedules. Doesn’t work for VI vs VR schedules. Momentary maximization theory – choose best alternative at the time. Delay reduction theory – choose what will get the reward the fastest.

Aversive Theories: Explaining Avoidance The existence of avoidance behavior implies a cognitive process: Behaving in order to prevent an aversive event. Behaviorists like Hull needed to explain this without cognition. Mowrer’s two-factor theory was developed to explain this – but it has problems needing explanation.

Mowrer’s Two-Factor Theory Mowrer proposed a drive-based two-factor theory to avoid explaining avoidance using cognitive (mentalistic) concepts. Avoidance involves two stages: Fear is classically conditioned to the environmental conditions preceding an aversive event. Cues evoke fear -- an instrumental response occurs to terminate the fear.

Mowrer’s View (Cont.) We are not actually avoiding an event but escaping from a feared object (environmental cue). Miller’s white/black chamber – rats escaped the feared white chamber, not avoided an anticipated shock. Fear reduction rewards the escape behavior.

Criticisms of Two-Factory Theory Avoidance behavior is extremely resistant to extinction. Should extinguish with exposure to CS without UCS, but does not. Levis & Boyd found that animals do not get sufficient exposure duration because their behavior prevents it. Avoidance persists if long latency cues exist closer to the aversive event.

Is Fear Really Present? When avoidance behavior is well-learned the animals don’t seem to be afraid. An avoidance CS does not suppress operant responding (no fear). However, this could mean that the animal’s hunger is stronger than the fear. Strong fear (drive strength) is not needed if habit strength is large.

Avoidance without a CS Sidman avoidance task – an avoidance response delays an aversive event for a period of time. There is no external cue to when the aversive event will occur – just duration. Temporal conditioning. How do animals learn to avoid shock without any external cues for the classical conditioning of fear?

Kamin’s Findings Avoidance of the UCS, not just termination of the CS (and the fear) matters in avoidance learning. Four conditions: Response ends CS and prevents UCS. Reponse ends CS but doesn’t stop UCS. Response prevents UCS but CS stays. CS and UCS, response does nothing (control condition).

Both Factors are Important Termination and Avoidance both show greater learning

D’Amato’s Acquired Motive View D’Amato proposed that both pain and relief motivate avoidance. Anticipatory pain & relief responses. Shock elicits unconditioned pain response RP and stimulus SP motivates escape. Classically conditioned cues sP elicit anticipatory pain response rP that motivates escape from the CS.

Anticipatory Relief Response Termination of the UCS produces an unconditioned relief response RR with stimulus consequences SR. Conditioned cues elicit an anticipatory relief response rR with stimulus consequences sR. Example: dog bite elicits pain response, sight of dog elicits anticipatory pain, house elicits relief

A Discriminative Cue is Needed During trace conditioning no cue is present when UCS occurs and no avoidance learning occurs. A second cue presented during avoidance behavior slowly acquires rR-sR conditioning. Similarly, in a Sidman task, cues predict relief -- associated with avoidance behavior, not the UCS.

A Second Cue Helps Trace Learning Group TS saw a second cue associated with termination of shock

Thorndike’s Negative Law of Effect Thorndike suggested that punishment weakens an S-R bond. Skinner’s finding that suppression of behavior is temporary contradicts this. The effect of punishment must be something different than weakening of the S-R bond.

Guthrie’s View of Punishment When punishment occurs, the response to it is conditioned to the environment during the event. Freezing, jumping, flinching. The effect on behavior depends on the UCR elicited by the shock. Shock to forepaws inhibits running but a shock to hindpaws facilitates it. Monkeys struggle more when shocked.

Guthrie’s Competing Response Theory Guthrie suggested that punishment works only if the response elicited by the punishment is incompatible with the punished behavior. Gerbils punished for standing upright do it more, not less.

Problems with Guthrie’s Theory Response competition alone is insufficient to make punishment effective. When punishment is contingent instead of just co-occurring, it is more effective. Contingent means the punishment happens only when the behavior occurs, not independent of it, randomly

Este’s Motivational View When a behavior is rewarded, the motivational system becomes associated with the behavior. The response occurs the next time the motivational system is activated. Punishment works by changing the motives. Stimuli associated with punishment inhibit the motivational state.

Support for Estes Thirsty rats were trained to lever press for water and “dry lick” for air on alternate days. Punishment of both behaviors had a greater effect on dry licking (a thirst-related behavior) than lever pressing. If the behavior rather than the motive were being suppressed no such difference should occur. Results differed with hungry rats.