Operant conditioning

In classical conditioning, the presence of one stimulus (e.g. meat powder) is conditional on the presence of another stimulus (e.g., a bell) What else can an animal learn, besides the relationship of two stimuli?

It is also possible for the animal to generate a response and for that response to have consequences: Operant conditioning Act cute, you get pet Poop on the rug, you get scolded

Note that the thing to be learned is not a UR. Animal emits a response (pooping, acting cute), and it is rewarded or punished.

Edward Thorndike

Thorndike’s Law of Effect “If a response in the presence of a stimulus is followed by a satisfying event, the association between the stimulus and the response is strengthened. If the response is followed by an annoying event, the association is weakened.”

Today we’ll cover: Basics of operant conditioning What makes operant conditioning effective. The problem of definition in o.c. (not just that “animals seek rewards”).

Thorndike’s method was limited because each trial took so long.

A stripped-down environment

Free operant curve, from a cumulative recorder Steep slope=many responses Shallow slope=few responses

What would the curve look like if 20 bar presses  food?

To really teach the animal you would shape it’s behavior...

Fixed ratio Consistent ratio of number of responses & number of reinforcers Example: factory Piece work Steady response Easy to extinguish

Variable ratio Set ratio of number of responses & number of reinforcers, but can vary locally Example: slot machine Rapid response Hard to extinguish

Fixed interval First response after a specific amount of time since the last reinforcement Example: studying for exams Little response until just before reinforcement: then rapid response Fairly easy to extinguish

Variable interval First response after a some amount of time since the last reinforcement: amount of time can vary, locally Example: checking Steady response Hard to extinguish

Contingencies don’t just add good stuff... Positive Reinforcement Negative Reinforcement Punishment Add to environment Negative Punishment (Extinction) Take away from environment Increase probability of behavior Decrease probability of behavior Result Action e.g., food e.g., escape e.g., spanking e.g., being grounded

Complex contingencies: Would this work? Bar press reinforced, but ONLY when red light is on. YES! This is called differential reinforcement

How does differential reinforcement apply here?

Reinforcer = food. Response = hovering Differential signal = looking up.

What’s happening, and what should the birds do?

What’s happening= differential sign has changed What should the birds do = stop responding

Moments later, birds are leaving

Operant conditioning--what makes it effective? Schedule of reinforcement Temporal contingency Belongingness Quality, quantity of reinforcer What else the animal might do

T-maze: temporal contingency

Condition 1: immediate reward (.5 sec) !

Condition 2: delayed reward (5 sec) !

Effectiveness--temporal contingency The delay between the animal’s act that you are reinforcing, and the reinforcer.

WHY does learning drop off with delay??

Condition 2: delayed reward (5 sec) !

Operant conditioning--what makes it effective? Schedule of reinforcement Temporal contingency Belongingness Quality, quantity of reinforcer What else the animal might do

Belongingness Thorndike tried to condition his cat to yawn or scratch to escape box--he proposed belongingness

Instinctive drift A concept related to belongingness: instinctive drift (Breland & Breland.)

Motivational state can also influence; a hungry animal does more food-seeking behaviors... Digging Digging, scratching, rearing

Quality/quantity of reinforcer Works as you would expect.

What else might the animal do? It’s not as simple as “the animal Maximizes good things, minimizes” bad things. Even humans don’t do this, if the situation gets moderately complex.

Example Variable ratio Variable interval What’s the optimal strategy?

Variable ratio Variable interval Optimal is to hit VR almost exclusively and occasionally hit the VI. Instead, they respond to equalize ratios of work/reward

The problem of definition What is a reinforcer?

The problem of definition Thorndike called a reinforcer something “that brings about a satisfying state of affairs.” How do we know when animal is satisfied? Presumably, when the animal will work to achieve this state of satisfaction.

But that’s circular What will the animal work for (e.g., peck)? What’s a reinforcer? Something pleasurable. What’s pleasurable? Something that increases behavior, that animal will work to get

Another definition: physiological homeostasis Animal seeks to lessen thirst, hunger, etc. Definition of reinforcement is based on biological drives. Learning = a “stamping in” of the work that needs to be done to reduce hunger. E.g, “I must not only consume and chew to get nourishment. I also must press the bar, then consume, then chew.

Problems Too many drives were proposed. Animals (and people) do things that seem more likely to raise drives, not lower them

Reinforcement as behavioral regulation Premack principle: Given two responses arranged in an operant conditioning procedure, the more probable response will reinforce the less likely behavior.

Which do you want to do: play pinball or eat candy? Must eat candy to play pinball These kids treat candy eating as work: do it to get to play pinball. These kids eat candy but don’t care that they have earned pinball time.

Behavioral homeostasis & bliss point—a clever, not-quite-right idea Normally, animal likes to be at gray spot (15 minutes of each--now it can’t be at gray spot. What will it do?

IN THEORY you should be able to predict what animal will do--it will select spot on blue line that is as close as possible to it’s “bliss point”. IN REALITY this prediction sometimes works, sometimes doesn’t.

Reinforcement--final word In the end, we still don’t have a good definition of the concept. Premack Principle is as close as we get. Nevertheless, the concept of reinforcement seems useful.

Applications Animal training Biofeedback Education Token Economies

Biofeedback Operant conditioning of the autonomic nervous system. For years, not explored because no one thought it could possibly work.

Apply operant conditioning principles to education 1. Make sure student doesn’t make mistakes; guide behavior. 2. Review frequently.

Little enthusiasm. Teachers didn’t like it for their own reasons. Students were bored.

Token economies Used in some mental health institutions, and some classrooms.

Mrs. Ahlersmeyer’s 3 rd grade class, Lafayette Elementary, Lafayette, IN Students earn a “salary” (marbles). Outstanding work or behavior earns bonuses. Students allowed 5 sick days per quarter, after that, they are docked pay. Students charged rent for their use of desk, and for any school property lost or damaged. Students docked pay for inappropriate behavior.

Use is controversial because it seems “dehumanizing” (mental patients) or because it seems that you’re “paying” students for behavior that they should want to do.

Applications Animal training Biofeedback Education Token Economies