Chapter 5 Learning

What is Learning? Learning: experience leads to a relatively permanent change in behavior Learning: experience leads to a relatively permanent change in behavior Conditioning: a behavior becomes associated with a stimulus Conditioning: a behavior becomes associated with a stimulus Stimulus: anything that influences behavior Stimulus: anything that influences behavior

Classical Conditioning Classical (Pavlovian) Conditioning: a response (salivation) naturally elicited by one stimulus (food) comes to be elicited by a different stimulus (bell) Classical (Pavlovian) Conditioning: a response (salivation) naturally elicited by one stimulus (food) comes to be elicited by a different stimulus (bell) Ivan Pavlov: Russian physiologist, discovered classical conditioning Ivan Pavlov: Russian physiologist, discovered classical conditioning

4 Classical Conditioning Terms 1. unconditioned stimulus (US): stimulus that naturally (reflexively) “elicits” a response 1. unconditioned stimulus (US): stimulus that naturally (reflexively) “elicits” a response – food 2. unconditioned response (UR): response (reflexively) elicited by the unconditioned stimulus 2. unconditioned response (UR): response (reflexively) elicited by the unconditioned stimulus – salivation

4 Classical Conditioning Terms 3. conditioned stimulus (CS): through pairing with an US (food) the CS (bell) comes to elicit the same response (salivation) 3. conditioned stimulus (CS): through pairing with an US (food) the CS (bell) comes to elicit the same response (salivation) 4. conditioned response (CR): same as the unconditioned response (salivation) BUT is elicited by the CS (bell) NOT the US (food) 4. conditioned response (CR): same as the unconditioned response (salivation) BUT is elicited by the CS (bell) NOT the US (food)

Classical Conditioning Procedure Before Conditioning Food (US) Salivation (UR) Bell (CS) No Salivation (CR)

Classical Conditioning Procedure During Conditioning Bell (CS) Food (US) Salivation (UR)

Classical Conditioning Procedure After Conditioning Bell (CS) Salivation (CR)

J.B. Watson’s Little Albert Study By “pairing” a loud noise with a white rat (cute), the white rat became a CS for fear in little Albert By “pairing” a loud noise with a white rat (cute), the white rat became a CS for fear in little Albert What is US? > Loud Noise What is US? > Loud Noise What is UR? > Crying What is UR? > Crying What is CS? > White Rat What is CS? > White Rat What is CR? > Crying What is CR? > Crying

Factors Affecting Conditioning (not in book) Order of presentation: Conditioning is much more effective if CS precedes US (e.g., first the white rat, then the noise) Order of presentation: Conditioning is much more effective if CS precedes US (e.g., first the white rat, then the noise) Inter-Stimulus Interval (ISI): about 1/2 second to a few seconds is best Inter-Stimulus Interval (ISI): about 1/2 second to a few seconds is best Numer of CS - US pairings: usually many are needed (except in conditioned taste aversion!) Numer of CS - US pairings: usually many are needed (except in conditioned taste aversion!)

Biological Preparedness Hypothesis Martin Seligman: evolution has made us more likely to become conditioned to stimuli that are “potentially dangerous” Martin Seligman: evolution has made us more likely to become conditioned to stimuli that are “potentially dangerous” – heights, thunder, animals, water, fire, people, insects, etc.

Conditioned Taste Aversion An exception to the need for multiple CS - US pairings An exception to the need for multiple CS - US pairings One trial learning: animals (and sometimes humans) will learn to avoid a taste/smell that has been associated with sickness ONE time. This has “survival value” One trial learning: animals (and sometimes humans) will learn to avoid a taste/smell that has been associated with sickness ONE time. This has “survival value”

Operant Conditioning Operant conditioning: organism “operates” on the environment to cause the occurrence (or non- ocurrence) of some event (an if - then statement) Operant conditioning: organism “operates” on the environment to cause the occurrence (or non- ocurrence) of some event (an if - then statement) Instrumental Conditioning: another name for operant conditioning (organism is “instrumental” in its own learning Instrumental Conditioning: another name for operant conditioning (organism is “instrumental” in its own learning

Edward L. Thorndike used his “puzzle box” with cats to study operant type behavior used his “puzzle box” with cats to study operant type behavior

Thorndike’s Law of Effect Behavior that leads to a pleasant outcome is “stamped in” Behavior that leads to a pleasant outcome is “stamped in” – e.g., cat hits lever and escapes box Behavior that leads to an unpleasant outcome is “stamped out” Behavior that leads to an unpleasant outcome is “stamped out” – e.g., cat hits lever and gets shocked

B. F. Skinner Greatly expanded on Thorndike’s ideas Greatly expanded on Thorndike’s ideas Invented the “Skinner Box” Invented the “Skinner Box” – A box with a lever (for a rat to press) or a disk (for a pigeon to peck – a system for delivering food – a metal floor that could deliver a mild shock Believed all behaviors, thoughts, words, were “learned” and could be studies and, perhaps, changed Believed all behaviors, thoughts, words, were “learned” and could be studies and, perhaps, changed

Language of Operant Conditioning reinforcement: following a behavior with a consequence (event) that INCREASES the probability that the behavior will be repeated reinforcement: following a behavior with a consequence (event) that INCREASES the probability that the behavior will be repeated Positive: something pleasant is added to the situation (e.g., candy) Positive: something pleasant is added to the situation (e.g., candy) Negative: something unpleasant is removed from the situation (e.g., no homework) Negative: something unpleasant is removed from the situation (e.g., no homework)

Language of Operant Conditioning punishment: following a behavior with a consequence (event) that DECREASES the probability that the behavior will be repeated punishment: following a behavior with a consequence (event) that DECREASES the probability that the behavior will be repeated Positive: something is added to the situation (e.g., a shock) Positive: something is added to the situation (e.g., a shock) Negative: something is removed from the situation (e.g., fine for speeding) Negative: something is removed from the situation (e.g., fine for speeding)

Four Operant Procedures 1. Positive Reinforcement: behavior results in something pleasant being added (behavior increases) 1. Positive Reinforcement: behavior results in something pleasant being added (behavior increases) – rat presses lever > rat gets food 2. Negative Reinforcement: behavior results in something unpleasant being removed (behavior increases) 2. Negative Reinforcement: behavior results in something unpleasant being removed (behavior increases) – rat presses lever > bigger aggressive rat is removed

Four Operant Procedures 3. Positive Punishment: behavior results in something unpleasant being added (behavior decreases) 3. Positive Punishment: behavior results in something unpleasant being added (behavior decreases) – rat presses lever > rat gets shocked 4. Negative Punishment: behavior results in something pleasant being removed (behavior decreases) 4. Negative Punishment: behavior results in something pleasant being removed (behavior decreases) – rat presses lever > rat’s food gets taken away

Superstitious Behavior Discovered by Skinner Discovered by Skinner Organism learns “incorrectly” that a behavior produces an outcome Organism learns “incorrectly” that a behavior produces an outcome Example: Example: – pigeon is turning as food pellet drops – pigeon “assumes” turning will bring food

Learned Helplessness Learned helplessness: may be one cause of human depression Learned helplessness: may be one cause of human depression Organism learns it has no control over situation (environment) Organism learns it has no control over situation (environment) Martin Seligman: demonstrated this using dogs and shock Martin Seligman: demonstrated this using dogs and shock

Learned Helplessness Study

Seligman’s Learned Helplessness Study Two groups of dogs are exposed to shock Two groups of dogs are exposed to shock – control group could escape shock – “no escape” group could NOT escape shock Later, when escape was possible, “no escape” dogs didn’t even try Later, when escape was possible, “no escape” dogs didn’t even try Learned that they had NO CONTROL Learned that they had NO CONTROL

Shaping shaping: complex behaviors are learned in small steps (“reinforcing successive approximations to a goal”) shaping: complex behaviors are learned in small steps (“reinforcing successive approximations to a goal”) any complex behavior (e.g., dog fetching a stick) is learned in small steps any complex behavior (e.g., dog fetching a stick) is learned in small steps

Response Acquisition Learning is slow at first, becomes more rapid, then levels off Learning is slow at first, becomes more rapid, then levels off Trials CR

Extinction Repeated presentation of CS without US weakens, then eliminates the CR Repeated presentation of CS without US weakens, then eliminates the CR Trials CR NO US (food)

Spontaneous Recovery After extinction, and then a period of rest, the CR returns without any additional conditioning After extinction, and then a period of rest, the CR returns without any additional conditioning Trials CR CR reappears

Generalization - Discrimination stimulus generalization: response is made to original AND “similar” CS stimulus generalization: response is made to original AND “similar” CS – Little Albert stimulus discrimination: organism learns to respond to original CS but NOT to similar ones stimulus discrimination: organism learns to respond to original CS but NOT to similar ones – pigeon learns to peck at red light but not at green light

Higher Order Conditioning Can a CS be used as if it were a US to condition a new second CS? Can a CS be used as if it were a US to condition a new second CS? YES YES A CS (bell) is used as a US to condition a new CS (light) to elicit the same CR (salivation) A CS (bell) is used as a US to condition a new CS (light) to elicit the same CR (salivation)

HIgher Order Conditioning Before Higher Order Conditioning Bell (CS1) Salivation (UR) Light (CS2) No Response

Higher Order Conditioning During HIgher Order Conditioning Light (CS2) Bell (CS1) Salivation (UR)

Higher Order Conditioning After Higher Order Conditioning Light (CS2) Salivation (CR)

Primary and SecondaryReinforcers primary reinforcers: naturally reinforcing food, water, sex secondary reinforcers: become reinforcing through association with primary reinforcers for humans, MONEY is the most potent secondary reinforcer

The Blocking Effect & Contingencies A problem for Pavlov: A problem for Pavlov: – Pavlov - Thought all that is needed for conditioning is for a CS and US to be presented together Leon Kamin: showed that it was not this simple Leon Kamin: showed that it was not this simple – prior conditioning of a tone with shock prevented rats from later being able to associate a light with the shock

Original Fear Conditioning Tone (CS) Shock (US) Fear (UR) Tone is paired with a shock

After Original Fear Conditioning Tone signals shock and elicits fear Tone (CS) Fear (CR)

Try to Condition Light Light is presented WITH tone AND shock Tone (CS) Fear (UR/CR) Light (CS) Shock (US) IMPORTANT! In contrast to “higher order conditioning,” the US “shock” is still present

Light has NOT Conditioned Light - shock association has NOT been learned Light does not tell the rat anything it doesn’t already know, rat ignores light Prior learning of the tone-shock relationship “BLOCKED” learning of the light-shock relationship Light (CS) NO Fear (CR)

Contingency Theory Explains the blocking effect Explains the blocking effect For conditioning to occur, the CS must provide “useful information” about the US to the organism For conditioning to occur, the CS must provide “useful information” about the US to the organism

Schedules of Reinforcement Continuous Reinforcement: each response is reinforced Continuous Reinforcement: each response is reinforced partial (intermittent) Reinforcement: reinforcement does not follow every response partial (intermittent) Reinforcement: reinforcement does not follow every response – ratio schedules: several responses are required before reinforcement – interval schedules: a certain amount of time must pass before next reinforcement

Fixed Ratio FR-10: FR-10: a set number of bar presses (10) must be performed to get food rat presses 10 times, gets food, rests, starts again – produces a “stepped” response record – ex. being paid for “piece work”

Variable Ratio VR-10: VR-10: number of bar presses needed for food will vary but will AVERAGE a certain number (10) Rat never knows when food is coming so he keeps pressing – produces a steady steep response record – ex. Playing a slot machine – **** “Highest” rate of responding ****

Fixed Interval FI-10: FI-10: food is available every 10 seconds IF bar is being pressed – rat senses time to as 10 sec. Mark approaches, he presses more quickly – produces a “scallop shaped” record – ex. more frequent trips to the employment office when more jobs are available

Variable Interval VI-10: VI-10: food is available ABOUT every 10 seconds but rat must be pressing bar as the time approaches – steady low rate response record – ex. studying for a teacher who gives “pop” quizzes

Cumulative records of a rat bar-pressing in a Skinner Box. The lines are made up of dots with each dot (moving upward) representing a bar press. If you want a lot of work from a rat or a person, use a variable ratio schedule!,

Latent Learning Latent Learning: learning that is not apparent in behavior Latent Learning: learning that is not apparent in behavior Demonstrated in an experiment by “Tolman and Honzik” Demonstrated in an experiment by “Tolman and Honzik” Problem for Skinner’s idea that learning requires “reinforcement” or “punishment” Problem for Skinner’s idea that learning requires “reinforcement” or “punishment”

Tolman and Honzik’s Study rats (3 groups) learned a maze over about 2 weeks rats (3 groups) learned a maze over about 2 weeks Group A: Group A: – no food at end, their speed through the maze did not increase muchat all Group B: Group B: – food at end daily, they ran the maze a little faster each day Group C: (actually a sub-group of A) Group C: (actually a sub-group of A) – no food till day 11, then they immediately ran the maze as fast as group B

Influence of Tolman’s Study It showed that learning CAN occur without “reinforcement” It showed that learning CAN occur without “reinforcement” Group C rats were learning about the maze but didn’t show it until given a good reason (food at end of maze) Group C rats were learning about the maze but didn’t show it until given a good reason (food at end of maze) Learning may be naturally reinforcing Learning may be naturally reinforcing So, there is more to operant learning than Skinner thought So, there is more to operant learning than Skinner thought

© Prentice Hall, 1999 Albert Bandura’s Social Learning Theory We can learn by watching We can learn by watching Observational (vicarious) learning - We observe the behaviors of others and the consequences of those behaviors. Observational (vicarious) learning - We observe the behaviors of others and the consequences of those behaviors. Vicarious reinforcement - If their behaviors are reinforced we tend to imitate the behaviors Vicarious reinforcement - If their behaviors are reinforced we tend to imitate the behaviors Vicarious punishment - If their behaviors are punished we tend NOT to imitate the behaviors Vicarious punishment - If their behaviors are punished we tend NOT to imitate the behaviors

the “Bobo Doll Study” - Bandura’s classic experiment demonstrating observational learning two groups of children watched an adult get either rewarded or punished for behaving aggressively with a doll children who saw the adult rewarded were later more likely to be aggressive when placed in the same situation Social Learning Theory (cont.)

Learning and Human Behavior

Classical Conditioning Explains how some phobias develop Explains how some phobias develop – dog is not feared – dog bites person – dog becomes CS for fear

Two Factor Theory of Avoidance O. H. Mowrer (not in book) O. H. Mowrer (not in book) – the two “factors” are classical and operant conditioning 1. person learns to fear a dog via classical conditioning 1. person learns to fear a dog via classical conditioning 2. each time dog is avoided, avoidance behavior is “reinforced” by? 2. each time dog is avoided, avoidance behavior is “reinforced” by? negative reinforcement negative reinforcement – avoidance (removal) of dog lowers anxiety so behavior is repeated