1. Chapter 6 Learning Through Conditioning

2. Classical conditioning
Pavlov’s Demonstration
Terminology
Unconditioned Stimulus (UCS)
Conditioned Stimulus (CS)
Unconditioned Response (UCR)
Conditioned Response (CR)
Classical conditioning explains how a neutral stimulus can acquire the capacity to elicit (or draw forth) a response originally elicited by another stimulus.
Ivan Pavlov, a prominent Russian physiologist in the early 1900’s, who did Nobel prize winning research on digestion, discovered partly by accident that dogs will salivate in response to the sound of a tone. In doing so, he discovered classical, sometimes called Pavlovian, conditioning.
In classical conditioning, the UCS is a stimulus that elicits an unconditioned response without previous conditioning…Pavlov’s meat powder.
The UCR is an unlearned reaction to a UCS that occurs without previous conditioning…salivating.
The CS is a previously neutral stimulus that has acquired the capacity to elicit a conditioned response...the sound of a tone.
The CR is a learned reaction to a conditioned stimulus…salivating to the tone.

3. Figure 6.1
Classical conditioning apparatus. An experimental arrangement similar to the one depicted here (taken from Yerkes & Morgulis, 1909) has typically been used in demonstrations of classical conditioning, although Pavlov’s original setup (see inset) was quite a bit simpler. The dog is restrained in a harness. A tone is used as the conditioned stimulus (CS), and the presentation of meat powder is used as the unconditioned stimulus (UCS). The tube inserted into the dog’s salivary gland allows precise measurement of its salivation response. The pen and rotating drum of paper on the left are used to maintain a continuous record of salivary flow. (Inset) The less elaborate setup that Pavlov originally used to collect saliva on each trial is shown here (Goodwin, 1991).

4. Figure 6.2
The sequence of events in classical conditioning. Moving from top to bottom, this series of diagrams outlines the sequence of events in classical conditioning, using Pavlov’s original demonstration as an example. As we encounter other examples of classical conditioning throughout the book, we will see many diagrams like the one in the fourth panel, which summarizes the process.

5. Figure 6.3
Classical conditioning of a fear response. Many emotional responses that would otherwise be puzzling can be explained by classical conditioning. In the case of one woman’s bridge phobia, the fear originally elicited by her father’s scare tactics became a conditioned response to the stimulus of bridges.

6. Classical Conditioning: Terminology Continued
Trial = pairing of UCS and CS
Acquisition = initial stage in learning
Stimulus contiguity = occurring together in time and space
3 types of Classical Conditioning
Simultaneous conditioning: CS and UCS begin and end together
Short-delayed conditioning: CS begins just before the UCS, end together
Trace conditioning: CS begins and ends before UCS is presented
In classical conditioning research, a trial is a pairing of the UCS and the CS. (How many times have the tone and the meat powder been paired?) Some behaviors are learned after only one trial or pairing, while others take many trials.
Acquisition, refers to the initial stage of learning a response…acquiring the response.
Conditioning has been shown to depend on stimulus contiguity, that is, the occurring of stimuli together in time and space.
So when do you sound the tone in a classical conditioning task? What works best? Of the three types of conditioning (simultaneous, short-delayed, and trace), short-delayed conditioning appears to most promote acquisition of a classically conditioned response…ideally the delay should be very brief, about ½ a second.

7. Figure 6.9
Temporal relations of stimuli in classical conditioning. The effects of classical conditioning depend in part on the timing of the stimuli. Three ways of pairing the CS and UCS are diagrammed here. The most effective arrangement is short-delayed conditioning, in which the CS begins just before the UCS and stops at the same time as the UCS.

8. Processes in Classical Conditioning
Extinction
Spontaneous Recovery
Stimulus Generalization
Discrimination
Higher-order conditioning
Extinction occurs when the CS and UCS are no longer paired and the response to the CS is weakened. We know that the response is still there, just not being active, because of spontaneous recovery. That is, an extinguished response may reappear after a period of non- pairing.
Generalization occurs when conditioning generalizes to additional stimuli that are similar to the CS, for example, Watson and Rayner’s study with Little Albert, who was conditioned to fear a white rat, but later came to be afraid of many white, furry objects.
Discrimination is the opposite of generalization…that is, the response is to a specific stimulus; similar stimuli don’t work.
Higher order conditioning occurs when a CS functions as if it were a UCS, to establish new conditioning…condition to respond to a tone with saliva, pair the tone with a light.

9. Figure 6.6
Classical conditioning of immunosuppression. When a neutral stimulus is paired with a drug that chemically causes immunosuppression, it can become a CS that elicits immunosuppression on its own. Thus, even the immune response can be influenced by classical conditioning.

11. Figure 6.12
Generalization gradients. In a study of stimulus generalization, an organism is typically conditioned to respond to a specific CS, such as a 1200 hertz tone, and then tested with similar stimuli, such as other tones between 400 and 2000 hertz. Graphs of the organisms’ responding are called generalization gradients. The graphs normally show, as depicted here, that generalization declines as the similarity between the original CS and the new stimuli decreases. When an organism gradually learns to discriminate between a CS and similar stimuli, the generalization gradient tends to narrow around the original CS.

12. Figure 6.13
Higher-order conditioning. Higher-order conditioning is a two-phase process. In the first phase, a neutral stimulus (such as a tone) is paired with an unconditioned stimulus (such as meat powder) until it becomes a conditioned stimulus that elicits the response originally evoked by the UCS (such as salivation). In the second phase, another neutral stimulus (such as a red light) is paired with the previously established CS, so that it also acquires the capacity to elicit the response originally evoked by the UCS.

14. Operant Conditioning or Instrumental Learning
Edward L. Thorndike (1913) – the law of effect
B.F. Skinner (1953) – principle of reinforcement
Operant chamber
Emission of response
Reinforcement contingencies
Cumulative recorder
Thorndike’s law of effect stated that if a response in the presence of a stimulus leads to satisfying effects, the association between the stimulus and the response is strengthened.
This law became the cornerstone of Skinner’s theory.
Skinner’s principle of reinforcement holds that organisms tend to repeat those responses that are followed by favorable consequences or reinforcement.
Skinner defined reinforcement as when an event following a response increases an organism’s tendency to make that response.
Skinner created a prototype experimental procedure, using animals and an operant chamber or “Skinner box”. This is a small enclosure in which an animal can make a specific response that is recorded while the consequences of the response are systematically controlled. Rats, for example, press a lever.
Because operant responses tend to be voluntary, they are said to be emitted rather than elicited.
Reinforcement contingencies are the circumstances or rules that determine whether responses lead to the presentation of reinforcers.
The cumulative recorder creates a graphic record of responding and reinforcement in a Skinner box as a function of time.

15. Figure 6.14
The learning curve of one of Thorndike’s cats. The inset shows one of Thorndike’s puzzle boxes. The cat had to perform three separate actions to escape the box, including depressing the pedal on the right. The learning curve shows how the cat’s escape time declined gradually over a number of trials.

16. Figure 6.16
Skinner box and cumulative recorder. (a) This diagram highlights some of the key features of an operant chamber, or Skinner box. In this apparatus designed for rats, the response under study is lever pressing. Food pellets, which may serve as reinforcers, are delivered into the food cup on the right. The speaker and light permit manipulations of visual and auditory stimuli, and the electric grid gives the experimenter control over aversive consequences (shock) in the box.

18. Launch
Video
Figure 6.17
A graphic portrayal of operant responding. The results of operant conditioning are often summarized in a graph of cumulative responses over time. The insets magnify small segments of the curve to show how an increasing response rate yields a progressively steeper slope (bottom); a high, steady response rate yields a steep, stable slope (middle); and a decreasing response rate yields a progressively flatter slope (top).

19. Basic Processes in Operant Conditioning
Acquisition
Shaping
Extinction
Stimulus Control
Generalization
Discrimination
As in classical conditioning, acquisition refers to the initial stage of learning.
Learning operant responses usually occurs through a gradual process called shaping, which consists of the reinforcement of closer and closer approximations of a desired response…key in pet tricks.
Extinction in operant conditioning refers to the gradual weakening and disappearance of a response tendency because the response is no longer followed by a reinforcer…stop giving food when the rat presses the lever…results in a brief surge of responding followed by a gradual decline until it approaches zero.
Stimuli that precede a response can exert considerable influence over operant behavior, basically becoming “signals” that a reinforcer is coming. Discriminative stimuli are cues that influence operant behavior by indicating the probable consequences of a response (ex. slow down when the highway is wet, ask Mom when she’s in a good mood, etc.).
Discrimination occurs when an organism responds to one stimulus, but not another one similar to it, while generalization occurs when a new stimulus is responded to as if it were the original. (ex. cat runs to the sound of a can-opener which signals food, but not to the sound of the mixer…discrimination…get a new blender, cat runs to it...generalization).

20. Fig 6.18
Operant conditioning shapes behavior as a sculptor shapes a lump of clay.

21. Reinforcement: Consequences that Strengthen Responses
Delayed Reinforcement
Longer delay, slower conditioning
Primary Reinforcers
Satisfy biological needs
Secondary Reinforcers
Conditioned reinforcement
Skinner said that reinforcement occurs whenever an outcome strengthens a response…this definition avoids the use of “experience of pleasure”, which is an unobservable feeling.
In operant conditioning, immediate reinforcement produces the fastest conditioning.
Operant theorists distinguish between primary reinforcers, which are events that are inherently reinforcing because they satisfy biological needs, and secondary reinforcers, which are events that acquire reinforcing qualities by being associated with primary reinforcers.
Primary reinforcers in humans include food, water, warmth, sex, and maybe affection expressed through hugging and close bodily contact.
Secondary reinforcers in humans include things like money, good grades, attention, flattery, praise, and applause.

22. Schedules of Reinforcement
Continuous reinforcement
Intermittent (partial) reinforcement
Ratio schedules
Fixed
Variable
Interval schedules
A schedule of reinforcement determines which occurrences of a specific response result in the presentation of a reinforcer.
Continuous reinforcement occurs when every instance of a designated response is reinforced (faster acquisition, faster extinction).
Intermittent reinforcement occurs when a designated response is reinforced only some of the time (greater resistance to extinction).
Ratio schedules require the organism to make the designated response a certain number of times to gain each reinforcer. A fixed-ratio schedule entails giving a reinforcer after a fixed number of nonreinforced responses. A variable ratio schedule entails giving a reinforcer after a variable number of nonreinforced responses.
Interval schedules require a time period to pass between the presentation of reinforcers. A fixed-interval schedule entails reinforcing the first response that occurs after a fixed time interval has elapsed. A variable-interval schedule entails giving the reinforcer for the first response after a variable time interval has elapsed.
More than 50 years of research on these schedules has yielded an enormous amount of information about how organisms respond to different schedules.
Figure 6.19, presented on the next slide, depicts some of what has been discovered.

23. Figure 6.19
Schedules of reinforcement and patterns of response. Each type of reinforcement schedule tends to generate a characteristic pattern of responding. In general, ratio schedules tend to produce more rapid responding than interval schedules (note the steep slopes of the FR and VR curves). In comparison to fixed schedules, variable schedules tend to yield steadier responding (note the smoother lines for the VR and VI schedules on the right) and greater resistance to extinction.

26. Consequences: Reinforcement and Punishment
Increasing a response:
Positive reinforcement = response followed by rewarding stimulus
Negative reinforcement = response followed by removal of an aversive stimulus
Escape learning
Avoidance learning
Decreasing a response:
Punishment
Problems with punishment
Responses can be strengthened either by presenting positive reinforcers or by removing negative reinforcers.
Negative reinforcement regulates escape and avoidance learning. In escape learning, an organism learns to perform a behavior that decreases or ends aversive stimulation (turning on the air conditioner). In avoidance learning an organism learns to prevent or avoid some aversive stimulation (turn on the a/c before it gets too hot).
Punishment occurs when an event following a response weakens the tendency to make that response. Punishment is much more than disciplinary procedures…wear a new outfit and friends laugh…punishing.
Some of the problems associated with punishment are an increase in aggressive behavior and suppression of behavioral activity.

27. Figure 6.21
Positive reinforcement versus negative reinforcement. In positive reinforcement, a response leads to the presentation of a rewarding stimulus. In negative reinforcement, a response leads to the removal of an aversive stimulus. Both types of reinforcement involve favorable consequences and both have the same effect on behavior: The organism’s tendency to emit the reinforced response is strengthened.

28. Figure 6.22
Escape and avoidance learning.  (a) Escape and avoidance learning are often studied with a shuttle box like that shown here. Warning signals, shock, and the animal’s ability to flee from one compartment to another can be controlled by the experimenter. (b) According to Mowrer’s two-process theory, avoidance begins because classical conditioning creates a conditioned fear that is elicited by the warning signal (panel 1). Avoidance continues because it is maintained by operant conditioning (panel 2). Specifically, the avoidance response is strengthened through negative reinforcement, since it leads to removal of the conditioned fear.

29. Figure 6.23
Comparison of negative reinforcement and punishment. Although punishment can occur when a response leads to the removal of a rewarding stimulus, it more typically involves the presentation of an aversive stimulus. Students often confuse punishment with negative reinforcement because they associate both with aversive stimuli. However, as this diagram shows, punishment and negative reinforcement represent opposite procedures that have opposite effects on behavior.

30. Changes in Our Understanding of Conditioning
Biological Constraints on Conditioning
Instinctive Drift
Conditioned Taste Aversion
Preparedness and Phobias
Cognitive Influences on Conditioning
Signal relations
Response-outcome relations
New research has greatly changed the way we think about conditioning, with both biological and cognitive influences having been discovered.
Instinctive drift occurs when an animal’s innate response tendencies interfere with conditioning (the raccoon who would rather rub the coins together than obtain the reinforcer).
Conditioned taste aversions can be readily acquired, after only one trial and when the stimuli are not contiguous (i.e. becoming ill occurs hours after eating a food), suggesting that there is a biological mechanism at work.
Martin Seligman has outlined the fact that some phobias are more easily conditioned than others, suggesting the concept of preparedness…that we are biologically prepared to learn to fear objects or events that have inherent danger.
Signal relations theory (Rescorla) illustrates that the predictive value of a CS is an influential factor governing classical conditioning.
Response-outcome relations holds that when a response is followed by a desired outcome, it is more easily strengthened if it seems that it caused the outcome (predicts)…you study for an exam and listen to Smash Mouth…you make an A. What is strengthened, studying or listening to Smash Mouth?
Signal relations and response-outcome research suggest that cognitive processes play a larger role in conditioning.

31. Figure 6.24
Conditioned taste aversion. Taste aversions can be established through classical conditioning, as in the “sauce béarnaise syndrome.” However, as the text explains, taste aversions can be acquired in ways that seem to violate basic principles of classical conditioning.

32. Figure 6.25
Garcia and Koelling’s research on conditioned taste aversion. In a landmark series of studies, Garcia and Koelling (1966) demonstrated that some stimulus-response associations are much easier to condition than others. (a) Their procedure allowed them to pair a taste stimulus (saccharin-flavored water) with visual and auditory stimuli (a bright light and noisy buzzer), and/or pain-inducing shock or nausea-inducing radiation. (b) They found that taste-nausea associations were acquired easily, as were associations between auditory-visual stimuli and pain, whereas other associations were difficult to acquire. As your text discusses, they explained their findings in terms of evolutionary considerations.

34. Observational Learning: Basic Processes
Albert Bandura (1977, 1986)
Observational learning
Vicarious conditioning
4 key processes
attention
retention
reproduction
motivation
acquisition vs performance
Albert Bandura outlined the theory of observational learning. In observational learning, vicarious conditioning occurs by an organism watching another organism (a model) be conditioned. Observational learning can occur for both classical and operant conditioning.
In order for observational learning to take place, 4 key processes are at work. First the organism must pay attention to the model, retain the information observed, and be able to reproduce the behavior. Finally, an observed response is unlikely to be reproduced unless the organism is motivated to do so, i.e., they believe there will be a pay off.
Bandura distinguishes between acquisition (having the response in your repertoire) and performance (actually engaging in the behavior). Bandura asserts that reinforcement usually influences already acquired responses, more than the acquisition of new responses.

35. Figure 6.26
Observational learning. In observational learning, an observer attends to and stores a mental representation of a model’s behavior (example: assertive bargaining) and its consequences (example: a good buy on a car). If the observer sees the modeled response lead to a favorable outcome, the observer’s tendency to emit the modeled response will be strengthened.

36. Figure 6.27
Steps in a self-modification program. This flowchart provides an overview of the steps necessary to execute a self-modification program.