Conditioning: Classical and Operant Learning

Classical conditioning is learning in which a neutral stimulus becomes associated with a stimulus that already triggers a reflex, so that it triggers a similar response, while operant conditioning is learning in which the consequences of a behavior (reinforcement or punishment) change how often the behavior is repeated. In brief: in Pavlov’s experiments, dogs learned to salivate at the sound of a bell that was repeatedly followed by food. In Skinner’s experiments, a rat learned to press a lever because a food pellet followed. Both are forms of associative learning, in which an animal learns the relationship between events.

These two forms of learning explain how animals (including humans) acquire habits, fears, preferences and skills. This guide explains the vocabulary and the process of classical conditioning, operant conditioning with reinforcement and punishment, schedules of reinforcement, applications, compares the two types, works through a numerical example of acquisition and extinction and corrects common mistakes.

What is classical conditioning?

The Russian physiologist Ivan Pavlov discovered classical (Pavlovian) conditioning around 1900 while studying digestion in dogs. He noticed that the dogs began to salivate when they saw the assistant who fed them, before the food arrived. He then tested the effect systematically with a bell. The terms are:

• Unconditioned stimulus (US): a stimulus that automatically causes a reflex response, without learning. Food.
• Unconditioned response (UR): the automatic reaction to the US. Salivation to the food.
• Neutral stimulus (NS): a stimulus that causes no relevant response at first. The bell.
• Conditioned stimulus (CS): the neutral stimulus after it has been paired with the US, so that it causes a response by itself. The bell after training.
• Conditioned response (CR): the learned response to the CS. Salivation to the bell alone. It is usually similar to the UR, but often weaker.

The process: (1) before conditioning, food → salivation (US → UR), bell → no salivation; (2) during conditioning, bell + food → salivation, repeated several times; (3) after conditioning, bell alone → salivation (CS → CR). The CS works best when it comes just before the US (by a fraction of a second to a few seconds), and when it predicts the US reliably. Rescorla and Wagner showed that learning depends on how surprising the US is, and not only on how often the two appear together.

What are acquisition, extinction and generalization?

• Acquisition: the initial stage of learning, when the CR gets stronger with each pairing.
• Extinction: if the CS is presented repeatedly without the US, the CR gradually weakens and disappears. It does not erase the learning. After a rest, the CR may return (spontaneous recovery).
• Generalization: stimuli similar to the CS also cause the CR: a dog trained with a bell of one pitch also salivates to other pitches.
• Discrimination: the animal learns to respond to the CS but not to similar stimuli that are not paired with the US.

Classical conditioning explains many emotional responses. In the famous but ethically unacceptable “Little Albert” experiment of John Watson and Rosalie Rayner in 1920, a baby was conditioned to fear a white rat by pairing it with a loud noise. Phobias, food aversions and the craving of drug users when they see cues linked with drug use can result from conditioning. Taste aversion is a special form: after nausea following the eating of a certain food, even a single experience can produce lasting avoidance, even if the nausea occurs hours later. John Garcia’s research on this in the 1950s and 1960s showed that animals are biologically prepared to learn some associations more easily than others.

What is operant conditioning?

Edward Thorndike studied cats escaping from puzzle boxes in the 1890s and formulated the law of effect: behaviors followed by satisfying consequences are likely to be repeated, and those followed by unpleasant consequences are less likely to be repeated. B. F. Skinner developed this in the 1930s with the “Skinner box”, a cage with a lever that delivers food, and named the process operant conditioning, because the animal “operates” on its environment. The key is the consequence of the behavior:

• Positive reinforcement: adding a pleasant stimulus after a behavior increases the behavior. A pellet of food after the lever press; praise after good work.
• Negative reinforcement: removing an unpleasant stimulus after a behavior increases the behavior. A rat presses a lever to turn off an electric shock; a person puts on a seat belt to stop an annoying alarm. It is not punishment.
• Positive punishment: adding an unpleasant stimulus after a behavior decreases the behavior. A scolding.
• Negative punishment: removing a pleasant stimulus after a behavior decreases the behavior. Taking away a toy.

“Positive” and “negative” mean adding or removing a stimulus, not good or bad. Reinforcement always increases behavior, and punishment always decreases it. Skinner and later researchers found that reinforcement is generally more effective than punishment for changing behavior, because punishment can cause fear and does not teach the right behavior. Shaping trains complex behaviors by reinforcing successive approximations: first the animal is rewarded for turning toward the lever, then for touching it, then for pressing it. Animal trainers use shaping with dolphins, dogs and pigeons.

What are schedules of reinforcement?

The pattern of reinforcement affects how quickly learning occurs and how resistant the behavior is to extinction.

• Continuous reinforcement: every response is reinforced. Learning is fast, but it is quickly extinguished when the reward stops.
• Fixed-ratio: a reward after a fixed number of responses (for example, every 10th). Gives a high rate with a short pause after each reward.
• Variable-ratio: a reward after an unpredictable number of responses (on average 10). Gives the highest and steadiest rate, and the greatest resistance to extinction. Slot machines work in this way.
• Fixed-interval: the first response after a fixed time is rewarded. The rate rises toward the end of each interval (as when students study before exams).
• Variable-interval: the first response after an unpredictable time is rewarded. Gives a slow, steady rate (such as checking messages).

Classical and operant conditioning compared

Where are these principles applied?

• Animal training: clicker training with a conditioned reinforcer, guide dogs, circus animals, and farm and zoo animals trained to cooperate in veterinary care.
• Education: praise, rewards and feedback; classroom management.
• Therapy: systematic desensitization for phobias (extinguishing a fear response by gradual exposure), aversion therapy, token systems in behavioral therapy and the management of habits.
• Advertising and design: the pairing of products with pleasant images (classical), and rewards and badges in apps and games (operant).
• Pest and wildlife management: conditioned taste aversion has been used to teach coyotes to avoid sheep, with meat that makes them sick.

Worked example: acquisition and extinction data

In a classical conditioning experiment, a dog is given a tone for 5 seconds and then food. After every 5 trials, the volume of saliva (in drops) during a tone-only test is measured. Then the tone is presented 5 times in each block without food (extinction).

1. Acquisition: the number of drops rises from 0 to 12 over 20 pairings. The increase is fast at first (6 drops between trials 5 and 10) and slower later (4 drops between trials 10 and 20), a typical learning curve that levels off.
2. Extinction: the CR falls from 12 to 6 and then to 1, as the tone is no longer followed by food. The fall is 11 drops out of 12 (92 percent).
3. Spontaneous recovery: after a rest, the response returns to 4 drops, which is one third of the maximum (4 ÷ 12). This shows that extinction was not complete unlearning, and that the animal retained some of the association.

The same shape applies in operant conditioning. If a rat is rewarded with a pellet each time that it presses a lever, the rate may go from 1 press per minute to 10 per minute within a few sessions, and if the pellets are stopped, the rate falls gradually back toward 1 per minute. With a variable-ratio schedule, the rate would decrease more slowly after the stop of the rewards than with continuous reinforcement. The data in the table are illustrative.

How were these forms of learning discovered?

Ivan Pavlov received the 1904 Nobel Prize in Physiology or Medicine for his work on digestive glands, and his study of conditioned reflexes followed. Edward Thorndike published his animal intelligence studies in 1898. John B. Watson promoted behaviorism in 1913, the view that psychology should study observable behavior. B. F. Skinner published The Behavior of Organisms in 1938. The cognitive revolution of the 1960s showed that learning also involves mental processes, as in Edward Tolman’s cognitive maps in rats. Today, researchers study the neural basis: dopamine neurons signal reward prediction errors, linking conditioning to brain function.

Which mistakes are common?

Mistake 1: Confusing negative reinforcement with punishment

Negative reinforcement removes an unpleasant stimulus and increases a behavior. Punishment decreases a behavior.

Mistake 2: Mixing up the US and CS

The US triggers the response without learning (food). The CS is the learned trigger (bell).

Mistake 3: Thinking that extinction erases the learning

Spontaneous recovery shows that some of the association remains.

Mistake 4: Using “positive” to mean “good”

In conditioning, positive means that something is added, and negative that something is removed.