Part I · 2020. 1. 16. · Razran (1957 ), in summarizing 161 Russian studies relevant to effects of CS intensity, concluded that there was abundant evidence that increasing CS intensity

Basic Classical Conditioning Part I

COPYRIG

HTED M

ATERIAL

1

Principles of Pavlovian Conditioning

Description, Content, Function Daniel A. Gottlieb and Elizabeth L. Begej

Although both early human memory researchers and behaviorists studied processes that today would be considered Pavlovian, Ivan Pavlov is credited with discovering classical conditioning and fi rst offi cially describing the process to an English-speaking audience with the publication of Conditioned refl exes in 1927. Pavlov ’ s systematic sinvestigation of Pavlovian conditioning uncovered most of the primary phenomena, and his sharp and nuanced discussions are still relevant today.

Gottlieb ( 2011 ) defi ned Pavlovian, or classical, conditioning as the “adjustments organisms make in response to observing the temporal relations among environmen-tal or proprioceptive stimuli.” The most well-known form involves pairing a neutral conditioned stimulus (CS) with a biologically relevant unconditioned stimulus (US) that automatically elicits an unconditioned response (UR), leading the CS to elicit aconditioned response (CR) qualitatively similar to the UR. Pavlov ’ s fi rst reported example used dogs as subjects, a metronome CS, food as the US, and salivation asthe UR and CR ( Pavlov, 1927 , p. 26).

Pavlovian conditioning is most clearly defi ned and constrained by its method, which involves maintaining strict control over the presentation of stimuli. Thereappear to be few clear principles that distinguish Pavlovian conditioning from other forms of associative learning, such as instrumental learning and human associative memory, and we agree with Rescorla and Solomon ( 1967 ) ’ s conclusion the Pavlovian conditioning is most distinct in how determined the form of the learned response is by the choice of US. Pavlovian responding is also characteristically resistant to instru-mental contingencies. For example, it is diffi cult to prevent a pigeon from pecking (CR) at a discrete key light (CS) paired with food (US), even if pecking prevents the delivery of food ( Williams & Williams, 1969 ).

The Wiley Blackwell Handbook of Operant and Classical Conditioning, First Edition.ggEdited by Frances K. McSweeney and Eric S. Murphy.© 2014 John Wiley & Sons, Ltd. Published 2014 by John Wiley & Sons, Ltd.

4 Daniel A. Gottlieb and Elizabeth L. Begej

Overview

Basic Excitatory Phenomena

Acquisition is the primary phenomenon of Pavlovian conditioning and refers to the growth in conditioned responding resulting from pairing a CS and US over time( Pavlov, 1927 , p. 26). A CS that produces conditioned responding is sometimes referred to as an excitor. An example of an excitor is a brief tone that, due to repeated pairings with an air puff to the eye, elicits anticipatory eyeblinking.

An animal that has learned to blink to a tone may also blink when presented with a novel auditory stimulus. The magnitude of this generalized responding is a functionof the similarity of the new stimulus to the originally trained CS ( Pavlov, 1927 ,p. 111). Generalization functions can be modifi ed by discrimination learning, in which some stimuli are reinforced while others are nonreinforced. For example, thegeneralized responding from a tone to a burst of white noise can be reduced by interspersing presentations of the noise alone.

A tone paired with an air puff to the eye gains more than the ability to elicit anticipatory blinking. It also develops the ability to serve as a conditioned reinforcerfor another, second-order, CS ( Pavlov, 1927 , p. 33). Second-order conditioning tends to lead to lower levels of responding than does fi rst-order conditioning. Thus,a light paired with a tone excitor will likely elicit less conditioned eyeblinking than will the tone itself. Conditioned reinforcers are contrasted with primary reinforcers that do not need prior training to be able to establish conditioned responding. Unlike in instrumental conditioning, Pavlovian reinforcers refer to stimuli that may be appe-titive or aversive.

Basic Inhibitory Phenomena

Inhibitory phenomena are those that manifest in opposition to conditioned respond-ing. When stimuli both reduce responding to simultaneously presented excitors andare slow to become excitors themselves, as compared to neutral stimuli, they are called inhibitors ( Rescorla, 1969a ). This type of inhibition is sometimes referred to as operational inhibition. It is contrasted with the theoretical inhibition that is used as an explanation for transient decreases in excitatory responding.

Extinction refers to the loss in conditioned responding that occurs when an excitor is subsequently presented in a manner that breaks the CS-US relationship ( Pavlov, 1927 , p. 49). For example, our tone excitor will stop eliciting conditioned eye-blinks if it is repeatedly presented alone. Although extinction is explained by appeal to inhibitory mechanisms, extinguished stimuli do not typically become operational inhibitors ( Rescorla, 1969a ).

Suppression of responding may also be observed when a nonreinforced stimulus is interspersed among the reinforced trials of another CS and a US. For example, if noise alone presentations are interspersed among tone-air puff pairings, generalized responding to the noise will be reduced. The suppression of responding observed in a discrimination procedure is referred to as differential inhibition. Unlike extin-guished stimuli, differentially conditioned stimuli may become operational inhibitors ( Pavlov, 1927 , p. 120).

Principles of Pavlovian Conditioning 5g

A particularly powerful form of differential inhibition results from a conditioned inhibition procedure in which a stimulus is nonreinforced in the presence of an excitorthat is reinforced when presented alone. For example, if our tone signals an air puff except when it is presented together with a light, the light will become a conditionedinhibitor capable of passing the operational tests for inhibition ( Pavlov, 1927 , p. 68).

Perhaps it is not surprising that with extended training, organisms may come to withhold responding to a CS until closer to the time of the US. The decrease inresponding to early parts of a CS that may result from extended training is referred to as inhibition of delay ( Pavlov, 1927 , p. 88).

Basic Framework

Early theorists most often adopted a contiguity view of learning, in which the pairing of stimuli in time and space was the necessary and suffi cient condition for generating conditioned responding. Although Hull ( 1943 ) expanded on this view inarguing that drive reduction was also necessary for learning, it is his other contribu-tions that have made him central to contemporary understandings of Pavlovian processes.

Hull ( 1943 ) adopted the view of Pavlovian conditioning as an associative process by which the magnitude of the association determined the magnitude of conditioned responding. He presented a simple mathematical learning rule that specifi ed the change in associative (habit) strength that resulted from the pairing of a CS and a US. The vast majority of subsequent quantitative models developed within the asso-ciative framework contain Hull ’ s simple learning algorithm ( Mackintosh, 1975 ; Pearce, 1987 ; Rescorla & Wagner, 1972 ; Wagner, 1981 ).

Hull ( 1943 ) helped to cement a view of Pavlovian conditioning as an incremental trial-based process involving changes in the associative strength between stimuli. Theparameters of his model are what were originally thought to be the determinants of Pavlovian learning. Primary was the number of trials. Secondary included three dura-tions related to the CS-US interval: the length of the CS, the temporal gap between the CS and US, and the extension of the CS past the US; as well as two salienceparameters, one for the CS and one for the US.

Hull ’ s Choices

CS - US Interval

Delay Conditioning. Most demonstrations of acquisition use a delay conditioning procedure where CS offset coincides with US onset. As such, CS duration is the only determinant of the CS-US interval in delay conditioning. There appear to be twoconfl icting ideas related to effects of CS duration. The fi rst is that there is an optimal CS-US interval that is highly procedure-dependent. For skeletal muscle CRs, likeeyeblink conditioning, the optimal CS-US interval is often a fraction of a second( Gormezano & Kehoe, 1981 ), but for other types of CRs, such as sign-tracking and conditioned fear, the optimal interval may be multiple seconds ( Gibbon, Baldock, Locurto, Gold, & Terrace, 1977 ; Yeo, 1974 ).


That there is an optimal CS-US interval confl icts with the other main idea about CS duration, that shortening it promotes acquisition ( Lattal, 1999 ). This confl ict islikely a result of optimal durations being at the low end of the range of durations that can establish conditioned responding. What appears most relevant is that certain procedures allow for conditioning with CS-US intervals much longer than others. In fear conditioning, CS durations of several minutes can be effective ( Kamin, 1965 ), and in conditioned taste aversion, CS-US trace intervals of up to 24 hours can estab-lish conditioned aversions ( Etscorn & Stephens, 1973 ).

Trace Conditioning. When the CS terminates prior to US onset, acquisition of conditioned responding may be strongly impaired ( Ellison, 1964 ). This procedure isknown as trace conditioning, referring to one explanation for the conditioned responding that does occur: A memory trace of the CS is contiguously paired with the US. Although a gap between CS offset and US onset impairs conditionedresponding, the impairment does not necessarily refl ect a failure to learn. Trace con-ditioning can lead a CS to become inhibitory ( LoLordo & Fairliss , 1985 ). It alsomay lead to increasing levels of responding during the time between CS offset andUS onset ( Marlin, 1981 ).

Simultaneous Conditioning. Early research suggested that simultaneous CS-US presentation did not lead to conditioned responding ( Kimble, 1961 ), but it is clear that in many cases it does ( Heth & Rescorla, 1973 ). Simultaneous conditioning may lead to different forms of learning than does forward conditioning. For example, in fl avor conditioning, simultaneous, but not delay, conditioning prevents the reacquisi-tion of an extinguished preference ( Higgins & Rescorla, 2004 ).

Backward Conditioning. Backward conditioning refers to presenting the US prior to presenting the CS. Typically, US offset and CS onset coincide. After a small number of trials, it is not uncommon to observe excitatory conditioned responding,but after further trials, the responding diminishes and the CS may become an inhibi-tor ( Heth, 1976 ). One complexity is that a backward CS may pass an operational test for inhibition while at the same time establishing excitatory second-order con-ditioning to another CS ( Barnet & Miller, 1996 ). Romaniuk and Williams ( 2000 ) provided an explanation for this by demonstrating that the initial 3 s of a 30 s back-wards CS was excitatory while the remaining 27 s was inhibitory. Adding a 3 s gap between US offset and CS onset eliminated excitatory responding, strengthened inhibition, and prevented second-order conditioning.

Stimulus Intensity

US Magnitude. Some argue that acquisition of conditioned responding should be broken down into components that include the point of acquisition and the asymp-totic magnitude of responding (Gallistel, Fairhurst, & Gibbon, 2004). In general, increasing US intensity or number tends to increase the magnitude of conditioned responding ( Mackintosh, 1974 , pp. 70–71); however, although Morris and Bouton ( 2006 ) detected effects of US magnitude on the point of acquisition in aversive and appetitive procedures with rats, how strongly US magnitude infl uences the point of acquisition is still not clear. Unlike US intensity, the effects of increasing US durationare not consistent across procedures ( Mackintosh, 1974 ).


CS Intensity. Razran ( 1957 ), in summarizing 161 Russian studies relevant toeffects of CS intensity, concluded that there was abundant evidence that increasing CS intensity increased both the rate of acquisition and the asymptotic magnitude of performance. A caveat was that when CS intensity was very high, performance was sometimes impaired. Outside of Russia, results of varying CS intensity have been more equivocal ( Mackintosh, 1974 , pp. 41–45).

Mackintosh ( 1974 ) suggests that effects of CS intensity, like those of US duration, are more likely to be observed in within-subject than in between-subject experiments, perhaps explaining some of the earlier Russian fi ndings. This explanation is consistent with fi ndings showing that when stimuli are presented in a way in which they are available for comparison, higher valued ones are augmented (positive contrast) while lower valued ones are diminished (negative contrast) ( Flaherty, 1996 ).

Form of Acquisition

A second aspect of Hull ’ s ( 1943 ) model was that it predicted a particular geometric form of growth. Explicit is the mathematical form. Implicit is the gradual and increas-ingly slow progression of learning. The left side of Figure 1.1 shows a group-averagedlearning curve for 16 pigeons in a 32 trials/day autoshaping procedure. Plotted is the rate/min of keypecking to a 5 s keylight CS that immediately preceded a 5 s access to grain US. The right side of Figure 1.1 shows a learning curve for one of the 16 pigeons. Both the learning curves start with a period characterized by the absenceof conditioned responding, followed by a growth in conditioned responding until an asymptotic level is reached. The curves differ in that the individual curve shows a longer latency before conditioned responding emerges and a steeper rise. Gallistel et al. ( 2004 ) demonstrated that over a wide range of Pavlovian procedures,

Figure 1.1 Left: group-averaged learning curve for 16 pigeons in a 32 trials/day autoshapingprocedure. Right: learning curve for 1 of the 16 pigeons.


group-averaged learning curves obscure the relative abruptness of individual learningcurves. This abruptness suggests that two separate components of the learning curvecan be meaningfully distinguished, the point of acquisition and the asymptotic mag-nitude of conditioned responding.

Trial Repetition

Repeatedly pairing a CS and a US leads to acquisition of conditioned responding.This does not mean that the number of repetitions is a primary determinant; increas-ing the number of trials is almost always accompanied by a corresponding increasein cumulative training time and number of sessions. It is reasonable to ask what isknown about each of these factors in isolation.

Training Time. It is well established that increasing training time by lengthening the intertrial interval (ITI) increases the likelihood that an animal will develop con-ditioned responding, speeds acquisition, and promotes higher levels of conditionedresponding. A particularly striking example is a quantitative analysis of the effects of ITI on pigeon autoshaping ( Gibbon et al., 1977 ), replotted in Figure 1.2 . The left side shows trials to criterion as a function of ITI. The right side shows hours of cumulative ITI to criterion. Cumulative ITI more strongly predicted rate of acquisi-tion than did number of trials.

Number of Sessions. Kehoe and Macrae ( 1994 ) presented the most impressiveevidence that increasing the number of sessions independently promotes acquisition. Their analysis of the literature that showed that rabbit eyeblink conditioning could be either fast or slow to develop depending on the number of trials per session. Theleft panel of Figure 1.3 replots their best-fi tting function relating trials per session to trials to acquisition along with a new curve relating trials per session to sessions tocriterion. As with cumulative ITI, number of sessions more strongly predicted rate of acquisition than did number of trials.

Number of Trials. Gottlieb ( 2008 ) assessed for the effects of number of trials on acquisition of conditioned approach behavior in mice and rats by varying the number of trials animals received in daily sessions of the same duration. In between-subject experiments, there was little evidence that up to 16-fold differences in the number

Figure 1.2 Replot of pigeon autoshaping data from Gibbon et al. ( 1977 ), Experiment 1.Left: # trials to criterion as a function of ITI. Right: # cumulative training hours to criterionas a function of ITI. Groups differed in CS duration (4–32 s).


of trials per session promoted acquisition, although some impairment has been observed in the case of animals receiving only a single trial per session ( Gottlieb & Prince, 2012 ). Gottlieb and Prince ( 2012 ) concluded that within a session, adding trials between any pair has little effect. As with US magnitude and CS intensity,within-subject effects of number of trials appear more readily obtainable than dobetween-subject effects ( Gottlieb & Rescorla, 2010 ).

Figures 1.2 and 1.3 clearly point to repetitions over time being more important than number of repetitions in establishing conditioned responding. Figure 1.1 illus-trates that the gradual nature of learning is still being called into question. Hull ’ s ( 1943 ) choices may have been reasonable, but only those relating to the CS-US interval have received unequivocal support. Even there, Hull did not have a mecha-nism by which trace and backward conditioning could lead to inhibition.

Content of Learning

The simplest framework for understanding what is learned in Pavlovian conditioning is the Stimulus-Response (S-R) framework of behaviorism that pre-dated the fi rst unambiguous reports of Pavlovian conditioning ( Guthrie, 1935 ; Hull, 1943 ; Thorndike, 1911 ). Within this framework, animals come to refl exively respond to a CS that has been contiguously paired with a UR.

Pavlov ( 1927 ) envisioned learning as a process by which CS and US neural centers become linked through asynchronous mutual activation. As a result, the CS comes to activate the US center, leading to responding. This refl ects a view from within the Stimulus-Stimulus (S-S) framework, in which learning results in representations of the CS and US becoming linked. From an S-S view, a tone activates the idea of shock,and so the rabbit blinks. From an S-R view, a tone elicits a blink. The idea of shockis a mental event that is not directly observable, and so S-S theories are inherently cognitive in nature, early attempts to model the mind to explain behavior. Even so, like the early behaviorists, Pavlov considered the CR to be automatic and identicalto the UR in form.

Two strategies have been used to try to show evidence of S-S learning. The fi rst attempted to demonstrate learning in the absence of a UR. For instance, Brogden

Figure 1.3 Replot of function determined by Kehoe and Macrae ( 1994 ) relating trials per session to trials to acquisition across a number of rabbit eyeblink conditioning studies. Overlaidis function relating trials per session to sessions to acquisition.


( 1939 ), showed learning in a sensory preconditioning procedure that involved fi rst pairing two neutral CSs before pairing one with the US and showing that the otheralso came to elicit conditioned responding. A more recent strategy has been to modify the value of the US after it has been used to train a CS. Any post-conditioning infl u-ence on the CS would be evidence for S-S learning.

One way to accomplish this is to use second-order conditioning and to modify the fi rst-order stimulus through extinction. For example, one might fi rst pair a tonewith an air puff, then a light with the tone. The key question is whether the condi-tioned eyeblinking to the light is then affected by extinction of the tone. Supportingan S-R view, in a number of appetitive and aversive procedures and species, extin-guishing the fi rst-order stimulus has little effect on responding to the second-order stimulus and so provides no evidence of S-S learning ( Rizley & Rescorla, 1972 ).

A second way to assess for S-S learning is to directly modify the primary reinforcerin a fi rst-order conditioning experiment. Often, this is done by pairing a food rein-forcer with illness ( Holland & Straub, 1979 ). The results of primary reinforcerdevaluation studies show decreased conditioned responding due to US devaluationand so provide evidence for S-S learning.

A way to reconcile these fi ndings can be found in the few second-order condition-ing studies that have devalued the primary reinforcer and assessed for effects on the second-order CS. Devaluation both through poisoning and motivational shifts hasbeen found to reduce responding to a second-order CS ( Ross, 1986 ; Winterbauer & Balleine, 2005 ). This suggests that in some cases the failure to adjust conditionedresponding of a second-order CS after extinction of the fi rst-order CS may not bedue to exclusive S-R learning. Rather, organisms may have formed a direct association between the second-order CS and the primary US, despite those stimuli not having been contiguously paired.

Tolman ( 1932 ) adopted an S-S framework but rejected Pavlov ’ s ( 1927 ) notion of stimulus substitution. He held instead that a CS predicts US occurrence and so leadsto preparatory responding. One distinction between Pavlov ’ s view and Tolman ’ s isthat for Pavlov the key element that determines behavior is the UR, while for Tolman it is the US. This allows Tolman to accommodate CRs that appear qualitatively dif-ferent from URs. Although Mackintosh ( 1974 ) points out that most CRs look likeURs, and that the exceptions may not prove fatal to the idea of CR-UR identity, there are a number of cases where the measured CR is either unrelated or superfi cially opposite to the UR. For example, the CR to shock is often freezing, whereas theUR involves vigorous motor activity. In conditioned tolerance procedures, the CR is may be exactly the opposite of the UR, as in the case of cues paired with morphine that lead to hyperalgesia and not analgesia ( Siegel, 1983 ).

Contingency

Degraded Contingency

Partial Reinforcement. A major change in thinking about Pavlovian conditioning came about from studies that varied the proportion of trials in which the CS and USwere paired. One way to vary this is to intersperse nonreinforced presentations of


the CS. This has the effect of reducing the probability that the US will follow theCS and is referred to as partial reinforcement ( Gottlieb, 2005 ).

Although the results from studies of partial reinforcement are not entirely consist-ent, reducing probability of reinforcement most often impairs acquisition of condi-tioned responding ( Mackintosh, 1974 ). When the number of reinforced trials and the time between CS presentations are held constant, continuous reinforcement more often leads to higher levels of asymptotic responding than does partial reinforcement,though it has relatively little infl uence on the number of reinforced trials before conditioned responding emerges. There is also some evidence that deleterious effects of reducing the probability of reinforcement might manifest most strongly in whether organisms acquire the conditioned response at all and not in how rapidly it emerges or how vigorous it becomes ( Gottlieb, 2005 ).

Intertrial USs and Explicit Unpairing. The effects on acquisition of presenting USs that are not preceded by CSs (unsignaled USs) are less ambiguous than the effects of partial reinforcement: Unsignaled USs strongly impair conditioned respond-ing. This impairment is graded such that when suffi cient USs are presented duringthe ITI to equate reinforcement rate during the CS and in its absence, a CS will not come to elicit conditioned responding ( Rescorla, 1966, 1968 ). Furthermore, when the ITI reinforcement rate exceeds that during the CS, the CS will develop inhibitory properties ( Rescorla, 1969b ). This means that CS-US contiguity is not suffi cient to generate conditioned responding. Nor may contiguity in the operational sense benecessary for learning. The explicitly unpaired condition, where USs are only deliv-ered during the ITI, leads to inhibitory learning even though the CS and US are never contiguously paired. It appears that what most predicts conditioned respondingis the relative likelihood of the US in the presence and absence of the CS, a quantity referred to as CS-US contingency.

Content: Additions to Associative Structure

It appears that what is essential in acquiring conditioned responding is discriminating the predictive value of the CS from that of the background. These fi ndings neces-sitated that the associative structure expand to include associations between the background (context) and discrete stimuli, associations that either interfere with the acquisition or the expression of conditioned responding ( Bouton, 1993 ; Gibbon & Balsam, 1981 ; Miller & Schachtman, 1985 ; Rescorla & Wagner, 1972 ).

Relative Contingency: Cue Competition Phenomena

Stimulus Compounds

Stimuli are never presented in isolation. When the experimenter lacks control over other stimuli in the environment, those stimuli are referred to as background or contextual stimuli. When the experimenter creates complexes of stimuli from indi-vidual elements, those stimuli are referred to as compounds. The study of compoundconditioning refl ects the understanding that organisms face the problem of discrimi-nating stimuli that are informative from all the others that may be co-occurring.


When an excitor is fi rst presented in compound with a neutral CS, it is not uncom-mon for there to be less responding than to the excitor alone. This is referred to asexternal inhibition, as the neutral stimulus appears to be inhibiting the conditioned response ( Pavlov, 1927 , p. 44). Alternatively, the decrement can be thought of as aresult of the stimulus compound being represented as its own holistic stimulus that only partially generalizes to the excitor ( Pearce, 1987 ). This distinction refl ects twodistinct views of Pavlovian processes. In one, stimuli can be reduced to elements that, when combined, maintain all their essential properties ( Rescorla & Wagner, 1972 ). In the other, every stimulus, compound or not, is a distinct element ( Guthrie, 1935 ; Pearce, 1987 ). Although this conceptual distinction is nontrivial, the quantitativerealization of it appears to rely largely on the degree to which the predictive valuesof simultaneously presented stimuli are summed or averaged ( Wagner, 2003 ). Reportsof excitatory summation are common ( Pavlov, 1927 ; Wagner, 1971 ), but less so areaccounts that both compare compound responding to the maximal element and that attempt to rule out performance factors relating to greater stimulation when thecompound stimulus, and not an element, is presented ( Rescorla, 1997 ).

Compound Conditioning

Overshadowing. Overshadowing refers to a reduction in conditioned respondingwhen a CS is reinforced in compound compared to if it were reinforced alone. Forexample, a light paired with an air puff may lead to substantially less anticipatory eyeblinking if it is conditioned in the presence of a tone than if it is conditionedalone. When stimuli differ in salience, one stimulus may completely overshadow theother ( Mackintosh, 1971 ). Although overshadowing may not be complete, reportsof mutual overshadowing in which both elements overshadow each other are uncom-mon ( Kehoe, 1982 ).

Blocking. Blocking is one of the most important and well-established contempo-rary phenomena. It was fi rst demonstrated by Kamin ( 1968 ) in a rat fear conditioningprocedure. Kamin found that when a compound stimulus made up of a light and anoise was followed by shock, animals came to suppress ongoing bar pressing to the light alone. However, if the noise had previously been paired with shock, the light did not suppress bar pressing. Blocking provides strong evidence that contiguity isinsuffi cient to generate conditioned responding, but it also suggests that contingency is insuffi cient, as well; in both of Kamin ’ s groups, the contiguity and contingency between the light and the shock was equivalent.

Superconditioning. Blocking is an example of a more general principle by whichindividual CSs determine the acquisition of conditioned responding to other simul-taneously presented stimuli. For example, if a light is established as a conditioned inhibitor of eyeblinking, subsequently conditioning a tone in the presence of the light can facilitate CR acquisition to the tone. This phenomenon is referred to as super-conditioning ( Williams & McDevitt, 2002 ; Wagner, 1971 ). A general principle toemerge is that the acquisition of conditioned responding to a CS trained in com-pound is an inverse function of the degree to which other simultaneously presentedstimuli are predictive.

Protection from Extinction. If reinforcing a light in the presence of an excitatory tone can block the light from developing conditioned responding, nonreinforcing


the light-tone compound may lead the light to become inhibitory ( Holland, 1988 ). This phenomenon is referred to as protection from extinction, referring to the target CS preventing the blocking CS from fully extinguishing. When the target CS is an inhibitor, the protective effect may be even greater ( Rescorla, 2003a ).

Contingency Becomes Relative Contingency. Blocking and protection fromextinction have their analogs in Rescorla ’ s ( 1966, 1968, 1969b ) contingency experi-ments. A random pairing procedure becomes a one-stage blocking procedure with the context playing the role of the blocker. Similarly, including the context as a CS turns the explicitly unpaired procedure into a one-stage protection from extinction procedure (the context is protected from extinction when the CS is present), which is the typical way of generating conditioned inhibition. Rescorla ’ s contingency experi-ments, then, can be viewed as special cases of cue competition experiments.

Linear regression is the primary statistic researchers use to determine the relative contributions of predictor variables to particular outcomes. The outputs of regression analyses are measures of predictive strength (beta coeffi cients) that have been adjusted to take into account correlations amount predictor variables themselves. Cue com-petition phenomena suggest that in determining the predictive value of a stimulus,organisms take into account the predictive value of other co-present stimuli in a similar way. Indeed, the most infl uential associative model, the Rescorla-Wagner (R-W) Model ( Rescorla & Wagner, 1972 ), is an algorithm that, when given enough information, asymptotically computes beta coeffi cients; when given insuffi cient infor-mation to parcel out predictive value to each element, as in the case of overshadow-ing, the R-W Model splits the difference among possible predictors. A more explicit regression mechanism is incorporated in the modeling of Gallistel and Gibbon ( 2000 ). It is informative that Pavlovian processes appear to be serving a similar func-tion as a statistic developed to optimally allocate predictive value.

Overexpectation. If organisms are computing something like linear regression coeffi cients, then a compound made up of two predictors should lead to an expecta-tion of an outcome that is greater than to either predictor alone. Returning to ourexample preparation, consider the situation in which both a light and a tone are separately trained to elicit conditioned eyeblinking. When they are then presented incompound, there should be a greater air puff expectation than when either CS is presented alone. But the compound is followed by the same air puff used to establish the individual CSs. This confl icting information should lead either to confi guring (generation of an interaction term) or to a decrement in predictive value attributedto each element of the compound. Indeed, reinforcing a compound of two excitorscan lead to a decrease in responding to either element, a phenomenon known as overexpectation ( Lattal & Nakajima, 1998 ).

Relative Validity. Although the R-W Model was developed to account for cue competition, the phenomenon of relative validity poses a unique challenge ( Rescorla& Wagner, 1972 ). Demonstrating relative validity involves comparing conditioned responding to a CS element that has been presented in two different kinds of dis-criminations. In one, the common element (X) has been reinforced in the presence of one CS and nonreinforced in the presence of another (AX +, BX − ). In the other,the element is equally often reinforced and nonreinforced in the presence of the two other stimuli (AX +− , BX +− ). Wagner, Logan, Haberlandt, and Price ( 1968 ) found that X leads to greater conditioned responding in the latter discrimination,


demonstrating that the validity of a predictor depends on whether the other stimuli are better or worse predictors. Relative validity is clearly a cue competition phenom-enon, but models like the R-W Model accommodate the effect only when the partially reinforced compounds have an asymptotic associative strength closer to that of a continuously reinforced stimulus than to that of a nonreinforced stimulus. However,recent evidence suggests that the associative strength of a partially reinforced stimulusis unbiased ( Andrew & Harris, 2011 ).

Extinction of Conditioned Inhibibition. The most serious problem from a regression framework is that posed by conditioned inhibition: Nonreinforced pres-entations of an inhibitor do not extinguish the inhibition ( Williams, 1986 ). Similarly, nonreinforcing a compound of an inhibitor and a neutral CS does not lead the neutral CS to become excitatory ( Baker, 1974 ). Pavlovian theories fi nd this intuitive fi nding diffi cult to accommodate.

Conditional Contingency

Occasion Setting

One complexity in viewing regression (or relative contingency) as a normative model of attributing predictive value is that it assumes a particular structure of the world in which the predictive values of stimuli do not systematically vary within a particularstimulus environment; however, it is possible for stimuli to have different predictivevalue depending on the local context. Consider, for example, a discrimination (A − ,AX+ ) in which a tone-light compound (AX) is followed by air puff while the light (A) is separately nonreinforced. Although the regression solution attributes full pre-dictive power to the tone (X), it is possible instead to frame the problem as one in which the light is predictive conditional on the presence of the tone. That is, insteadof signaling the air puff, the tone signals that when the light is present the airpuff is likely to occur. From this perspective, it is the light that controls conditionedeyeblinking and the tone that modulates this control.

To distinguish these possibilities, Ross and Holland ( 1981 ) took advantage of thefact that visual and auditory CSs paired with food evoke different behaviors in rats. They demonstrated that in an A −, AX + simultaneous discrimination, the conditionedresponse was consistent with control by X. However, when X preceded and over-lapped with A, the response was consistent with control by A. It appeared that in theserial discrimination, X did not gain the ability to elicit conditioned responding but rather set the occasion for A to elicit conditioned responding. This Pavlovian phe-nomenon of occasion setting has two common forms, positive and negative, referringto whether the occasion setter signals that a CS otherwise nonreinforced will be reinforced (positive occasion setting) or that a CS otherwise reinforced will be non-reinforced (negative occasion setting).

Holland ( 1992 ) distinguished occasion setters from simple CSs in three ways. Thefi rst, relating to response form, only applies to positive and not negative occasionsetting. The second is that occasion setters do not lose their properties when they are either extinguished or reinforced on their own. The third distinguishing featureof occasion setters is that they tend to modulate responding to a smaller set of stimuli


than do typical CSs. Holland ( 1992 ) originally concluded that occasion settersshowed CS-US selectivity, as they failed to modulate responding to other CSs pairedwith the same US or to the original target CS paired with a different US. However,Rescorla ( 1985 ) showed that occasion setters can modulate responding to nontarget stimuli that have been targets of different occasion setters. This form of generaliza-tion refl ects a level of abstraction that is more often associated with inferential accounts of information processing.

Interference Phenomena

The existence of occasion setting suggests that in attributing predictive value to stimuli, Pavlovian mechanisms are sensitive not only to relative but also to conditional contingencies. Behavior consistent with computations of conditional contingency can also be observed when the experimentally determined contingencies are solely con-ditional on time. This is the case when contingency is abruptly changed, for example when a tone signaling an air puff is suddenly presented alone. It is possible that organisms simply update their estimates as new information is acquired, but it is also possible that they maintain some form of conditional knowledge that certain contin-gencies are in effect at different times. Although there are a number of different treatments that involve abrupt changes in stimulus environment, we have chosen to focus on the three most common ways in which the predictive value of a single CSis abruptly changed.

CS Preexposure. Latent inhibition refers to a reduction in conditioned respond-ing that is observed when nonreinforced CS trials are presented prior to conditioning( Lubow, 1973 ). Despite the name, latent inhibitors are slow to be established as appetitive or aversive excitors or inhibitors ( Reiss & Wagner, 1972 ; Rescorla, 1971 ).rThat is, presenting a tone repeatedly without consequence makes it hard to learn anything about that tone. Context plays a large role. Latent inhibition is attenuated gwhen a CS is preexposed in a different context from that of subsequent training and is also attenuated as a direct function of the amount of post-conditioning exposureto the preexposure/training context ( Lubow, Rifkin, & Alek, 1976 ).

CS Pre-training. Counterconditioning refers to the procedure by which a CS is fi rst trained with one US and then with another of opposite valence. This form of pre-training typically interferes with the acquisition of new conditioned responding ( Konorski & Szwejkowska, 1956 ). For example, pairing a tone with shock makes it more diffi cult to then pair the tone with food and vice versa. Aversive-to-appetitive counterconditioning more reliably shows the interference effect than does appetitive-to-aversive counterconditioning ( Bouton & Peck, 1992 ), though an impressive set of experiments by Nasser and McNally ( 2012 ) clearly establishes that appetitive-to-aversive interference can be consistently obtained.

CS Postexposure. Extinction refers to the procedure by which the contingency of an excitatory CS with a US is abruptly removed. It also refers to the reductionin conditioned responding that results (Vurbic & Bouton (this volume); Urcelay, 2012 ). Typically, extinction is accomplished by presenting the CS without the US, but it also occurs when the CS-US relationship is degraded in other ways, such asintroducing a random relationship between CS and US ( Delamater, 1996 ). Characteristic of extinction is that extinguished conditioned responding is highly


susceptible to recovery, as with the passage of time (spontaneous recovery). Anextinguished CS may also show recovery of responding when presented concurrently with a novel stimulus (disinhibition) or in a context in which an animal has had rela-tively little experience since an unsignaled US was last presented (reinstatement). Recent work has focused on the recovery that occurs when animals are tested outsidethe extinction context (renewal). Renewal is strongest when animals are trained andextinguished in different contexts and are tested in their training context (ABA renewal). It is weaker when training, extinction, and testing are conducted in differ-ent contexts (ABC renewal), and it is weakest, though still apparent, when trainingand extinction are conducted in the same context and testing occurs in a different one (AAB renewal) ( Bouton, 2002 ).

It now appears that extinction is not unique. Whenever a second phase of traininginvolves opposition of a response acquired in the fi rst stage, as with countercondition-ing and overexpectation, the fi rst stage response shows recovery ( Bouton & Peck, 1992 ; Peck & Bouton, 1990 ; Rescorla, 2007 ). These results strongly argue that excitatory learning is not fully erased but rather that its expression is selectively opposed by the superimposition of new inhibitory learning.

Findings such as these have led to the suggestion that inhibitory processes areparticularly sensitive to disruption and may fade in time. Another possibility is that what is learned second is more sensitive to disruption than what is learned fi rst.Distinguishing these possibilities depends on what happens to a stimulus that is fi rst trained as a latent or conditioned inhibitor and then trained as an excitor. Althoughthere is now some evidence for recovery of latent and conditioned inhibition ( De LaCasa & Lubow, 2002 ; Nelson, 2002 ), studies are few and there are notable failuresto obtain ( Rescorla, 2005 ).

The study of interference phenomena has shown that new learning does not fully erase old learning and that the subsequent expression of conditioned respondingdepends on temporal and physical context. Both occasion setting and the interference phenomena suggest that organisms are sensitive to conditional contingencies at dif-ferent timescales.

Content: Hierarchical Associations

Within associative structure, conditional likelihoods are represented by hierarchicalassociations between modulators and CS-US associations. In occasion setting, modu-lators tend to be discrete stimuli, whereas in the interference phenomena, modulators tend to be contextual stimuli that may vary with the passage of time.

Time

Time is of central importance to Pavlovian conditioning, and there is evidence that organisms are not merely sensitive to it but that they encode specifi c durations. Inthe peak procedure, in which occasional nonreinforced probe trials with extended duration CSs are interspersed throughout acquisition, what is typically observed is normally distributed responding that peaks at the time of US delivery ( Gallistel & Gibbon, 2000 ; Pavlov, 1927 , p. 41). This timed behavior has been observed to


emerge as soon as does conditioned responding ( Balsam, Drew, & Yang, 2002 ) and to maintain throughout extinction ( Guilhardi & Church, 2006 ). There is even evi-dence that this temporal encoding may precede the emergence of conditioned responding ( Ohyama & Mauk, 2001 ).

In discussing the role of CS duration and ITI on acquisition, we did not mention another important regularity. If both the CS duration and the ITI are proportionately shortened or lengthened, the opposing benefi cial and detrimental effects appear to cancel out. The ratio of ITI: CS-US interval (also called the C : T ratio) appears to be a powerful, though not perfect, predictor of the rate and magnitude of condi-tioned responding ( Gibbon et al., 1977 ; Lattal, 1999 ).

Content: Addition of US Features

The growing acceptance of the importance of time as both a variable that infl uences conditioning as well as a variable that may be encoded has led to the most recent expansion of associative structure to include distinct features of the US ( Delamater& Oakeshott, 2007 ). Although the view that USs have distinct sensory and motiva-tional components is not new, that they may also have temporal features that repre-sent when they occur relative to other stimuli, as well as hedonic and other features, marks a further departure from the view fi rst proposed by Hull ( 1943 ). At the extreme are those who believe that the features of the US that are encoded include explicit statistical information about rates or probabilities of occurrence (e.g., Gallistel& Gibbon, 2000 ). From this perspective, associative strength is no longer a necessary construct, as the information provided by variations in associative strength is instead provided by representations of specifi c environmental quantities.

From Prediction to Performance

How Pavlovian Learning Translates into Conditioned Responding

Associative learning is inferred from amounts of conditioned responding, but how underlying learning manifests in behavior is neither well understood nor well speci-fi ed. The typical assumption is that increases in responding refl ect increases in esti-mates of associative strength. Unfortunately, the weakness of this assumption has made a number of important questions diffi cult to address. Recently, Rescorla ( 2000a ) has described a compound testing technique that partially overcomes this limitation. He has used this technique to empirically address a number of important issues for the fi rst time, such as the relative amounts of associative change when stimuli are conditioned in compound ( Rescorla 2000a, 2001c ), whether the superiority some-times observed in retraining is due to difference in initial associative strength or difference in learning rate ( Rescorla, 2002a ), and whether the generalization gradient is broader for inhibition than excitation ( Rescorla, 2002b ). Although the questions Rescorla addressed are themselves important, it is the methodological advancement that will likely have the widest impact. We see no comparable situation whereadvancement in the logic of experimental design has allowed such a variety of theo-retically important ideas to become testable. A sample of these designs and the fi rst reported results from their use are presented in Table 1.1 .

Tab

le 1

.1

Rec

ent

expe

rim

enta

l des

igns

tha

t ad

dres

s pr

evio

usly

unt

esta

ble

ques

tions

Ref

eren

cePh

ase

1Ph

ase

2Ph

ase

3Te

st/R

esul

tIn

terp

reta

tion

Res

corl

a ( 2

001a

)A

+,

C +

, B

− ,

D−

A+

, B

+B

C >

AD

Mor

e ch

ange

on

earl

y ac

quis

ition

tri

als

than

on

late

r tr

ials

Res

corl

a ( 2

001a

)A

+,

B +

, C

+ ,

D +

A−

, C

−A

−,

B −

BC

< A

DM

ore

chan

ge o

n ea

rly

extin

ctio

n tr

ials

tha

n on

late

r tr

ials

Res

corl

a ( 2

002b

)A

+,

B −

, C

+ ,

D −

A−

, B

− ,

C+

, D

+1

AD

> B

CM

ore

chan

ge o

n re

info

rced

tri

als

than

on

nonr

einf

orce

d tr

ials

Res

corl

a ( 2

002b

)A

+,

B −

, C

+ ,

D −

A−

, D

+2

AD

> B

CM

ore

inhi

bito

ry g

ener

aliz

atio

n th

an e

xcita

tory

ge

nera

lizat

ion

Res

corl

a ( 2

002a

)A

−,

B −

, C

− ,

D −

A −

, C

−A

+,

B +

BC

> A

DSl

ower

rat

e of

lear

ning

with

late

nt in

hibi

tor

than

with

nov

el C

SR

esco

rla

( 200

2a )

A +

, B

−,

C +

, D

−A

−,

B −

, C

− ,

D−

A +

, B

+B

C >

AD

Rea

cqui

sitio

n fa

ster

due

to

head

sta

rt a

nd n

ot

fast

er le

arni

ng r

ate

Res

corl

a ( 2

003b

)A

+,

B −

, C

+ ,

D −

A −

, B

− ,

C−

, D

−A

B +

AD

> B

CR

eacq

uisi

tion

also

fas

ter

due

to h

ighe

r le

arni

ng

rate

Not

e: L

ette

rs r

efer

to

CSs

, +

= r

einf

orce

men

t, −

= n

onre

info

rcem

ent.

1 B

rief

tra

inin

g in

Pha

se 2

. 2 E

xten

ded

trai

ning

in P

hase

2.


How Pavlovian Learning Motivates Appetitive Instrumental Responding

The interaction of Pavlovian and instrumental processes has been a topic of interest throughout the study of learning and motivation. Although early interest focused on fear conditioning and avoidance learning, recent emphasis has been on the infl uence of appetitive CSs on instrumentally trained responses; this will be our focus, as well.

Pavlovian-Instrumental Transfer. Pavlovian to instrumental transfer (PIT) refers to the augmenting of an instrumentally trained response by a CS that has been sepa-rately trained with the same outcome. It is typically studied in rats where lever press-ing for food may be increased by presentation of a CS separately paired with food.Appetitive PIT is both outcome specifi c and nonspecifi c. Nonspecifi c PIT refers to the ability of a CS to elicit more instrumental behavior than occurs in its absence. It is dependent on the strength of competing goal-directed responses directly elicited by the appetitive CS. More interesting is outcome-specifi c PIT, in which a CS acts selectively on instrumental responses trained with the CS ’ s own US as opposed toresponses trained with a different US. For example, a tone paired with a food pellet will lead rats to press a lever rewarded with food pellets more than a lever rewarded with liquid sucrose. Notably, a CS may elicit outcome-specifi c PIT while still sup-pressing instrumental responding and so failing to show nonspecifi c PIT ( Holmes, Marchand, & Coutureau, 2010 ).

The most startling property of outcome-specifi c PIT is that it does not depend on the current predictive value of the CS, as its magnitude appears unaffected by both extinction and US devaluation ( Delamater, 1996 ; Holland, 2004 ). Furthermore,Rescorla ( 2000b ) demonstrated outcome-specifi c transfer to a CS randomly paired with a US, even when the CS did not come to elicit conditioned responding. It appears that outcome-specifi c transfer is established whenever there are surprising CS-US pairings and, once established, is resistant to change. Indeed, as far as we know, Delamater ( 2012 ) has provided the only report of a loss in transfer once the CS-US relationship has been established, the result of extinguishing a CS that had been given a small number of acquisition trials. This pattern of transfer implies that Pavlovian processes function as more than mechanisms of prediction.

If PIT refl ects motivational properties of the CS, the motivation appears to be guided by an aspect of the reinforcer that is independent of obtaining and consuming it. This is consistent with a mechanism by which the CS activates a particular featureof the US representation that cannot be devalued and that subsequently activates theresponses that brought it about ( Balleine & Ostlund, 2007 ; Holland, 2004 ; Holmes et al., 2010 ).

Behavior Systems Approach to PIT. Timberlake ( 1994 ) promoted a behavior systems approach to conditioning that envisions CSs as eliciting a variety of species- and motivation-specifi c behaviors that depend on temporal proximity to the US. When food is more temporally distant, a CS elicits in a rat a general search mode characterized by increased attention to environmental stimuli relevant to food. Asthe US becomes more proximal, the general search mode gives way to a focal searchmode which includes more specifi cally food-directed behavior ( Silva & Timberlake,1997 ).

It may be the case that outcome-specifi c PIT is a behavior characteristic of a general reward-directed search mode evoked in times when a proximal reward is not


expected. This would explain why CS extinction may facilitate PIT and why it is easier to detect when the instrumental responses are being extinguished and the reinforcersare not proximal. The simultaneous facilitation of reinforcer-related behaviors and indifference toward the occurrence of the reinforcer suggests to us that the searchbehavior is not motivated toward directly obtaining reward. Rather, it may be aimed toward investigating which responses are effective in obtaining reward. Such a moti-vation toward focused exploration is consistent with a view of Pavlovian conditioningsummarized by Domjan, Cusato, and Krause ( 2004 , p. 232): an “adaptive processthat promotes effi cient interactions of the organism with signifi cant biological events in its natural environment.” It is also an adaptive understanding consistent with themechanistic views of Balleine and Ostlund ( 2007 ) and Holland ( 2004 ).

Summary and Conclusion

Conditioned responding emerges as a function of observing contingent relationships among stimuli over time. It is sensitive to both relative and conditional contingencies.For the simplest form of associative formation, that between a single salient stimulusand a biologically relevant outcome, large numbers of observations are not needed, but observations must be separated in time. Predictive value, once established, is sensitive to changes in contingency. New contingencies are learned, but old ones arenot forgotten, and old habits can reemerge in times of uncertainty. Once an associa-tion has been formed, it is hard to un-form, and that association will guide instru-mental behavior independent of contingency.

Pavlovian processes are most strongly linked to predicting the likelihood that an important event will or will not occur at a certain time, allowing organisms to engage in complicated sequences of preparatory behavior that culminate in appropriately timed conditioned responding. Independent of this, Pavlovian processes may serve a role analogous to the hypothesis generating roles of theories in scientifi c research: to focus the search for additional control of the environment.

After almost a century of investigation, Pavlovian “simple learning” processes appear more and more like nonverbal analogs of many of the skills we spend count-less hours trying to teach in statistics and research methods courses. In that context,we call them critical thinking.

Acknowledgments

We thank Robert Rescorla for advice and discussions as well as for his remarkable contributions to the study of mind and behavior. We also thank Peter Holland for useful discussion.

References

Andrew , B. J. , & Harris , J. A. ( 2011 ). Summation of reinforcement rates when conditionedstimuli are presented in compound . Journal of Experimental Psychology: Animal Behavior Processes, ss 37, 385 – 393 . 77


Baker , A. G. ( 1974 ). Conditioned inhibition is not the symmetrical opposite of conditioned excitation: A test of the Rescorla-Wagner model . Learning and Motivation, n 5 ,5 369 – 379 .

Balleine , B. W. , & Ostlund , S. B. ( 2007 ). Still at the choice-point: Action selection and initia-tion in instrumental conditioning . Annals of the New York Academy of Sciences , ss 1104 , 4 147 – 171 .

Balsam , P. D , Drew , M. R. , & Yang , C. ( 2002 ). Timing at the start of associative learning . Learning and Motivation, n 33, 141 – 155 . 3

Barnet , R. C. , & Miller , R. R. ( 1996 ). Second-order excitation mediated by a backward con-ditioned inhibitor . Journal of Experimental Psychology: Animal Behavior Processes , ss 22 , 2 279 – 296 .

Bouton , M. E. ( 1993 ). Context, time, and memory retrieval in the interference paradigms of Pavlovian learning . Psychological Bulletin, n 114, 80 – 99 . 4

Bouton , M. E. ( 2002 ). Context, ambiguity, and unlearning: Sources of relapse after behavioral extinction . Biological Psychiatry, y 52 , 976 – 986 .2

Bouton , M. E. , & Peck , C. A. ( 1992 ). Spontaneous recovery in cross-motivational transfer (counterconditioning) . Animal Learning & Behavior , r 20 , 313 – 321 .0

Brogden , W. J. ( 1939 ). Sensory preconditioning . Journal of Experimental Psychology , y 25 , 5 323 – 332 .

De La Casa , L. G. , & Lubow , R. E. ( 2002 ). An empirical analysis of the super-latent inhibition effect . Animal Learning & Behavior , r 30, 112 – 120 . 0

Delamater , A. R. ( 1996 ). Effects of several extinction treatments upon the integrity of Pavlovian stimulus-outcome associations . Animal Learning & Behavior , r 24 , 437 – 449 .4

Delamater , A. R. ( 2012 ). Issues in the extinction of specifi c stimulus-outcome associations in Pavlovian conditioning . Behavioural Processes , ss 90, 9 – 19 . 0

Delamater , A. R. , & Oakeshott , S. ( 2007 ). Learning about multiple attributes of reward in Pavlovian conditioning . Annals of the New York Academy of Sciences , ss 1104 , 1 – 20 .4

Domjan , M. , Cusato , B. , & Krause , M. ( 2004 ). Learning with arbitrary versus ecological conditioned stimuli: Evidence from sexual conditioning . Psychonomic Bulletin & Review, w11, 232 – 246 .

Ellison , G. D. ( 1964 ). Differential salivary conditioning to traces . Journal of Comparative and Physiological Psychology, y 57 , 373 – 380 .77

Etscorn , F. , & Stephens , R. ( 1973 ). Establishment of conditioned taste aversions with a 24-hour CS-US interval . Physiological Psychology, y 1, 251 – 253 .

Flaherty , C. F. ( 1996 ). Incentive relativity. New York : Cambridge University Press . yGallistel , C. R. , Fairhurst , S. , & Balsam , P. ( 2004 ). The learning curve: Implications of a

quantitative analysis . Proceedings of the National Academy of Sciences of the United States of America, a 101 , 13124 – 13131 .

Gallistel , C. R. , & Gibbon , J. ( 2000 ). Time, rate, and conditioning . Psychological Review, w 107 ,77 289 – 344 .

Gibbon , J. , Baldock , M. D. , Locurto , C. , Gold , L. , & Terrace , H. S. ( 1977 ). Trial and intertrialdurations in autoshaping . Journal of Experimental Psychology: Animal Behavior Processes, ss3, 264 – 284 . 3

Gibbon , J. , & Balsam , P. ( 1981 ). Spreading associations in time . In C. M. Locurto , H. S. Terrace , & J. Gibbon (Eds.), Autoshaping and conditioning theory (pp. 219 – 253 ). New yYork : Academic Press .

Gormezano , I. , & Kehoe , E. J. ( 1981 ). Classical conditioning and the law of contiguity . In P. Harzem & M. D. Zeiler (Eds.), Advances in analysis of behaviour: Vol. 2. Predictability, correlation, and contiguity (pp. 1 – 45 ). Chichester, UK : John Wiley & Sons, Ltd . y

Gottlieb , D. A. ( 2005 ). Acquisition with partial and continuous reinforcement in rat magazine-approach . Journal of Experimental Psychology: Animal Behavior Processes , ss 31 , 319 – 333 .


Gottlieb , D. A. ( 2008 ). Is number of trials a primary determinant of conditioned responding? Journal of Experimental Psychology: Animal Behavior Processes, ss 34, 185 – 201 . 4

Gottlieb , D. A. ( 2011 ). Pavlovian conditioning . In Encyclopedia of the Sciences of Learning.g New York : Springer .

Gottlieb , D. A. , & Prince , E. B. ( 2012 ). Isolated effects of number of acquisition trials on extinction of rat conditioned approach behavior . Behavioural Processes, ss 90 , 34 – 48 .0

Gottlieb , D. A. , & Rescorla , R. A. ( 2010 ). Within-subject effects of number of trials in rat conditioning procedures . Journal of Experimental Psychology: Animal Behavior Processes ,ss36, 217 – 231 .6

Guilhardi , P. , & Church , R. M. ( 2006 ). The pattern of responding after extensive extinction . Learning & Behavior, r 34, 269 – 284 . 4

Guthrie , E. R. ( 1935 ). The psychology of learning. New York : Harper .gHeth , C. D. ( 1976 ). Simultaneous and backward fear conditioning as a function of number

of CS-US pairings . Journal of Experimental Psychology: Animal Behavior Processes, ss 2 ,2 117 – 129 .

Heth , C. D. , & Rescorla , R. A. ( 1973 ). Simultaneous and backward fear conditioning in therat . Journal of Comparative and Physiological Psychology, y 82, 434 – 443 . 2

Higgins , T. , & Rescorla , R. A. ( 2004 ). Extinction and retraining of simultaneous and succes-sive fl avor conditioning . Learning & Behavior, r 32, 213 – 219 .2

Holland , P. C. ( 1988 ). Excitation and inhibition in unblocking . Journal of Experimental Psychology: Animal Behavior Processes, ss 14 , 261 – 279 .4

Holland , P. C. ( 1992 ). Occasion setting in Pavlovian conditioning . In G. Bower (Ed.), The psychology of learning and motivation ( Vol. 28 , pp. 69 – 125 ). Orlando, FL : AcademicnPress .

Holland , P. C. ( 2004 ). Relations between Pavlovian-instrumental transfer and reinforcer devaluation . Journal of Experimental Psychology: Animal Behavior Processes , ss 30 ,0 104 – 117 .

Holland , P. C. , & Straub , J. J. ( 1979 ). Differential effects of two ways of devaluing the uncon-ditioned stimulus after Pavlovian appetitive conditioning . Journal of Experimental Psychology: Animal Behavior Processes, ss 5 , 65 – 78 .5

Holmes , N. M. , Marchand , A. R. , & Coutureau , E. ( 2010 ). Pavlovian to instrumental transfer: A neurobehavioural perspective . Neuroscience and Biobehavioural Reviews, ss 34 ,4 1277 – 1295 .

Hull , C. L. ( 1943 ). Principles of behavior. New York : Appleton-Century-Crofts .rKamin , L. J. ( 1965 ). Temporal and intensity characteristics of the conditioned stimulus . In

W. F. Prokasy (Ed.), Classical conditioning: A symposium (pp. 118 – 147 ). New York :m Appleton-Century-Crofts .

Kamin , L. J. ( 1968 ). “ Attention-like” processes in classical conditioning . In M. R. Jones (Ed.), Miami symposium on the prediction of behavior: Aversive stimulation (pp. 9 – 31 ). Miami, nFL : University of Miami Press .

Kehoe , E. J. ( 1982 ). Overshadowing and summation in compound stimulus conditioning of the rabbit ’ s nictitating membrane response . Journal of Experimental Psychology: Animal Behavior Processes , ss 8 , 313 – 328 .8

Kehoe , E. J. , & Macrae , M. ( 1994 ). Classical conditioning of the rabbit nictitating membraneresponse can be fast or slow: Implications for Lennartz and Weinberger ’ s (1992). two-factor theory . Psychobiology, y 22 , 1 – 4 . 2

Kimble , G. A. ( 1961 ). Hilgard and Marquis ’ conditioning and learning. New York :g Appleton-Century-Crofts .

Konorski , J. , & Szwejkowska , G. ( 1956 ). Reciprocal transformations of heterogeneous con-ditioned refl exes . Acta Biologiae Experimentalis , ss 17, 141 – 165 .77


Lattal , K. M. ( 1999 ). Trial and intertrial durations in Pavlovian conditioning: Issues of learning and performance . Journal of Experimental Psychology: Animal Behavior Processes , ss 25 ,5 433 – 450 .

Lattal , K. M. , & Nakajima , S. ( 1998 ). Overexpectation in appetitive Pavlovian and instrumen-tal conditioning . Animal Learning & Behavior , r 26, 351 – 360 . 6

Lolordo , V. M. , & Fairless , J. ( 1985 ). Pavlovian conditioned inhibition: The literature since 1969 . In R. R. Miller & N. E. Spear (Eds.), Information processing in animals: Conditioned inhibition (pp. 1 – 50 ). Hillsdale, NJ : Erlbaum . n

Lubow , R. E. ( 1973 ). Latent inhibition . Psychological Bulletin, n 79, 398 – 407 . 9Lubow , R. E. , Rifkin , B. , & Alek , M. ( 1976 ). The context effect: The relationship between

stimulus preexposure and environmental preexposure determines subsequent learning . Journal of Experimental Psychology: Animal Behavior Processes, ss 2, 38 – 47 . 2

Mackintosh , N. J. ( 1971 ). An analysis of overshadowing and blocking . Quarterly Journal of Experimental Psychology, y 23, 118 – 125 . 3

Mackintosh , N. J. ( 1974 ). The psychology of animal learning. New York : Academic Press . gMackintosh , N. J. ( 1975 ). A theory of attention: Variations in the associability of stimuli with

reinforcement . Psychological Review, w 82 , 276 – 298 .2Marlin , N. A. ( 1981 ). Contextual associations in trace conditioning . Animal Learning &

Behavior, r 9, 519 – 523 . 9Miller , R. R. , & Schachtman , T. R. ( 1985 ). Conditioning context as an associative baseline:

Implications for response generation and the nature of conditioned inhibition . In R. R. Miller & N. E. Spear (Eds.), Information processing in animals: Conditioned inhibition(pp. 51 – 88 ). Hillsdale, NJ : Erlbaum .

Morris , R. W. , & Bouton , M. E. ( 2006 ). Effect of unconditioned stimulus magnitude on the emergence of conditioned responding . Journal of Experimental Psychology: Animal Behavior Processes, ss 32 , 371 – 385 .2

Nasser , H. M. , & McNally , G. P. ( 2012 ). Appetitive-aversive interactions in Pavlovian fear conditioning . Behavioral Neuroscience , e 126 , 404 – 422 .6

Nelson , J. B. ( 2002 ). Context specifi city of excitation and inhibition in ambiguous stimuli . Learning and Motivation, n 33, 284 – 310 . 3

Ohyama , T. , & Mauk , M. D. ( 2001 ). Latent acquisition of timed responses in cerebellar cortex . Journal of Neuroscience, e 21, 682 – 690 .

Pavlov , I. P. ( 1927 ). Conditioned refl exes. New York : Dover .sPearce , J. M. ( 1987 ). A model for stimulus generalization in Pavlovian conditioning .

Psychological Review, w 94 , 61 – 73 .4Peck , C. A. , & Bouton , M. E. ( 1990 ). Context and performance in aversive-to-appetitive and

appetitive-to-aversive transfer . Learning and Motivation, n 21 , 1 – 31 .Razran , G. ( 1957 ). The dominance-contiguity theory of the acquisition of classical condition-

ing . Psychological Bulletin, n 54, 1 – 46 . 4Reiss , S. , & Wagner , A. R. ( 1972 ). CS habituation produces a “latent inhibition effect” but

no active “conditioned inhibition .” Learning and Motivation, n 3 , 237 – 245 .3Rescorla , R. A. ( 1966 ) Predictability and number of pairings in Pavlovian fear conditioning .

Psychonomic Science, e 4, 383 – 384 . 4Rescorla , R. A. ( 1968 ). Probability of shock in the presence and absence of CS in fear condi-

tioning . Journal of Comparative and Physiological Psychology, y 66 , 1 – 5 .6Rescorla , R. A. ( 1969a ). Pavlovian conditioned inhibition . Psychological Bulletin, n 72 , 77 – 94 .2Rescorla , R. A. ( 1969b ). Conditioned inhibition of fear resulting from negative CS-US con-

tingencies . Journal of Comparative and Physiological Psychology , y 67, 504 – 509 . 77Rescorla , R. A. ( 1971 ). Summation and retardation tests of latent inhibition . Journal of

Comparative and Physiological Psychology, y 75 , 77 – 81 .5


Rescorla , R. A. ( 1985 ). Conditioned inhibition and facilitation . In R. R. Miller and N. E. Spear (Eds.), Information processing in animals: Conditioned inhibition (pp. 299 – 326 ).nHillsdale, NJ : Erlbaum .

Rescorla , R. A. ( 1997 ). Summation: Assessment of a confi gural theory . Animal Learning & Behavior, r 25, 200 – 209 . 5

Rescorla , R. A. ( 2000a ). Associative changes in excitors and inhibitors differ when they are conditioned in compound . Journal of Experimental Psychology: Animal Behavior Processes,ss26, 428 – 438 .6

Rescorla , R. A. ( 2000b ). Associative changes with a random CS-US relationship . Quarterly Journal of Experimental Psychology: Comparative and Physiological Psychology, y 53B ,B325 – 340 .

Rescorla , R. A. ( 2001a ). Are associative changes in acquisition and extinction negatively accel-erated? Journal of Experimental Psychology: Animal Behavior Processes , ss 27 , 307 – 315 .77

Rescorla , R. A. ( 2001b ). Experimental extinction . In R. R. Mowrer & S. B. Klein (Eds.), Handbook of contemporary learning theories (pp. 119 – 154 ). Mahwah, NJ : Erlbaum . s

Rescorla , R. A. ( 2001c ). Unequal associative changes when excitors and neutral stimuli are conditioned in compound . Quarterly Journal of Experimental Psychology, y 54B, 53 – 68 . B

Rescorla , R. A. ( 2002a ). Savings tests: Separating differences in rate of learning from differ-ences in initial levels . Journal of Experimental Psychology: Animal Behavior Processes, ss 28 ,8369 – 377 .

Rescorla , R. A. ( 2002b ). Comparison of rates of associative change during acquisition andextinction . Journal of Experimental Psychology: Animal Behavior Processes , ss 28 , 406 – 415 .8

Rescorla , R. A. ( 2003a ). Protection from extinction . Learning & Behavior, r 31 , 124 – 132 .Rescorla , R. A. ( 2003b ). More rapid associative change with retraining than with initial train-

ing . Journal of Experimental Psychology: Animal Behavior Processes , ss 29, 251 – 260 . 9 Rescorla , R. A. ( 2005 ). Spontaneous recovery of excitation but not inhibition . Journal of

Experimental Psychology: Animal Behavior Processes, ss 31 , 277 – 288 . Rescorla , R. A. ( 2007 ). Renewal after overexpectation . Learning & Behavior, r 35, 19 – 26 . 5 Rescorla , R. A. , & Solomon , R. L. ( 1967 ). Two-process learning theory: Relationships

between Pavlovian conditioning and instrumental learning . Psychological Review, w 74 ,4151 – 182 .

Rescorla , R. A. , & Wagner , A. R. ( 1972 ). A Theory of Pavlovian conditioning: Variations inthe effectiveness of reinforcement and nonreinforcement . In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning: II. Current research and theory (pp. 64 – 99 ). New York : y Appleton-Century-Crofts .

Rizley , R. C. & Rescorla , R. A. ( 1972 ). Associations in second-order conditioning and sensory preconditioning . Journal of Comparative and Physiological Psychology , y 81 , 1 – 11 .

Romaniuk , C. B. , & Williams , D. A. ( 2000 ). Conditioning across the duration of a backwardcontioned stimulus . Journal of Experimental Psychology: Animal Behavior Processes, ss 26 ,6454 – 461 .

Ross , R. T. ( 1986 ). Pavlovian second-order conditioned analgesia . Journal of Experimental Psychology: Animal Behavior Processes, ss 12 , 32 – 39 .2

Ross , R. T. , & Holland , P. D. ( 1981 ). Conditioning of simultaneous and serial feature-positivediscriminations . Animal Learning & Behavior , r 9 , 293 – 303 . 9

Siegel , S. ( 1983 ). Classical conditioning, drug tolerance, and drug dependence . In R. G. Smart , F. B. Glaser , Y. Israel , H. Kalant , R. E. Popham , & W. Schmidt (Eds.), Research advances in alcohol and drug problems ( Vol. 7 , pp. 207 – 246 ). New York : Plenum .s

Silva , K. M. , & Timberlake , W. ( 1997 ). A behavior systems view of conditioned states duringlong and short CS-US intervals . Learning & Motivation, n 28, 465 – 490 . 8

Tolman , E. C. ( 1932 ). Purposive behavior in animals and men. New York : Century .n Thorndike , E. L. ( 1911 ). Animal intelligence: Experimental studies . New York : Macmillan . s


Timberlake , W. ( 1994 ). Behavior systems, associationism, and Pavlovian conditioning . Psychonomic Bulletin & Review, w 1 , 405 – 420 .

Urcelay , G. P. ( 2012 ). Exposure techniques: The role of extinction learning . In P. Neudeck & H. Wittchen (Eds.), Exposure therapy (pp. 35 – 63 ). New York : Springer .y

Wagner , A. R. ( 1971 ). Elementary associations . In H. H. Kendler & J. T. Spence (Eds.), Essays in neobehaviorism: A memorial volume to Kenneth W. Spence (pp. 187 – 213 ). New York :e Appleton-Century-Crofts .

Wagner , A. R. ( 1981 ). SOP: A model of automatic memory processing in animal behavior . In N. E. Spear & R. R. Miller (Eds.), Information processing in animals: Memory mechanisms(pp. 5 – 47 ). Hillsdale, NJ : Erlbaum .

Wagner , A. R. ( 2003 ). Context-sensitive elemental theory . The Quarterly Journal of Experimental Psychology, y 56B, 7 – 29 . B

Wagner , A. R. , Logan , F. A. , Haberlandt , K. , & Price , T. ( 1968 ). Stimulus selection in animaldiscrimination learning . Journal of Experimental Psychology , y 76, 171 – 180 . 6

Wagner , A. R. , & Rescorla , R. A. Inhibition in Pavlovian conditioning: Application of a theory . In M. S. Halliday & R. A. Boakes (Eds.), Inhibition and learning. London : AcademicgPress , 1972 .

Williams , B. A. , & McDevitt , M. A. ( 2002 ). Inhibition and superconditioning . Psychological Science, e 13, 454 – 459 . 3

Williams , D. A. ( 1986 ). On extinction of inhibition: Do explicitly unpaired conditioned inhibi-tors extinguish? American Journal of Psychology , y 99, 515 – 525 .9

Williams , D. R. , & Williams , H. ( 1969 ). Automaintenance in the pigeon: Sustained pecking despite contingent nonreinforcement . Journal of the Experimental Analysis of Behavior, r12, 511 – 520 . 2

Winterbauer , N. E. , & Balleine , B. W. ( 2005 ). Motivational control of second-order condition-ing . Journal of Experimental Psychology: Animal Behavior Processes , ss 31, 334 – 340 .

Yeo , A. G. ( 1974 ). The acquisition of conditioned suppression as a function of interstimulus interval duration . Quarterly Journal of Experimental Psychology, y 26, 405 – 416 . 6

Documents

Part I · 2020. 1. 16. · Razran (1957 ), in summarizing 161 Russian studies relevant to effects of CS intensity, concluded that there was abundant evidence that increasing CS intensity