POSITIVE REINFORCEMENT PRODUCED BY ELECTRICAL STIMULATION OFSEPTAL AREA AND OTHER REGIONS OF RAT BRAIN'

JAMRS OLDS2 AND PETER MILNKR

McGUl University

Stimuli have eliciting and reinforcing func-tions. In studying the former, one concentrateson the responses which come after the stimu-lus. In studying the latter, one looks mainlyat the responses which precede it. In itsreinforcing capacity, a stimulus increases,decreases, or leaves unchanged the frequencyof preceding responses, and accordingly it iscalled a reward, a punishment, or a neutralstimulus (cf. 16).

Previous studies using chronic implantationof electrodes have tended to focus on theeliciting functions of electrical stimuli de-livered to the brain (2, 3, 4, 5, 7, 10, 12, 14).The present study, on the other hand, hasbeen concerned with the reinforcing functionof the electrical st imulation.3

METHOD

GeneralStimulation was carried out by means of chronically

implanted electrodes which did not interfere with thehealth or free behavior of Ss to any appreciable extent.The .9s were 15 male hooded rats, weighing approxi-mately 250 gm. at the start of the experiment. EachS was tested in a Skinner box which delivered alternat-ing current to the brain so long as a lever was depressed.The current was delivered over a loose lead, suspendedfrom the ceiling, which connected the stimulator to therat's electrode. The 5s were given a total of 6 to 12 hr.of acquisition testing, and 1 to 2 hr. of extinction test-ing. During acquisition, the stimulator was turned on

1 The research reported here was made possibleby grants from the Rockefeller Foundation and theNational Institute of Mental Health of the U.S. PublicHealth Service. The authors particularly wish to expresstheir thanks to Professor IX (). Hcbb, who providedgerminal ideas for the research and who backed it withenthusiastic encouragement as well as laboratoryfacilities and funds. The authors are also grateful toMiss Joann Fcindcl, who performed the histologicalreconstructions reported here.

2 National Institute of Mental Health PostdoctoraleFellow of the U.S. Public Health Service.

3 The present preliminary paper deals mainly withmethods and behavioral results. A detailed report ofthe locus of positive, negative, and neutral reinforcingeffects of electrical brain stimulation is being preparedby the first author.

so that a response produced electrical stimulation;during extinction, the stimulator was turned off sothat a response produced no electrical stimulation.Each S was given a percentage score denoting theproportion of his total acquisition time given to re-sponding. This score could be compared with theanimal's extinction score to determine whether thestimulation had a positive, negative, or neutral rein-forcing effect. After testing, the animal was sacrificed.Its brain was frozen, sectioned, stained, and examinedmicroscopically to determine which structure of thebrain had been stimulated. This permitted correlationof acquisition scores with anatomical structures.

Electrode Implantation

Electrodes are constructed by cementing a pair ofenameled silver wires of 0.010-in. diameter into a 1/ucilcblock, as shown in Figure 1. The parts of the wireswhich penetrate the brain arc cemented together toform a needle, and this is cut to the correct lengthto reach the desired structure in the brain. This lengthis determined from Krieg's rat brain atlas (11) withslight modifications as found necessary by experience.The exposed cross section of the wire is the only partof the needle not insulated from the brain by enamel;stimulation therefore occurs only at the tip. Contactwith the lead from the stimulator is made throughtwo blobs of solder on the upper ends of the electrodewires; these blobs make contact with the jaws of analligator clip which has been modified to insulate thetwo jaws from one another. A light, flexible hearing-aidlead connects the clip to the voltage source.

The operation of implantation is performed with therat under Nembutal anesthesia (0.88 cc/Kg) and heldin a Johnson-Krieg stereotaxic instrument (11). Amid-line incision is made in the scalp and the skinheld out of the way by muscle retractors. A small holeis drilled in the skull with a dental burr at the pointindicated by the stereotaxic instrument for the struc-ture it is desired to stimulate. The electrode, which isclamped into the needle carrier of the instrument, islowered until the flange of the F-ucite block rests firmlyon the skull. Four screw holes are then drilled in theskull through four fixing holes in the flange, and theelectrode, still clamped firmly in the instrument, isfastened to the skull with jeweler's screws which exceedthe diameter of the screw holes in the skull by 0.006 in.The electrode is then released from the clamp and thescalp wound closed with silk sutures. The skin is pulledlightly around the base of the Lucite block and keptwell away from the contact plates. A recovery periodof three days is allowed after the operation beforetesting. Figure 2 is an X-ray picture of an electrodein place.

41!)

420 JAMES OLDS .VXD PETER MII.NEK

FIG. 1. Electrode design (see text for detaileddescription)

TestingThe testing apparatus consisted of a large-levered

Skinner box 11 in. long, 5 in. wide, and 12 in. high. Thetop was open to allow passage for the stimulating lead.The lever actuated a microswitch in the stimulatingcircuit so that when it was depressed, the rat receivedelectrical stimulation. The current was obtained fromthe 60-cycle power line, through a step-down trans-former, and was adjustable between 0 and 10 v. r.m.s.by means of a variable potentiometer. In the experi-ments described here the stimulation continued as long

as the lever was pressed, though for some tests a time-delay switch was incorporated which cut the currentoff after a predetermined interval if t he rat continuedto hold the lever clown. Responses were recorded auto-matically on paper strip.

On the fourth day after the operation rats were givena pretesting session of about an hour in the boxes. Eachrat was placed in the box and on the lever by E withthe stimulus set at 0.5 v. During the hour, stimulationvoltage was varied to determine the threshold of a''just noticeable" effect on the rat's behavior. If theanimal did not respond regularly from the start, il wasplaced on the lever periodically (at about 5-min. in-tervals). Data collected on the first day were not usedin later calculations. On subsequent days, Ss wereplaced in the box for about 3'jj hr. a clay; these were3 hr. of acquisition and ' $ hr. of extinction. During theformer, the rats were allowed to stimulate themselveswith a voltage which was just high enough to producesome noticeable response in the resting animal. As thisthreshold voltage fluctuated with the passage of time,E would make a determination of it every half hour,unless S was responding regularly. At the beginning ofeach acquisition period, and after each voltage test, theanimal was placed on the lever once by E. During ex-tinction periods, conditions were precisely the sameexcept that a bar press produced no electrical stimula-tion. At the beginning of each extinction period, animalswhich were not responding regularly were placed onthe lever once by E. At first, rats were tested in thisway for four days, but as there appeared to be littledifference between the results on different days, thisperiod was reduced to three and then to two clays for

FIG. 2. X ray showing electrode in place in intact animal. There are two wires insulated completelyfrom each other, stimulating the brain with their tips.

K K I N F O R C K M E N T BY SEPTAI. STIMULATION 421

subsequent animals. Thus, the first rats had about 12hr. of acquisition after pretesting whereas later rats hadabout 6 hr. However, in computing the scores in ourtable, we have used only the first 6 hr. of acquisitionfor all animals, so the scores are strictly comparable.In behavioral curves, we have shown the ful l 12 hr. olacquisition on the earlier animals so as to illustrate thestability of the behavior over lime.

At mi t ime during the experiment were the ratsdeprived of food or water, and no reinforcement wasused except the electrical stimulus.

Animals were scored on the percentage of timewhich they spent bar pressing regularly during acquisi-tion. In order to find how much time the animal wouldspend in the absence of reward or punishment, a similarscore was computed for periods of extinction. Thisextinction score provided a base line. When the acquisi-tion score is above the extinction score, we have reward;when it is below the extinction score, we have punish-ment .

In order to determine percentage scores, periodswhen the animal was responding regularly (at least oneresponse every 30 sec.) were counted as periods ofresponding; i.e., intervals of 30 sec. or longer without dresponse were counted as periods of no responding. Thepercentage scores were computed as the proportion oftotal acquisition or extinction lime given to periods olresponding.

Determination of LocusOn completion of testing, animals were perfused

with physiological saline, followed by 1(1 per centformalin. The brains were removed, and after furtherfixation in formalin for about a week, frozen sections40 microns thick were cut through the region of theelectrode track. These were stained with cresyl violetand the position of the electrode tip determined. Figure3 is a photomicrograph showing the appearance of theelectrode track in a stained and mounted brain section.

RESULTS

LocusIn Table 1, acquisition and extinction scores

are correlated w i t h electrode placements,

TABLE 1Acquisition and Extinction Scores for All AnimalsTogether with Electrode Placements and Threshold

Voltages Used during Acquisition Tests

Ani-mal'sNo.

3234M-lM-44041318236

Locus ofElectrode

septalseplalseptalseptalc.c.caudatecingulatecingulalehip.

3 m. l .A-56

St imulat ionVoltager.m.s.

2.2-2.8

Percentage ofAcquisitionTime SpentResponding

7.S

PercentaKC ofExtinctionTime SpentResponding

,81.4 «2 61.7-4.8 85 212.3-4.8 88 13

.7-1.1 6 3

.9-1.2 4 41.8 37 9.5-1.8 30 10.8-2.8 11 14.5 0 4

m.t. 1.4 71 9m.g. .5

1 1 m.g.17y

teg.teg.

.5

.7

.5

0 3102

77

211

81

FIG. 3. Photomicrograph showing the electrodetrack in a cresyl-violet stained brain section. The sec-tion is 1 mm. in front of the anterior commissure. Theelectrode protruded through the lateral ventricle andits s t imu la t ing lip was in the septal area.

KEY: f.c., corpus callowm; Inp., hippocampus; m.l., mediallemniscus; m.t., M a m m i l l o l l i a l a m i c t ract ; wi.v., medial fjenicu-tate; te$., tejrmentum.

Figure 4 presents ihe acquisition scores again,this lime on three cross-sec I ional maps of iheral brain, one at the forebrain level, one atthe thalamic level, and one at t h e mid-brainlevel. The position of a score on the mapindicates the electrode placement from whichIhis acquisition score was obtained.

The highest scores are found together i i>the central portion of the forebrain. Beneaththe corpus callosum and between the twolateral ventricles in section I of Figure 4,we- find four acquis i t ion scores ranging from75 lo °2 per cent . This is ihe seplal area.The .S's which produced these scores are num-bered M, ,U, .M-l, and M-4 in Table 1. llw i l l be noticed t h a t w h i l e a l l of them spentmore than 75 per cent of their acquisitionlime responding, lliey all spent less t h a n 22per cent of the i r ex t inc t ion t ime responding.Thus the electrical s t i m u l u s in the seplal areahas an effect which is apparently equivalentto t h a t of a conventional primary reward asfar as t h e maintenance ol a lever-pressingresponse is concerned.

If we move outside the seplal area, eitherin the direction of t h e caudate nucleus (acrossthe lateral ventricle) or in the direction of thecorpus callosum, we find acquisition scoresdrop abruptly lo levels of from 4 lo 6 per

422 JAMKS OLDS AND PETKR M I L N K R

cent . These are definitely indications of neu-Iral (neither rewarding nor punishing) effects.

However, above Ihe corpus callomm in thecingulale cortex we find an acquisition scoreof 37 per cent. As Ihe extinction score in th i scase was 9 per cent, we may say that stimula-t ion was rewarding.

Al the thalamic level (section II of Fig. 4)we lino! a 36 per cent acquisition score pro-duced by an electrode placed again in thecingulale cortex, an 11 per cent score pro-duced by an electrode placed in the hippo-campus, a 71 per cent score produced by anelectrode placed exactly in the mammillo-

thalamic tract , and a zero per ccnl score pro-duced by an electrode placed in the mediallemniscus. The zero denotes negative rein-forcement.

At the mid-brain level (section III of Fig. 4)there are two zero scores produced by elec-trodes which arc in the posterior portion ofIhe medial geniculate bodies; here again, thescores indicate a negative effect, as the cor-responding extinction scores are 31 and 21per cent. There is an electrode deep in themedial, posterior legmenlurn which producesa 2 per cent score; this seems quite neutral , asthe extinction score in this case is 1 per cent .

m

Kic. 4. Maps of three sections, (I) through the forebrain, ( II) through the thalamus, ( I I I ) through the mid-brain of the rat. lioxcd numbers give acquisition percentage scores produced by animals with electrodes stimulat-ing al these points. On section I the acquisition scores 75, 88, 92, 85 fall in Ihe seplal forebrain area. On the samesection there is a score of 4 in the caudate nucleus, a score of 6 in the white matter below Ihe cortex, and a scoreof 37 in the medial (cingulale) cortex. On section tT the acquisition score of 36 is in the medial (cingulale) cortex,I I is in the hippocampus, 71 is in the mammillolhalamic tract, and 0 is in the medial lemniscus. On section IIIthe two zeroes are in the medial geniculate, 2 is in the legmental reticular substance, 77 falls 2 mm. anterior tothe section shown it is between the posterior commissure and the red nucleus.

REINFORCEMENT BY SEPTAL STIMULATION 423

Finally, there is an electrode shown on thissection which actually stands 1)̂ mm. anteriorto the point where it is shown; it was betweenthe red nucleus and the posterior commissure,It produced an acquisition score of 77 percent, but an extinction score of 81 per cent.This must be a rewarding placement, but thehigh extinction score makes it difficult tointerpret.

Behavior

We turn our attention briefly to the be-havioral data produced by the more rewardingelectrode placements.

The graph in Figure 5 is a smoothed cumu-lative response curve illustrating the rate of

responding of rat No. 32 (the lowest-scoringseptal area rat) during acquisition and extinc-tion. The animal gave a total of slightly over3000 responses in the 12 hr. of acquisition.When the current was turned on, the animalresponded at a rate of 285 responses an hour;when the current was turned off, the rate fellclose to zero.

The graph in Figure 6 gives similar data onrat No. 34 (the highest-scoring septal rat). Theanimal stimulated itself over 7500 times in 12hr. Its average response rate during acquisitionwas 742 responses an hour; during extinction,practically zero.

Figure 7 presents an unsmoothed cumula-tive response curve for one day of responding

«J<£

X)cQ

*L.-o

e!3

X

HoursFIG. 5. Smoothed cumulative response curve for rat No. 32. Cumulative response totals

are given along the ordinate, and hours along the abscissa. The steepness of the slope indi-cates the response rate. Stimulating voltages are given between black lines. Cross hatchingindicates extinction.

424 JAMES OLDS AND PETER MILNER

HoursFIG. 6. Smoothed cumulative response curve for rat No. 34

REINFORCEMENT BY SEPTAL STIMULATION 425

H'OURS

FIG. 7. Unsmoothed cumulative response curveshowing about % hr. of acquisition and % hr. extinc-tion for rat No. A-5. Shading indicates extinction.

for rat No. A-S. This is to illustrate in detailthe degree of control exercised by the electricalreward stimulus. While this rat was actuallybar pressing, it did so at 1920 responses anhour; that is, about one response for every 2sec. During the first period of the day it re-sponded regularly while on acquisition, extin-guished very rapidly when the current wasturned off, and reconditioned readily when thecurrent was turned on again. At reconditioningpoints, E gave S one stimulus to show that thecurrent was turned on again, but E did notplace S on the lever. During longer periodsof acquisition, 5 occasionally stopped re-sponding for short periods, but in the long run5 spent almost three-quarters of its acquisitiontime responding. During the long period ofextinction at the end of the day, there was verylittle responding, but S could be brought backto the lever quite quickly if a stimulus wasdelivered to show that the current had beenturned on again.

DISCUSSION

It is clear that electrical stimulation in cer-tain parts of the brain, particularly the septalarea, produces acquisition and extinctioncurves which compare favorably with thoseproduced by a conventional primary reward.With other electrode placements, the stimu-lation appears to be neutral or punishing.

Because the rewarding effect has been pro-duced maximally by electrical stimulation inthe septal area, but also in lesser degrees inthe mammillothalamic tract and cingulatecortex, we are led to speculate that a systemof structures previously attributed to therhinencephalon may provide the locus forthe reward phenomenon. However, as locali-

zation studies which will map the whole brainwith respect to the reward and punishmentdimension are continuing, we will not discussin detail the problem of locus. We will usethe term "reinforcing structures" in furtherdiscussion as a general name for the septalarea and other structures which produce thereward phenomenon.

To provide an adequate canvass of thepossible explanations for the rewarding effectwould require considerably more argumentthan could possibly fit within the confines ofa research paper. We have decided, therefore,to rule out briefly the possibility that theimplantation produces pain which is reducedby electrical stimulation of reinforcing struc-tures, and to confine further discussion tosuggestions of ways the phenomenon mayprovide a methodological basis for study ofphysiological mechanisms of reward.

The possibility that the implantation pro-duces some painful "drive stimulus" which isalleviated by electrical stimulation of rein-forcing structures does not comport with thefacts which we have observed. If there weresome chronic, painful drive state, it wouldbe indicated by emotional signs in the ani-mal's daily behavior. Our 5s, from the firstday after the operation, are normally quiet,nonaggressive; they eat regularly, sleep regu-larly, gain weight. There is no evidence intheir behavior to support the postulation ofchronic pain. Septal preparations which havelived healthy and normal lives for monthsafter the operation have given excellent re-sponse rates.

As there is no evidence of a painful condi-tion preceding the electrical stimulation, andas the animals are given free access to foodand water at all times except while actuallyin the Skinner boxes, there is no explicitlymanipulated drive to be reduced by electricalstimulation. Barring the possibility that stimu-lation of a reinforcing structure specificallyinhibits the "residual drive" state of the ani-mal, or the alternative possibility that thefirst electrical stimulus has noxious after-effects which are reduced by a second one,we have some evidence here for a primaryrewarding effect which is not associated withthe reduction of a primary drive state. It isperhaps fair in a discussion to report the

426 JAMES OLDS AND PETER MILNER

"clinical impression" of the Es that thephenomenon represents strong pursuit of apositive stimulus rather than escape fromsome negative condition.

Should the latter interpretation prove cor-rect, we have perhaps located a system withinthe brain whose peculiar function is to pro-duce a rewarding effect on behavior. Thelocation of such a system puts us in a positionto collect information that may lead to adecision among conflicting theories of re-ward. By physiological studies, for example,we may find that the reinforcing structuresact selectively on sensory or motor areas ofthe cortex. This would have relevance tocurrent S-S versus S-R controversies (8, 9, 13,16).

Similarly, extirpation studies may showwhether reinforcing structures have primarilya quieting or an activating effect on behavior;this would be relevant to activation versusnegative feedback theories of reward(6, 13, 15, 17). A recent study by Brady andNauta (1) already suggests that the septalarea is a quieting system, for its surgical re-moval produced an extremely active animal.

Such examples, we believe, make it reason-able to hope that the methodology reportedhere should have important consequences forphysiological studies of mechanisms of reward.

SUMMARY

A preliminary study was made of rewardingeffects produced by electrical stimulation ofcertain areas of the brain. In all cases ratswere used and stimulation was by 60-cyclealternating current with voltages ranging from% to 5 v. Bipolar needle electrodes were per-manently implanted at various points in thebrain. Animals were tested in Skinner boxeswhere they could stimulate themselves bypressing a lever. They received no other re-ward than the electrical stimulus in the courseof the experiments. The primary findings maybe listed as follows: (a) There are numerousplaces in the lower centers of the brain whereelectrical stimulation is rewarding in the sensethat the experimental animal will stimulateitself in these places frequently and regularlyfor long periods of time if permitted to do so.(b) It is possible to obtain these results from

as far back as the tegmentum, and as farforward as the septal area; from as far downas the subthalamus, and as far up as thecingulate gyrus of the cortex, (c) There arealso sites in the lower centers where the effectis just the opposite: animals do everythingpossible to avoid stimulation. And there areneutral sites: animals do nothing to obtain orto avoid stimulation, (d) The reward results areobtained more dependably with electrodeplacements in some areas than others, theseptal area being the most dependable to date.(e) In septal area preparations, the controlexercised over the animal's behavior by meansof this reward is extreme, possibly exceedingthat exercised by any other reward previouslyused in animal experimentation.

The possibility that the reward resultsdepended on some chronic painful conse-quences of the implantation operation wasruled out on the evidence that no physiologicalor behavioral signs of such pain could be found.The phenomenon was discussed as possiblylaying a methodological foundation for aphysiological study of the mechanisms ofreward.

REFERENCES

1. BRADY, J. V., & NAUTA, W. J. H. Subcorticalmechanisms in emotional behavior: affectivechanges following septal forebrain lesions in thealbino rat. /. camp, physiol. Psychol., 1953,46, 339-346.

2. DELGADO, J. M. R. Permanent implantation ofmultilead electrodes in the brain. Yale J. Biol.Med., 1952, 24, 351-358.

3. DELGADO, J. M. R. Responses evoked in waking catby electrical stimulation of motor cortex. Amer.J. Physiol, 1952, 171, 436-446.

4. DELGADO, J. M. R., & ANAND, B. K. Increase offood intake induced by electrical stimulationof the lateral hypothalamus. Amer. J. Physiol.,1953, 172, 162-168.

5. DELL, P. Correlations cntre le systSme vegetatifet le systftme de la vie relation: mesencephale,diencephale, et cortex cerebral. /. Physiol.(Paris), 1952, 44, 471-557.

6. DEUTSCH, J. A. A new type of behavior theory.Brit. J. Psychol., 1953, 44, 304-317.

7. GASTAUT, H. Correlations entre le systeme nerveuxvegetatif et le systfime de la vie de relation dansle rhinencephale. /. Physiol. (Paris), 1952, 44,431-470.

8. HEBB, D. 0. The organization of behavior. NewYork: Wiley, 1949.

9. HULL, C. L. Principles of behavior. New York:D. Appleton-Century, 1943,

REINFORCEMENT BY SEPTAL STIMULATION 427

10. HUNTER, J., & JASPER, H. H. Effects of thalamicstimulation in unanaesthetized animals. EECdin. Neurophysiol., 1949,1, 305-324.

11. KRIEG, W. J. S. Accurate placement of minutelesions in the brain of the albino rat. Quart. Bull.,Northwestern Univer. Med, School, 1946, 20,199-208.

12. MACLEAN, P. D., & DELGADO, J. M. R. Electricaland chemical stimulation of frontotemporalportion of limbic system in the waking animal.EEG din. Neurophysiol, 1953, 5, 91-100.

13. OLDS, J. A neural model for sign-gestalt theory.Psychol. Rev., 1954, 61, 59-72.

14. ROSVOLD, H. E., & DELGADO, J. M. R. The effecton the behavior of monkeys of electricallystimulating or destroying small areas within thefrontal lobes. Amer. Psychologist, 1953, 8, 425-426. (Abstract)

15. SEWARD, J. P. Introduction to a theory of motiva-tion in learning. Psychol. Rev., 1952, 69, 405-413.

16. SKINNER, B. F. The behavior of organisms. NewYork: D. Appleton-Century, 1938.

17. WIENER, N. Cybernetics. New York: Wiley, 1949.

Received July 15, 1954.