Behavioral/Cognitive

Stimulus Context and Reward Contingency InduceBehavioral Adaptation in a Rodent Tactile Detection TaskX Christian Waiblinger, Caroline M. Wu, Michael F. Bolus, X Peter Y. Borden, and X Garrett B. StanleyWallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, Georgia 30332

Behavioral adaptation is a prerequisite for survival in a constantly changing sensory environment, but the underlying strategies and relevantvariables driving adaptive behavior are not well understood. Many learning models and neural theories consider probabilistic computations asan efficient way to solve a variety of tasks, especially if uncertainty is involved. Although this suggests a possible role for probabilistic inferenceand expectation in adaptive behaviors, there is little if any evidence of this relationship experimentally. Here, we investigated adaptive behaviorin the rat model by using a well controlled behavioral paradigm within a psychophysical framework to predict and quantify changes in perfor-mance of animals trained on a simple whisker-based detection task. The sensory environment of the task was changed by transforming theprobabilistic distribution of whisker deflection amplitudes systematically while measuring the animal’s detection performance and correspond-ing rate of accumulated reward. We show that the psychometric function deviates significantly and reversibly depending on the probabilisticdistribution of stimuli. This change in performance relates to accumulating a constant reward count across trials, yet it is exempt from changesin reward volume. Our simple model of reward accumulation captures the observed change in psychometric sensitivity and predicts a strategyseeking to maintain reward expectation across trials in the face of the changing stimulus distribution. We conclude that rats are able maintain aconstant payoff under changing sensory conditions by flexibly adjusting their behavioral strategy. Our findings suggest the existence of aninternal probabilistic model that facilitates behavioral adaptation when sensory demands change.

Key words: adaptive; behavior; decision making; psychophysics; rodent; whisker

IntroductionOur decisions in everyday situations are governed by sensorystimuli but also, and most importantly, by our experience and bya constantly changing sensory environment. How do we adapt tosuch contextual changes? What is our strategy to deal with adynamically changing environment? In the last decade, therehave been major advances toward answering this question bystudying perceptual decision making across species and askinghow and to what degree neuronal activity reflects behavioralchoice and how sensory information is transformed into adaptive

action (Romo and Salinas, 2003; Gold and Shadlen, 2007; Nien-borg and Cumming, 2009). An important finding is that behav-ioral performance in psychophysical experiments is often notdetermined by sensory processes alone but by a wide range ofbiasing factors, among them prior probabilities, reward payoff,changing stimulus–action associations, and trial history (Busse etal., 2011; Stuttgen et al., 2013; Jaramillo et al., 2014; Akrami et al.,2018; Waiblinger et al., 2018), potentially pointing to a complex,dynamic behavioral strategy reliant on a continuous interplaybetween these factors. Claims about neuronal coding schemestherefore depend crucially on behavioral context and a preciseassessment of the observer’s active task strategy. By consideringthese dynamic cognitive processes, much of the variability ob-served in behavior could be much more predictable comparedwith using simplified psychophysical approaches that do not takethe contextual elements into account. Although there exist a va-riety of useful computational models to explain choice biases orcontextual dependencies in human and animal psychophysicaldatasets (Nassar et al., 2010; Frund et al., 2011, 2014; Wilson et al.,2013; Braun et al., 2018), a definitive understanding of behaviorwithin a dynamic framework does not yet exist.

Here, we directly investigate behavioral strategies in the face ofa dynamically changing sensory environment, using the rodent

Received Aug. 8, 2018; revised Oct. 30, 2018; accepted Nov. 21, 2018.Author contributions: C.W. wrote the first draft of the paper; C.M.W., M.F.B., P.Y.B., and G.B.S. edited the paper;

C.W. and G.B.S. designed research; C.W. and C.M.W. performed research; M.F.B. and P.Y.B. contributed unpublishedreagents/analytic tools; C.W., M.F.B., and P.Y.B. analyzed data; C.W. wrote the paper.

C.W. was supported by a fellowship from the Deutsche Forschungsgemeinschaft (GZ: WA 3862/1-1) and theNational Institutes of Health (Grant U01NS094302). P.Y.B. was supported by an NIH National Research Service Award(Predoctoral Fellowship F31NS09869). M.F.B. was supported by the National Science Foundation (Graduate Re-search Fellowship Grant DGE-1650044). C.M.W. was supported by an Undergraduate Research Scholarship from theGeorgia Tech Petit Institute. G.B.S. was supported by the NIH (Grant R01NS085447 and Brain Grants R01NS104928and U01NS094302).

The authors declare no competing financial interests.Correspondence should be addressed to Garrett B. Stanley at garrett.stanley@bme.gatech.edu.https://doi.org/10.1523/JNEUROSCI.2032-18.2018

Copyright © 2019 the authors 0270-6474/19/391088-12$15.00/0

Significance Statement

The strategy animals use to deal with a complex and ever-changing world is a key to understanding natural behavior. This studyprovides evidence that rodent behavioral performance is highly flexible in the face of a changing stimulus distribution, consistentwith a strategy to maintain a desired accumulation of reward.

1088 • The Journal of Neuroscience, February 6, 2019 • 39(6):1088 –1099

vibrissa pathway as a model system. Specifically, we challengehead-fixed rats in a Go/No-Go detection task with a variablewhisker deflection amplitude drawn from a controlled but prob-abilistic distribution of amplitudes, which varies across sessions.In response to changing stimulus distribution, we observe a dy-namic behavioral strategy reflected in a shift in psychometricsensitivity that is qualitatively consistent with the animal main-taining the expected accumulated reward in the face of a changingenvironment. We quantitatively challenge this hypothesis withina simple model framework of reward accumulation to predictand capture systematic changes in the performance of expert ratsin a range of experiments with changing stimulus and rewardcontingencies. The observed change in psychometric sensitivity islargely predicted by a strategy seeking to maintain reward expec-tation in the face of a changing stimulus distribution and revers-ibly persists in the face of a range of transformations of thestimulus distribution. Finally, analysis of psychometric sensitiv-ity within sessions reveals an asymmetric shift in sensitivity oneven shorter time scales, suggestive of a dynamic strategy basedon an internal model of reward accumulation. A direct manipu-lation of the reward feedback thereby reveals that the number ofpast rewarded trials not the volume corresponds with task en-gagement. Together, the results here provide a simple predictiveframework for adaptive reward-based behaviors in a changingsensory environment.

Materials and MethodsAnimals, surgery, and general procedures for behavioral testingAll experimental and surgical procedures were approved by the GeorgiaInstitute of Technology Institutional Animal Care and Use Committeeand were in agreement with guidelines established by the National Insti-tutes of Health. Subjects were seven female Sprague Dawley rats aged12–16 weeks at time of implantation. The basic procedures of head-capsurgery, habituation to head fixation, and behavioral training exactlyfollowed the ones published in a technical review (Schwarz et al., 2010)and more recent studies (Waiblinger et al., 2015, 2018). In the followingtext, only procedures pertaining to the specific paradigm established hereare described in detail.

Oral antibiotics (Baytril; Bayer injectable solution 2.27%, 20 ml) wereprovided for 1 d before surgery and 1 week postoperatively. The animalswere anesthetized using isofluorane and a head cap for head fixation wasimplanted. The wound was treated with antibiotic ointment and sutured.Analgesia and warmth were provided after surgery. Rats were allowed torecover for at least 10 d before habituation training. Subjects were housedtogether with a maximum number of two in one group cage and keptunder a 12/12 h inverted light/dark cycle. During successive days of be-havioral testing, water intake was restricted to the experimental sessionsin which animals were given the opportunity to earn water to satiety.Testing was paused and water was available ad libitum for 2 d a week.Body weight was monitored daily and was typically observed to increaseduring training. If the body weight dropped for more than �10 g due toa higher task difficulty, supplementary water was delivered outside oftraining sessions to keep the animal’s weight up. The first step of behav-ioral training was systematic habituation to head-fixation lasting for �2weeks. Once rats were trained on the behavioral task, one or two exper-iments were usually conducted per day, comprising 100 –200 trials. Dur-ing behavioral testing, a constant auditory white background noise (70dB) was produced by an arbitrary waveform generator to mask anysound emission of the galvo-motor-based whisker actuator.

Whisker stimulationFor whisker stimulation, a galvo-motor (galvanometer optical scannermodel 6210H; Cambridge Technology) as described in Chagas et al.(2013) was used. The rotating arm of the galvo-motor contacted a singlewhisker on the right of the rat’s face at 5 mm (�1 mm tolerance) distancefrom the skin and thus directly engaged the proximal whisker shaft,largely overriding bioelastic whisker properties. All of the remaining

whiskers were trimmed to prevent them from being touched by the ro-tating arm. Voltage commands for the actuator were programmed inMATLAB and Simulink (version 2015b; The MathWorks). A stimulusconsisted of a single event: a sinusoidal pulse (half period of a 100 Hz sinewave, starting at one minimum and ending at the next maximum). Thepulse amplitudes used (A � [0, 0.25, 1, 4, 16]° or maximal velocities,respectively: V � [0, 78.49, 313.95, 1255.81, 5023.24]°/s) were well withinthe range reported for frictional slips observed in natural whisker move-ment (Ritt et al., 2008; Wolfe et al., 2008).

Behavioral paradigm and trainingAll seven rats were trained on a standard Go/No-Go detection task usinga similar protocol as described before (Stuttgen et al., 2006; Ollerenshawet al., 2012, 2014). In this task, the whisker is deflected at intervals of 4 –10s (flat probability distribution) with a single pulse (detection target). Atrial was categorized a “hit” if the animal generated the “Go” indicatorresponse, a lick at a water spout within 1000 ms of target onset. If no lickwas emitted, the trial counted as a “miss.” In addition, catch trials wereincluded in which no deflection of the whisker occurred (A � 0°) and atrial was categorized as a correct rejection if licking was withheld (No-Go). However, a trial was categorized as a false alarm if random licksoccurred within 1000 ms of catch onset. Premature licking in a 2 s periodbefore the stimulus was mildly punished by resetting time (time-out) andstarting a new intertrial interval of 4 –10 s duration drawn at randomfrom a flat probability distribution. Note that these trial types were ex-cluded from the main psychometric data analysis because they can occurwith a different likelihood as a particular stimulus or catch trial. How-ever, we report these trial types separately and refer to them as “impulsivelicks” in the rest of the manuscript.

During the first training phase, a single pulse with fixed amplitude waspresented interspersed by catch trials (Pstim � 0.8, Pcatch � 0.2). Imme-diately following stimulus offset, a droplet of water became available atthe waterspout to condition the animal’s lick response thereby shapingthe stimulus–reward association. Once subjects showed stable and im-mediate consumption behavior (usually within a few sessions), water wasonly delivered after an indicator lick at the spout within 1000 ms, turningthe task into an operant conditioning paradigm in which the response isonly reinforced by reward if it is correctly emitted after the stimulus.

To assess differences in learning based on stimulus amplitude at thisstage of training, we separated animals into two groups: Group 1 (rats1–3) receiving only high-amplitude stimuli (A � [0 16]°) and Group 2(rats 4 and 5) receiving only small-amplitude stimuli (A � [0 4]°). Toassess learning, training was conducted without manual interference bythe experimenter and with equal conditions across sessions from here on.The learning curve consisted of detection indices across training sessionseach calculated by subtracting the false alarm rate from the hit rate for agiven session (Di � PHIT � PFA). A criterion of 0.75 was used to deter-mine expert level. After expert level was achieved, the psychometric curvewas measured using the method of constant stimuli, which entails thepresentation of repeated stimulus blocks containing multiple stimulusamplitudes. The sequence of stimuli within a block was pseudo-random(i.e., each stimulus and catch was presented once in shuffled order andthen repeated). The experimenter aborted a session when the animalstopped working on the task; that is, when it did not generate lick re-sponses for an entire stimulus block, including the highest-amplitudestimulus.

The main experiments of this study were performed using differentstimulus and reward conditions in a within-subject design. Each experi-mental condition consisted of 8 –10 sessions performed over �5 d. Eachanimal performed 1–2 sessions per day with a minimal break of 3 h inbetween. Stimulus or reward parameters changed once an animal hadcompleted an experimental condition. The different animal groups andexperimental conditions are described in detail in the following section(see also Fig. 1).

Experiment 1. Once our first group of rats (rats 1–3) had learned thetask, psychometric tests were conducted with a manipulation of the stim-ulus distribution range. In a given session, the sequence of trials consistedof repeated stimulus blocks, each block containing multiple stimuli andcatch trial once in shuffled order. Therefore, each stimulus occurred with

Waiblinger et al. • Perceptual Changes in the Awake Behaving Rat J. Neurosci., February 6, 2019 • 39(6):1088 –1099 • 1089

the same probability (i.e., a uniform distribution). In the “high-range”condition, three stimulus amplitudes plus catch trial were used (A � [0,1, 4, 16]°) and presented in multiple successive sessions (sessions 1– 8).Following this (sessions 9 –16), three new stimulus amplitudes were pre-sented (A � [0, 0.25, 1, 4]°), forming the low-range condition. Bothstimulus distributions shared two of the three stimulus amplitudes; how-ever, the largest stimulus amplitude of the high-range condition (16°)was not part of the low range and vice versa, the smallest amplitude of thelow-range condition (0.25°) was not part of the high range. To test thereversibility of potential behavioral changes, another high-range condi-tion was presented at the end (sessions 17–24). For the second group ofrats (rats 4 and 5), the order of experiments was reversed; that is, the firstpsychometric curve was measured by presenting the low-range conditionfirst (A � [0, 0.25, 1, 4]°) before switching to the high-range condition(A � [0, 1, 4, 16]°) and then again back to the low range. A third group ofrats (rats 6 and 7) served as a control group and underwent experimentswithout any changes to the stimulus distribution range; that is, the psy-chometric curves were measured with exactly the same set of stimulithroughout (sessions 1–24).

Experiment 2. In the next experiment, the same animal groups wereused as before, but this time the stimulus distribution range was fixedwhile the relative probabilities were changed (i.e., no longer uniform).Now, all four amplitudes plus a catch trial (A � [0, 0.25, 1, 4, 16]°) wereused for both the “big” and “small” conditions, but in the big condition,the two stimuli with large amplitudes were presented with a higher prob-ability than the two small amplitudes (Pbig � Psmall; P [4, 16]° � 0.36, P[0.25, 1]° � 0.09). In the small condition, the order of stimulus proba-bilities was reversed; that is, the two stimuli with small amplitudes werepresented with a higher probability than the two large amplitudes (Pbig �Psmall; P [4, 16]° � 0.09, P [0.25, 1]° � 0.36). The probability of a catchtrial always remained the same for all conditions (Pcatch � 0.1). Thesequence of manipulations was the same as in Experiment 1: rats 1–3started with the big condition (sessions 1– 8) and then underwent shiftsto small (sessions 9 –17) and back to the big condition (sessions 17–24),whereas the order of manipulations was reversed for rats 4 –5. In all of theabove described conditions, the reward size was kept constant through-out; that is, each correctly detected stimulus (hit) resulted in a waterreward of identical volume (volume per trial � 0.09 ml).

Experiment 3. In a subset of rats (rats 1–3), the distribution range andprobability of stimuli was kept constant throughout all conditions (A �[0, 1, 4, 16]°, p � 0.25), but the reward size was varied systematically; thatis, the drop size was small (0.04 ml), medium (0.09 ml), or big (0.18 ml)in a given hit trial. The reward volume changed only between experimen-tal conditions, not within sessions. The above-described high-range dis-tribution served as baseline and was performed first because it alreadyused the medium reward size. In the following sessions, the stimuli werekept constant but the drop size was decreased by half and then increasedto double the volume. Note that not all animals of this group experiencedall reward conditions (rat 1: half and medium only, rat 2–3: half, me-dium, and big).

Data analysis and statisticsPsychometric data were assessed as response probabilities averagedacross sessions within a given stimulus or reward condition. This wasdone separately for each of the seven animals. Psychometric curves werefit using Psignifit (Wichmann and Hill, 2001a,b; Frund et al., 2011).Briefly, a constrained maximum likelihood method was used to fit amodified logistic function with four parameters: � (the displacement ofthe curve), � (related to the inverse of slope of the curve), � (the lowerasymptote or guess rate), and � (the higher asymptote or lapse rate) asfollows:

P�GO � si � � � �1 � � � �1

1 � exp(�z)(si))

z�si �si � �

�

Response thresholds were calculated from the average psychometricfunction for a given experimental condition using Psignifit. The term

“response threshold” refers to the inverse of the psychometric function atsome particular performance level with respect to the stimulus dimen-sion. Throughout this study, we use a performance level of 50% ( prob-ability of detection � 0.5). Statistical differences between psychophysicalcurves were assessed using bootstrapped estimates of 95% confidencelimits for the response thresholds provided by the Psignifit toolbox.

Reward accumulationLet the stimulus amplitude delivered on the i th trial be denoted as si, thecorresponding reward as ri, and the accumulated reward for N trials asRN. Over N trials, the expected accumulated reward is as follows:

ERN� � �i�1

N

P�si P�GO � siri

where P(si) comes from the experimentally controlled stimulus distribu-tion, P(GO si) is the probability of a positive response (or “Go”) for thegiven stimulus amplitude, and E{} denotes statistical expectation.

We considered the null hypothesis of this behavioral paradigm to bethat animals do not adapt their behavior in response to an experimentallyforced change in stimulus distribution and thus operate from the samepsychometric curve (represented as dotted curves in Figs. 2, 3, and 4). Forexample, in moving from the high-range to the low-range stimulus con-dition (see Fig. 2), this would result in a decrease in the total accumulatedreward for the same number of trials.

As an alternative hypothesis, one possible strategy that the animalcould take in response to a change in the stimulus distribution would beto adjust behavior to maintain the same amount of accumulated rewardduring a session. For example, in moving from high-range stimuli tolow-range stimuli, the accumulated reward would be assumed to be fixedand we can determine a new set of probabilities P(GO si) that define anadapted psychometric function. Note that there is not a unique solution,but one simple possibility is that the original psychometric functionmaintains the same asymptotes (� and �) and false alarm rate, but iscompressed, with a decrease in response threshold and an increase inslope to maintain the same total accumulated reward. We denote thissituation as our hypothetical psychometric function, which is repre-sented as dashed curves in Figure 2.

As outlined above, Experiment 1 involves the high- and low-rangestimulus amplitude distributions, but both are uniform distributions, soP(si) is the same for all si. As shown in Figure 3, we also conductedexperiments (Experiment 2) with nonuniform distributions. Consider,for example, the distributions shown for the Pbig � Psmall in the top panelof Figure 3A, where the probability of the big-amplitude stimuli of 4 and16 degrees is higher than small amplitude stimuli of 0.25 and 1 degrees.Under the hypothesis that animals adapt their behavior to achieve thesame amount of reward when moving from this distribution to the in-verse distribution Pbig � Psmall, we can calculate a new psychometricfunction as done above. Again, there is not a unique solution, so we chosethe simplest possibility by compressing the original psychometric func-tion to the left so that it maintains the same asymptotes (� and �) andfalse alarm rate, but allows a change in response threshold and slope tomaintain the same total accumulated reward. As before, we denote thissituation as our hypothetical psychometric function, represented asdashed curves in Figure 3.

Finally, under the working hypothesis that the animal adjusts to main-tain the total accumulated reward in response to changes in the stimulusdistribution, then changing the amount of reward delivered upon a hittrial should produce predictable effects in the behavior. In a final set ofexperiments (Experiment 3; see Fig. 4), we thus changed the amount ofreward delivered in a successful trial, which changes the value of ri. Thisthen generates predictable changes in the behavior, and thus the psycho-metric function, in comparing the small, medium, and big rewards ofFigure 4A.

ResultsWe trained seven head-fixed rats using a tactile Go/No-Godetection task (Cook and Maunsell, 2002; Stuttgen et al., 2006;

1090 • J. Neurosci., February 6, 2019 • 39(6):1088 –1099 Waiblinger et al. • Perceptual Changes in the Awake Behaving Rat

Ollerenshaw et al., 2012). In this task, the animal must detectpulse-shaped deflections of a single whisker and report the deci-sion by either generating a lick on a waterspout (Go) or by with-holding licking (No-Go) if no stimulus is present (Fig. 1A,B). Atrial is categorized as a hit if the animal responds within 1 s uponstimulation and a miss if no response is emitted. Catch trials inwhich stimulation is absent are also included and a trial is cate-gorized as a false alarm if random licking occurs or as a correctrejection if licks are withheld. However, reward is only deliveredin hit trials. Premature licks 2 s before any trial were mildly pun-ished by a time-out. We report these events as impulsive licks, butdistinguish them from false alarms during catch trials for themain psychometric data analysis. All seven rats achieved expertlevel within 10 sessions of operant behavioral training (data notshown).

As in related studies (Busse et al., 2011), it is helpful to visualizethe relationship between presented sensory stimuli and the resultantbehavior in the form of a block diagram as in Figure 1C. At its core,the behavior is a function of the properties of the current sensory

stimulus through a classical psychometric relationship, where thepresentation of a sensory stimulus maps directly to a probability ofresponse (GO, lick) through a logistic function. To predict behav-ioral dynamics with regard to changing context, the behavioral sen-sitivity of the animal is potentially modulated by both “feedforward”representations of the stimulus distribution and corresponding re-ward expectation, requiring an internal model of stimulus history,and “feedback” representations of actual reward accumulation, re-quiring an internal model of reward history.

To investigate the influence of the contextual variables anddisentangle potential sources of behavioral modulation, we de-signed psychophysical experiments in which probabilistic stimu-lus distributions and reward contingencies were systematicallymanipulated (Fig. 1D). Experiment 1 manipulated the range ofthe distribution of whisker deflection amplitudes; that is, the upperand lower limits of amplitudes presented in a psychophysical test,but importantly involved amplitudes common to both high-rangeand low-range conditions. Experiment 2 held the stimulus rangefixed, but manipulated the shape of the distribution by changing the

RewardGo

1000 ms

R

C

Stimulator

Reward port

E1

Stim-amp (deg) Stim-prob (p) Reward (ml/trial)

0 1 4 16

0 0.25 1 4uniform (0.25)

0 0.25 1 4 16

0 1 4 16

0.36 0.36

0.36 0.36

0.1 0.09 0.09

0.1 0.09 0.09

uniform (0.25)

0

0

0

0.04 0.04 0.04

0.09 0.09 0.090.18 0.18 0.18

0.09 0.09 0.09 0.090

0 0.09 0.09 0.09

E2

E30 0.04 0.04 0.04

Stim-Block

Session

Experiment

Single trial (stim or catch)

[0 1 4 16]º

Block 1 Block 2 Block 3 ...

... ...

Cond 1 Cond 2

Session 1 Session 2 Session 1

Reward

Go/No-Go

Z-10

P

0

0.5

1

Logistic CDF

100

P

ZStimulus

Stim distribution,expected reward

F(...) accumul. rewardH/MImpulsive?

No-Go

FA/CR

Stim

Impulsive?

A Behavioral Setup B Go/No-Go Detection C Conceptual Framework

D Stimulus & Reward Conditions E Experimental Design

Figure 1. Experimental strategy. A, Behavioral setup with head-fixed rat, reward port, and whisker stimulator. A single whisker was deflected in the rostrocaudal plain (R, C). B, Go/No-Godetection task. A punctuate stimulus (10 ms) has to be detected by the rat with an indicator lick to receive reward. A Go trial is categorized as a hit (H) if the animal responds within 1 s upon stimulationand a miss (M) if no response is emitted. Catch trials in which stimulation is absent (No-Go) are categorized as a false alarm (FA) if random licking occurs or as a correct rejection (CR) if licks arewithheld. Reward is only delivered in hit trials. Impulsive licks in a 2 s period before trial onset are mildly punished by a time-out (arrow, 4 –10 s). C, Conceptual behavioral framework. The animal’schoice (Go or No-Go) is modeled as a function of the trial by trial response to a stimulus. The resulting variable z is passed through a logistic function that yields a probability for Go or No-Go responses.Two history-dependent variables are thought to influence the behavioral response: (1) reward expectation is formed by integrating the stimulus distribution (range or probability of a particularamplitude occurring) in a feedforward model (left gray box) and (2) the reward accumulation is integrated and leads to behavioral adjustments in a feedback model (right gray box). D, Overview ofall experiments. Experiment 1 manipulated the range of stimulus amplitudes; that is, the upper and lower limits (top, magenta vs green). Experiment 2 manipulated the probability of a stimuluspresentation (middle, magenta vs green). Experiment 3 manipulated the reward payoff determined by the water drop size in hit trials (bottom, different shades of blue). E, Experimental design.From top to bottom, A stimulus block consisted of three to four deflection amplitudes and one catch trial presented in pseudorandom order. Each session consisted of repeated stimulus blocks. Eachexperiment consisted of multiple sessions and was split into different conditions (colored boxes).

Waiblinger et al. • Perceptual Changes in the Awake Behaving Rat J. Neurosci., February 6, 2019 • 39(6):1088 –1099 • 1091

relative probabilities within the distribution. Experiment 3 held thestimulus distribution fixed, but manipulated the reward payoff de-termined by the water drop size in hit trials. The experiments weredesigned such that, on a single trial, one of four different stimuli or acatch trial was presented after a variable time interval, each withequal or unequal probability (Fig. 1E). A stimulus block consisted ofa trial sequence comprising three to four stimuli and a catch trial inpseudorandom order (e.g., each trial type once per block). A behav-ioral session consisted of repeated stimulus blocks until the animaldisengaged from the task. Therefore, the chosen stimuli occurredrepetitively but randomly within a session. Multiple behavioral ses-sions (maximum two per day) were then performed to measure thepsychometric performance for a given condition. An experimentconsisted of multiple sessions comprising two or three conditions,where each condition is defined by a specific stimulus distributionand reward contingency. To assess long-term behavioral effects, aspecific condition was always kept constant within and across mul-tiple behavioral sessions (seven to eight per condition) before thetask was changed.

Influence of a shift in stimulus distributionFor the first group of rats (rats 1–3), the stimulus distributionconsisted of three different stimulus amplitudes and a catch trial(A � [0, 1, 4, 16]°; Fig. 2A, magenta), each occurring with thesame probability throughout the first part of Experiment 1 (i.e., auniform distribution). We define this as the high-range condi-tion. In the second part, the stimulus distribution consisted of

three new stimulus amplitudes and a catch trial (A � [0, 0.25, 1,4]°; Fig. 2A, green), each occurring with the same probability,which we define as the low-range condition. Importantly, bothconditions share two of the three stimulus amplitudes. However,the largest amplitude of the high range (16°) is not part of the lowrange and, vice versa, the smallest amplitude of the low range(0.25°) is not part of the high range. Figure 2B depicts a typicalpsychometric curve (magenta) of an individual animal perform-ing the task under the high-range condition. In response to a shiftin stimulus distribution (Experiment 1), we consider two ex-treme hypotheses. The null hypothesis (H0) asserts that the ani-mal does not adjust its behavior and thus operates from the samepsychometric function (black dotted curve on top of magentacurve). In moving from the high-range to the low-range condi-tion, this would result in a decreased reward rate for the samenumber of trials. We consider an alternative hypothesis (H1), thatthe animal adapts its behavior to maintain accumulated rewardin the face of a changing stimulus distribution. In this case, we usea simple reward expectation model to predict a hypothetical psy-chometric performance that would maintain reward intakewhen the stimulus distribution changes from the high stimulusrange to the low stimulus range (see Materials and Methods).Note that this prediction represents one of many possible ways tomaintain the same reward in the face of the changing stimulusprobabilities. The black dashed curve in Figure 2B denotes thehypothetical psychometric function with the same lapse andguess rate as the original curve from the rat, but allowing it to shift

Session 1-8 9-16 17-24th

resh

old

(deg

)

0

1

2

3

4

high low high

thre

shol

d (d

eg)

0

1

2

3

4

low high low

thre

shol

d (d

eg)

0

1

2

3

4 rat 1rat 2rat 3

rat 6rat 7

rat 4rat 5

amplitude (deg)0 0.25 1 4 16

P re

spon

se

0

0.2

0.4

0.6

0.8

1 high rangelow rangeH0 H1

high low

P im

puls

ive

0.2

0.3

0.4

0.25 1 4 16

P

0

0.1

0.2

high range

amplitude (deg)0.25 1 4 16

P

0

0.1

0.2

low range

reaction time (s)0 0.3 0.6

Freq

uenc

y

0

20

40

H=278H=177

n trials50 100

Volu

me

(ml)

0

2

4

6

ml

0

2

4

6 n=100 n=103

0

A B C D

E F G

Figure 2. Experiment 1. A, Manipulation of stimulus range (magenta vs green). Every stimulus and catch trial (data not shown) was presented with equal probability ( p � 0.25). B, Psychometriccurves and response thresholds (vertical lines with CI95) for an example animal (rat 2). Each dot corresponds to response probabilities from a single session. Solid curves are logistic fits to the averagedata (seven to eight sessions). Dotted line is a hypothetical curve assuming no change in performance (H0) from high-range to low-range stimuli. Dashed line is a hypothetical curve assuming achange in performance to maintain the same amount of accumulated reward (H1) when switched from high-range to low-range stimuli. C, Water volume accumulated by the same animal underboth conditions. Each line corresponds to one session. Inset, Average total reward volume per session for a given condition. The average number of trials worked by this animal is shown. Error barsrepresent SD; otherwise, the figure conventions are the same as in B. D, Metrics of task engagement. Top, Average probability of impulsive licks leading to time-outs. Error bars indicate CI95. Bottom,Histogram of RTs (poststimulus lick) for stimuli close to response threshold (1° and 4°). Median RTs are indicated by arrows. The number of hits for each condition is shown. E, Response thresholdsin Experiment 1a with high-range stimuli presented first, followed by low-range stimuli and another high-range set (n � 3 rats). F, Response thresholds in Experiment 1b reversing the order ofconditions (n � 2 rats). G, Control experiment with no change in condition (n � 2 rats). The data were split into three equal parts. Data points in E–G represent means across sessions within thesame animal. Error bars indicate CI95.

1092 • J. Neurosci., February 6, 2019 • 39(6):1088 –1099 Waiblinger et al. • Perceptual Changes in the Awake Behaving Rat

to the left such that the expected reward per trial remains con-stant. The experimentally measured psychometric function in thelow-range condition (green) comes quite close to the hypotheti-cal performance level based on the assumption of maintainedreward expectation. Consistent with the model’s prediction, theobserved shift results in a significant decrease in response thresh-old (Thigh � 2.92, Tlow � 0.99, CI95high � [2.34 3.51], CI95low �[0.81 1.81], all in units of degrees; data are reported as mean and95% confidence interval, CI95) and an increase in slope (note thedifference in slope is obscured due to the logarithmic plot; theincrease in probability of response per degree near threshold issignificantly higher for the green compared with magenta curve).Figure 2C depicts the actual trial-by-trial accumulation of watervolume in each session with different conditions color coded.Note that overlaid are results for n � 8 sessions with the high-range distribution (magenta) and n � 8 sessions with the low-range distribution (green). The slope of reward accumulation inthe low-range condition almost matches that of the high-rangecondition and the slope for low case (green) is close to the pre-diction from the maintenance of accumulated reward hypothesis,H1, while being clearly separable from the slope representing thenull hypothesis (dotted line). The total reward volume acquiredon average per session in both conditions further confirms this(Vhigh � 4.04 � 0.88 ml, Vlow � 3.78 � 0.61 ml; Fig. 2C, inset,dashed line is H1 hypothesis, dotted line is H0 hypothesis). Analternative strategy to maintain the total reward in the low-rangecondition could also be to work more trials, which we could notconfirm in this particular dataset (see numbers in Fig. 2C, inset).

Can the behavioral adaptation be explained by a change in theanimal’s task engagement; for example, by overall changes inarousal, vigilance, or fluctuations in general attention and moti-vation that may tightly depend on the experimentally imposedtask structure? Although it is challenging to differentiate betweenthese variables within the Go/No-Go paradigm, we provide twomeasures to infer task engagement, the subject’s impulsivity (I �probability of impulsive lick) leading to time-outs between trials(Fig. 2D, top) and reaction times (RTs) in correctly detected Gotrials (Fig. 2D, bottom).

When the task is switched from the high-range to the low-range condition, the animal generates more impulsive licks onaverage (Ihigh � 0.25, Ilow � 0.34, CI95high � [0.2 0.31], CI95low �[0.3 0.38]). In addition, the same rat shows a trend of slightlyshorter lick RTs upon near threshold stimuli (median RThigh �360 ms, RTlow � 329 ms). Both the animal’s increase in impulsivityand slightly shorter RT suggest an increase in arousal. However, eventhough performance improves systematically across all animals un-dergoing task changes, this effect is clearly inconsistent (e.g., rat 3;Ihigh � 0.21, Ilow � 0.21, RThigh � 244 ms, RTlow � 259 ms, data notshown). To further rule out whether impulsivity has an effect onperformance, we repeated our psychometric analysis by including allspontaneous licks; that is, from a pool of impulsive trials (time-outs)and catch trials, we sampled randomly the same number as stimuluspresentations. Across animals and sessions, an increase in hits doesnot correspond to an increase in impulsivity and false alarms, indi-cating that changes in performance are not due to random guessing(data not shown).

It is possible that the observed transformation of the psycho-metric function is due to a steady increase in learning, whichcould be irreversible and not tied to the changing stimulus distri-bution itself. To address this question, we performed an addi-tional series of tests with reciprocal changes, as well as a controlexperiment with fixed conditions. Figure 2E shows responsethresholds from the first group of rats (rat 1–3) with high-range

stimulus distribution presented first, followed by low-range stimu-lus distribution and another high-range stimulus distribution.Changing the stimulus distribution range reciprocally from high tolow and back to high leads to a dynamic threshold progression,which closely follows the direction of task manipulation, showing adecrease at the first transition and a rebound after the second tran-sition. We also performed the inverse task sequence with a differentgroup of rats (Fig. 2F); that is, we trained naive subjects using low-range stimulus distributions first (rats 4 and 5) and then changed thetask to the high-range and back to the low-range condition. Again,all response thresholds follow the direction of task manipulation,indicating that the animals are able to apply a reciprocal task strategymodulated by the shift in stimulus distribution. Interestingly, thesecond group of rats started with a higher performance level, whichis likely due to their training history (early training with small am-plitude A � 4° and first psychophysical test with low-range stimulusdistribution A � [0, 0.25, 1, 4]°).

Although both the reversibility of the shift in psychometricsensitivity and the persistence of the effect in response to a rever-sal of switching (e.g., high to low and low to high) suggest that theobserved phenomenon cannot just be attributed to a learningeffect, learning obviously plays some role in these trained behav-iors. To better quantify this effect, we conducted an additional setof experiments and analyses. Figure 2G depicts response thresh-olds of two animals that never experienced any change in stimu-lus distributions (rats 6 and 7). In addition to the high variabilityof the individual animal’s response threshold from one session toanother, a permanent decrease in thresholds over many sessionswas observed (averages of n � 8 � 3 sessions). Because the con-text was never changed experimentally, we consider this steadyimprovement in detection performance to be shaped by pro-gressing training over the course of many experiments. Suchlong-term perceptual learning effects have been shown before inprimate psychophysical datasets (Gold et al., 2008).

Influence of stimulus redistributionIn Experiment 1, the change in behavioral performance was inresponse to a relatively large manipulation of the stimulus distri-bution in shifting the range. Is the animal’s strategy sensitive tomore subtle changes in the stimulus distribution? In Experiment2, we performed a separate set of experiments with the sameanimal group, where the stimulus distribution range was fixed,but the relative probabilities were changed (i.e., no longer uni-form). Specifically, all four amplitudes plus a catch trial (A � [0,0.25, 1, 4, 16]°) were used for both the big and small conditions,but in the big condition, the two stimuli with large amplitudeswere presented with a higher probability than the two small am-plitudes (Pbig � Psmall; P [4, 16]° � 0.36, P [0.25, 1]° � 0.09; Fig.3A, magenta). In the small condition, the order of stimulus prob-abilities was reversed; that is, the two stimuli with small ampli-tudes were presented with a higher probability than the two largeamplitudes (Pbig � Psmall; P [4, 16]° � 0.09, P [0.25, 1]° � 0.36;Fig. 3A, green). The probability of a catch trial remained the same(P [0°] � 0.1, data not shown). Figure 3B depicts typical mea-sured psychometric curves for an individual animal performingthe task under both conditions (magenta vs green). Based on thechanges in stimulus distributions, we again used the simple re-ward– expectation model to predict possible changes in psycho-metric sensitivity. Again, the null hypothesis H0 and themaintenance of reward expectation hypothesis H1 are shown asdotted and dashed lines, respectively. Similar effects become ap-parent as seen above with the modulated stimulus distributionrange: if the distribution of stimulus presentations tilts to the left;

Waiblinger et al. • Perceptual Changes in the Awake Behaving Rat J. Neurosci., February 6, 2019 • 39(6):1088 –1099 • 1093

that is, if stimuli with smaller amplitudes occur with a higherprobability, then the predicted psychometric curve also shiftstoward the left with an increase in slope. Again, note that this isplotted logarithmically and that the small (green) psychometriccurve has a significantly larger increase in probability of responseper degree near threshold compared with the big (magenta) psy-chometric curve. Even though the experimentally measured psy-chometric function does not reach this hypothetical performancelevel, it shows a significant decrease in response threshold (Thigh �3.12, Tlow � 1.22, CI95high � [2.63 3.96], CI95low � [1.08 1.43];all numbers in degrees, reported are means and CI95) consistentwith this kind of prediction. The reward volume acquired by therat is not the same between the two conditions (Vhigh � 5.29 �1.46 ml, Vlow � 3.73 � 1.17 ml), but it is clearly separable fromthe theoretical accumulation predicted by the null hypothesis(i.e., where performance does not change in response to thechange in distribution; Fig. 3C). Interestingly, this animal seemedto use the strategy of working more trials in the difficult condi-tion; however, across all animals, we did not find a significanteffect of trials worked under the different task conditions. Again,we performed this experiment for multiple animals with the dif-

ferent conditions in reversing order and further confirmed thefinding described above (Fig. 3D,E).

These results are consistent with the possibility that rewardaccumulation plays an important role in adaptively shaping theanimal’s behavior, where the reward-focused strategy incorpo-rates an inferred model of the stimulus statistics that enablesdetecting the same exact stimulus with different accuracy in dif-ferent contexts. We further find that changes in performance arereversible: the animal’s response threshold decreases significantlywith a change from an easy to a difficult condition and vice versa.

Influence of rewardThe results described above suggest that a change in sensory con-text can cause an adaptation of the animal’s performance, pre-sumably to maintain a certain level of reward. However, it ispossible that such a scenario exists either through an internalmodel of reward accumulation driven by the properties of thesensory stimulus or through direct feedback to the animal in theform of the actual reward. In Experiments 1 and 2, it is the casethat the stimulus distribution influences the reward accumula-tion, so these two possibilities are conflated. To further investi-

amplitude (deg)0 0.25 1 4 16

P re

spon

se

0

0.2

0.4

0.6

0.8

1 Pbig > Psmall

Pbig < Psmall

H0

H1 0.25 1 4 16

P

0

0.2

0.4Pbig > Psmall

amplitude (deg)0.25 1 4 16

P

0

0.2

0.4Pbig < Psmall

0

1

2

3

4

0

1

2

3

4

thre

shol

d (d

eg)

thre

shol

d (d

eg)rat 1rat 2rat 3rat 4

P big > P sm

all

P big < P sm

all

P big > P sm

all

P big < P sm

all

P big > P sm

all

P big < P sm

all

n trials50 100 150

Volu

me

(ml)

0

2

4

6

8

ml

02468 n=98 n=106

0

A B C

D E

Figure 3. Experiment 2. A, Manipulation of stimulus statistics. Stimuli were presented with unequal probabilities, either big amplitudes more often than small amplitudes (magenta, Pbig �0.36,Psmall � 0.09) or vice versa (green, Pbig � 0.09, Psmall � 0.36). Catch trials occurred at an intermediate probability (Pcatch � 0.1, data not shown). B, Psychometric curves and response thresholds(vertical lines with CI95) for an example animal (rat 1). Each dot corresponds to response probabilities from a single session. Solid curves are logistic fits to the average data (seven to eight sessions).Dotted line is a hypothetical curve assuming no change in performance (H0). Dashed line is a hypothetical curve assuming a change in performance to maintain the same amount of accumulatedreward (H1) when switched from easy (Pbig � Psmall) to difficult (Pbig � Psmall). C, Water volume accumulated by the same rat under both conditions. Each line corresponds to one session. Inset,Average total reward volume per session for a given condition. The average number of trials worked by this animal is shown. Error bars indicate SDs; otherwise, the figure conventions are the sameas in B. D, Response thresholds in Experiment 2a with the easy condition presented first (Pbig � Psmall), followed by the difficult condition (Pbig � Psmall) and another easy one (n � 3 rats). E,Response thresholds in Experiment 2b reversing the order of conditions (n � 2 rats). Note that the first and third conditions were always the same. Data points in D and E represent means acrosssessions within the same animal. Error bars indicate CI95.

1094 • J. Neurosci., February 6, 2019 • 39(6):1088 –1099 Waiblinger et al. • Perceptual Changes in the Awake Behaving Rat

gate this issue, we performed another set of experiments with thesame animal group by keeping the stimulus distribution constantthroughout (A � [0, 1, 4, 16]°), but systematically varying thevolume of deterministic reward delivery (Fig. 4A). Specifically,

the drop size in a hit trial was either small (0.04 ml, yellow),medium (0.09 ml, blue), or big (0.18 ml, magenta) in separateexperimental conditions. Figure 4B depicts typical experimen-tally measured psychometric curves for an individual animal per-

n trials50 100 150

Volu

me

(ml)

0

1

2

3

4

5

n trials50 100 150

Num

ber o

f rew

ards

0

10

20

30

40

50

60

Reward Volume

Reward Count

rew

ard

coun

t tot

al

0

20

40

60 n=75n=74

n=55

rew

ard

tota

l (m

l)

0

2

4

6

n=75n=74

n=55

small med big

0.25 1 4 16

P

00.10.2

medium reward0.25 1 4 16

P0

0.10.2

small reward

amplitude (deg)0.25 1 4 16

p

00.10.2

big reward

thre

sh (d

eg)

2

4

small med big

rat 1rat 2rat 3

amplitude (deg)0 0.25 1 4 16

P re

spon

se

0

0.2

0.4

0.6

0.8

1

smallmed big H0H1aH1b

0

0

A B

C D

E F

Figure 4. Experiment 3. A, Manipulation of reward. The water drop size in hit trials was changed from the original drop size (blue, medium � 0.09 ml/trial) to half the size (yellow, small � 0.04ml/trial) or to double the size (magenta, big � 018 ml/trial). B, Psychometric curves and response thresholds (vertical lines with CI95) for an example animal (rat 3) under the three rewardconditions. Same conventions as in Figure 2 and 3. Dotted line is a hypothetical curve assuming no change in performance (H0). Yellow dashed line is a hypothetical curve assuming a change inperformance to maintain the same volume of reward per trial (H1a) when switched from a medium drop size to a small drop size. Magenta dashed line is a hypothetical performance to maintain thevolume of reward (H1b) when switched from a medium drop size to a big drop size. C, Reward volume accumulation of the same rat under all three conditions. D, Average total reward volumeacquired per session for a given condition. The average number of trials worked by this rat is depicted on top. Error bars indicate SDs. E, Same data as in C but plotted as reward count. F, Top, Averagetotal reward count for a given condition. Bottom, Response thresholds of three rats under different conditions (rat 1 experienced only medium and small drops). Data points represent means acrosssessions within the same animal. Error bars indicate CI95.

Waiblinger et al. • Perceptual Changes in the Awake Behaving Rat J. Neurosci., February 6, 2019 • 39(6):1088 –1099 • 1095

forming the task under all three condition, starting with themedium drop size (blue curve). The null hypothesis assumes thatthe animal’s performance is independent of the delivered rewardvolume and thus operates from the same psychometric functionacross all conditions (black dotted curve on top of blue curve). Incontrast, the feedback model makes several predictions for anexperimentally forced change in volume. One possible strategycould be to adjust behavior and compensate to maintain a con-stant accumulated reward volume. In this case, the model pre-dicts the hypothetical performance levels needed to maintainreward volume when moving from the medium drop size to thesmaller (H1a) or the bigger (H1b) drop size. The dashed curves inFigure 4B represent hypothetical psychometric functions (yellowfor a change to small and magenta for a change to big rewards)with the same lapse and guess rate as the original experimentalcurve, but allowing it to shift left or right such that the expectedreward remains constant. Note that one of the model’s predic-tions (yellow dashed curve) is located outside the animal’s per-ceptual range because the chosen reduction from medium tosmall drop size was rather extreme. Interestingly, we did not findany differences in performance; the psychometric curves and re-sponse thresholds for all conditions are almost identical (RTsmall �3.54, RTmed � 3.38, RTbig � 3.24), suggesting that the rewardvolume itself has no effect on the animal’s task strategy. Indeed,when plotting the animal’s reward accumulation under the dif-ferent conditions (each hit trial multiplied with the respectivedrop volume), a drastic difference in slope and total acquiredreward volume per session becomes obvious (Fig. 4C,D). Eventhough the animal established a tendency to work for more trialswhen smaller drops were available, the acquired volume at theend of an experimental session never reached previous levels.Note that an experiment was always aborted when the animalstopped working on the task by not licking for an entire stimulusblock. Figure 4, E and F, shows the same data, but now plotted asa function of reward count (i.e., the cumulative number of timesrewarded), thereby ignoring volume (hit trials not multiplied). Inthis case, the accumulated reward counts per session are verysimilar, but the sessions had markedly different numbers of trials.Across all three animals that underwent this experiment, the ef-fects were robust because we did not see any significant changesin response threshold (Fig. 4F, bottom).

This finding suggests that a constant sensory environmentwith a systematic change in reward volume is not sufficient tomodulate the animal’s task strategy. However, we do not excludethe possibility that the number of past rewarded trials could beimportant for the animal’s task engagement.

Performance adaptation on a finer timescaleThe preceding results demonstrate clear, robust effects of dynam-ically changing stimulus context on behavioral performance.Thus far, we have only examined behavioral changes on a session-by-session or experiment-by-experiment basis. However, it isknown from various studies that a subject’s task strategy canchange within sessions (Boneau and Cole, 1967) and even fromone trial to another (Nienborg and Cumming, 2009; Busse et al.,2011; Frund et al., 2014; Waiblinger et al., 2018). This suggeststhat the adaptive behaviors that we observe could also be theresult of a dynamic process on a finer time scale. To estimatewithin-session fluctuations in performance, we parsed each ex-perimental session into three parts of equal trial numbers (e.g.,n � 3 � 50; Fig. 5A) and calculated the response probabilitiesseparately. Figure 5, B and C, shows separate psychometric func-tions of the first and the last part. Consistent with the classic

findings, the curves shift over to the right (arrows) during the runtime of a session. Interestingly, we find that the shift is clearlydependent on the stimulus context because it is qualitativelylarger in the easy high-range condition (Fig. 5B, magenta)compared with the difficult low-range condition (Fig. 5C,green). To rule out the possibility that this effect was due topoor fitting of the psychometric curve, we inspected thegoodness-of-fit metric of deviance ( D) as well as estimates ofwhere the goodness-of-fit lay in bootstrapped cumulativeprobability distributions of this error metric (CPE) using thepsignifit toolbox (Wichmann and Hill, 2001a). Our deviancestatistics revealed that the data was fit equally well across con-ditions with the exception of two outliers that likely occurreddue to subsampling of trials with highly variable performance(rat 1, low range, third part, D � 5.33, and rat 2, high range,third part; D � 9.51, both with a CPE � 95, indicating agoodness-of-fit outside the upper confidence limit).

When further comparing the derived response thresholds forfirst, second, and third part of the session (Fig. 5D), a significantincrease is obvious toward the end of the high-range condition(magenta, third part), whereas very little to almost no changeoccurs in the low-range condition (green, third part). Therefore,the difference in behavioral performance as measured by the psy-chometric curves between the high- and low-range stimulus dis-tributions in Experiment 1 and between the big and smallstimulus distributions in Experiment 2 does not appear to resultfrom a static, fixed property of the behavior in these differentstimulus contexts. Instead, it appears to result from a dynamicprocess whereby the animal relaxes vigilance gradually throughthe session for the “easy” (high-range and big) stimulus distribu-tions. In contrast, the behavioral performance was relatively in-variant for the more “difficult” stimulus distributions.

To rule out satiety effects, we repeated this analysis by parsingeach experimental session into three parts of accumulated rewardvolume (e.g., 0 –1 ml, 1–2 ml, 2–3 ml), thereby ignoring trialnumbers (data not shown). Because animals work different num-bers of trials and acquire different amounts of water in each ses-sion, the upper volume limit was determined for an animal’stypical session (rat 1: 4.5 ml, rat 2: 3 ml, rat3: 3.5 ml) and all trialsbeyond this limit were discarded. Consistent with the previousresults (Fig. 5), all three rats show performance changes that areagain more dramatic for the high-range condition and this effectpersists even though the same amount of water is consumedacross conditions. This indicates that the behavioral adaptationdescribed here is stimulus dependent and cannot solely be attrib-uted to satiety effects.

The results here validate our predictions of behavioralmodulation due to both feedforward representations of thestimulus distribution and corresponding reward expectationand feedback representations of actual reward accumulation(Fig. 1C). It is in principle difficult to disentangle these twosources of modulation, but the results here suggest that, in-deed, the statistical properties of the stimulus–reward rela-tionship strongly influence the behavior and that the effectsare clearly invariant to the absolute amount of reward, consis-tent with a feedforward influence. However, an alternate pos-sibility is that the relevant feedback signal is not theaccumulated volume of reward, but instead an accumulatedrunning count of rewards received.

DiscussionIn this study, we have investigated adaptive behavior in a rodenttactile detection task. Our findings provide evidence that a chang-

1096 • J. Neurosci., February 6, 2019 • 39(6):1088 –1099 Waiblinger et al. • Perceptual Changes in the Awake Behaving Rat

ing sensory environment and associated reward expectation havea substantial impact on the animal’s behavior. We present thefollowing novel aspects. First, we show that metrics of perfor-mance deviate significantly and reversibly depending on theprobabilistic distribution of stimulus amplitudes. Second, weshow that this change in performance relates to accumulating aconstant reward count across trials. Third, the behavioral adap-tation determines task engagement within a behavioral session.

Metrics of behavioral adaptationIt seems reasonable to assume that the behavioral adaptationdescribed here can be explained by cognitive aspects such asdifferent levels of arousal, vigilance, or fluctuations in generalattention and motivation that may tightly depend on the experi-mentally imposed task structure. Furthermore, learning and sa-tiety effects could dominate our results because they play a majorrole in Go/No-Go detection behavior. For instance, over the

rat 1 rat 2 rat 3

1st part

Block 1 Block 2 ... Block 1 Block 2Block 1 Block 2 ...

2nd part 3rd part

...

Session

amplitude (deg)0 0.25 1 4 16

P re

spon

se

0

0.2

0.4

0.6

0.8

1

amplitude (deg)0 0.25 1 4 16

0

0.2

0.4

0.6

0.8

1

amplitude (deg)0 0.25 1 4 16

0

0.2

0.4

0.6

0.8

1

amplitude (deg)0 0.25 1 4 16

P re

spon

se

0

0.2

0.4

0.6

0.8

1

amplitude (deg)0 0.25 1 4 16

0

0.2

0.4

0.6

0.8

1

amplitude (deg)0 0.25 1 4 16

0

0.2

0.4

0.6

0.8

1

1st 2nd 3rd

thre

shol

d

0

5

10

15

part of the session1st 2nd 3rd

0

5

10

15

1st 2nd 3rd0

5

10

15

A

C

D

B

Figure 5. Behavioral adaptation within sessions. A, Each session was subdivided into three equal parts to assess detection performance across the run time of a session. B, Psychometric curvesfor the first and last part of a session in the high-range experiment. C, Psychometric curves for the first and last part of a session in the low-range experiment. Circles represent average responseprobabilities (filled circles are the first part, empty circles are the last part). Curves are logistic fits. Arrows indicate the shift in response threshold. D, Evolving response thresholds for first, second,and third part of a session separately plotted for the high-range condition (magenta) and low-range condition (green). All data points represent means across corresponding parts of multiplesessions (n � 7– 8). Error bars indicate CI95. Data are shown separately for three different animals.

Waiblinger et al. • Perceptual Changes in the Awake Behaving Rat J. Neurosci., February 6, 2019 • 39(6):1088 –1099 • 1097

course of a session, the animal may respond progressively lessoften because of decreased motivation to obtain reward, whichcould lead to the false conclusion that the ability to detect stimulihas diminished.

On the experimental side, we address these issues in multipleways. We control each animal’s impulsivity by using time-outs uponearly guesses, therefore focusing the animal’s attention on the pres-ence of a stimulus. We control satiety by flexibly adjusting the num-ber of trials until responding stops or by disentangling theprobabilistic distribution of sensory inputs and accumulation of re-ward volume in separate tests (Experiments 1 and 2 vs Experiment3). Finally, we control learning effects by either reversing all changes(e.g., high to low, back to high range) or by keeping all task-relatedparameters constant throughout experiments.

On the analytical side, we infer behavioral performance fromestimates of the psychometric function (Wichmann and Hill,2001a). In our case, the function is obtained by fitting a logisticcurve to the measured data points that represent the animal’sresponse probabilities given a distribution of stimuli. We con-sider the response threshold at p � 0.5 as an optimal metric ofdetection performance because it refers to the critical horizontalshift of the psychometric function along the stimulus amplitudeaxis. In this context, it is important to note that theories thatassume a hard threshold cannot explain decision making in psy-chophysical tasks (Swets, 1961). Because our effects of changingresponse thresholds are highly significant, persistent across ex-periments (Experiment 1 and 2), and further reversible, we con-clude that changes in performance are clearly stimulus dependentand cannot be explained by learning effects.

In addition to psychometrics, we provide several measures oftask engagement: (1) the subject’s spontaneous guessing or im-pulsivity leading to time-outs between trials, (2) RTs, and (3) wereanalyzed the data given different satiety levels.

When the task is switched from the easy high-range conditionto the difficult low-range condition, some individuals indeedshow an increase in impulsivity, resulting in more time-outs andslightly shorter RTs, suggesting that performance changes due toincreased levels of arousal. However, the inconsistencies in theseresults across subjects and experiments lead us to conclude thatbehavioral adaptation is not solely due to a change in the animal’sarousal or motivational state.

To rule out satiety effects, we split sessions into equal subsetsand repeated the psychometric analysis (parsed by number oftrials or reward volume). Performance changes are more dra-matic for the easier stimulus distribution and this effect persistseven though the same amount of water is consumed across con-ditions. This result shows that satiety can have a general andsubstantial influence on behavior (increase in lapse rate), but itcannot account for the changes in performance under differenttask conditions. This notion is further supported by data fromExperiment 3 (Fig. 4) showing that psychometric curves arehighly persistent toward dramatic manipulations in reward vol-ume. Surprisingly, the same data suggest that the number of pastrewarded trials seems to determine the animal’s task engagement,thus showing the ability to integrate a distribution of accomplish-ments across trials.

The experiments here were designed to directly probe rewardexpectation in the case when the relationship between perfor-mance and reward is fixed (i.e., a hit was always rewarded). How-ever, the findings also predict that a manipulation of rewardprobability upon hit trials, an experiment that has not been per-formed here, would have an impact on task performance if theadaptive behavior were indeed tied to the actual reward as op-

posed to the stimulus properties that dictate rewards. Indeed,many studies using paradigms with asymmetric reward contin-gencies show that animals are highly sensitive to changes in thefrequency or probability of reward (Herrnstein, 1961; Reynolds,1961; Nevin et al., 1975; Balci et al., 2009; Teichert and Ferrera,2010; Stuttgen et al., 2011, 2013). This would open up a set ofadditional questions in the context of this study that would beimportant to explore.

These findings suggest that the behavioral adaptation de-scribed here cannot be explained by a single parameter such aschanges in arousal, satiation, or learning. Instead, we proposethat the change in performance results from a complex integra-tion of the above.

A recent study identifying relevant features for Go/No-Gobehavioral variability on timescales from a few trials to an entiresession further supports this notion (Waiblinger et al., 2018).

Decision theoretical aspectsOur focus on adaptive behavior in a dynamic sensory environ-ment is motivated by the fact that human and animal behavior isoften consistent with probabilistic computations (Bayes, 1958;De Finetti et al., 1993; Van Horn, 2003). Especially in tasks in-volving uncertainty, it is efficient to represent knowledge withprobability distributions and to acquire new knowledge by followingthe rules of probabilistic reasoning. Recent theories have evolved toinvestigate probabilistic computations in the sensory, motor, andcognitive domains at the level of neural circuits (for review, seePouget et al., 2013). An important aspect of these theories is that theycircumscribe a wide range of tasks from sensory processing to high-level cognition. However, insights into the neural basis of perceptualdecisions have come mainly from primates and computationallearning models have been characterized mostly for complex humanpsychophysical datasets (Nassar et al., 2010). New questions are aris-ing that might be ideally answered in the rodent, especially withmore recent advances in tools for measurement and manipulation(Knopfel et al., 2006; Jin et al., 2012; Borden et al., 2017).

By using principles of signal detection theory (Green andSwets, 1966), we consider two hypothetical scenarios how thechange in performance reported here could be explained withinneuronal circuits. Behavioral adaptation can either be due to in-ternal changes in sensitivity (discriminability, d ), decision crite-rion (bias), or both (Luo and Maunsell, 2018). A decision makermay improve sensitivity by reducing the overlap between signaland noise distributions. Alternatively, the decision maker mayvalue hits and false alarms differently by altering the criterion.These two changes can be distinguished by the decision makers’false alarm rate. An improvement in hit rate brought about by adecrease in criterion is associated with more false alarms, whereasthe same increase in hit rate brought about by an increase insensitivity is associated with fewer false alarms.

Our data clearly show an increase of hits for a particular stim-ulus within the low-range distribution; however, animals did notalways exhibit full adaptation as predicted by our reward model(Fig. 3B). Because there is no consistent change of false alarms orimpulsivity in our data, we rule against the interpretation of changesin criterion or sensitivity alone and propose that rats indeed adopt amixed strategy that is further compromised by some amount ofcognitive effort. Again, this implies that decisions do not occur inisolation, but rather depend on accomplishments or failures at dif-ferent points in time. This hypothesis is supported by a large body ofliterature suggesting that behavioral actions are not simply based oncurrent sensory observations, but are often based on a statisticallyoptimal integration of sensory observations and the subjects predic-

1098 • J. Neurosci., February 6, 2019 • 39(6):1088 –1099 Waiblinger et al. • Perceptual Changes in the Awake Behaving Rat

tions or prior knowledge (Shadmehr et al., 2010). Priors, historybiases, and changing stimulus–action associations can partly affectneuronal computations at primary sensory and higher-order corti-cal levels (Busse et al., 2011; Jaramillo et al., 2014; Akrami et al., 2018;Waiblinger et al., 2018).

Our current study supports the notion of statistical integra-tion and probabilistic computations in the rodent brain and pres-ents the novel aspect of behavioral adaptation in a well controlleddynamic framework. By systematically changing the sensory en-vironment, we are able to modulate the rats’ behavioral strategyconsistent with the probabilistic distribution of sensory inputs.Our simple model of behavioral adaptation captures the ob-served change in psychometric sensitivity and predicts a strategyseeking to maintain reward counts in the face of the changingstimulus distribution. Therefore, we propose that rats rely on aninternal model integrating the distribution of sensory inputsacross trials and altering their responses in a probabilistic mannerto maintain the desired payoff with minimal effort.

ReferencesAkrami A, Kopec CD, Diamond ME, Brody CD (2018) Posterior parietal

cortex represents sensory history and mediates its effects on behaviour.Nature 554:368 –372. CrossRef Medline

Balci F, Freestone D, Gallistel CR (2009) Risk assessment in man and mouse.Proc Natl Acad Sci U S A 106:2459 –2463. CrossRef Medline

Bayes T (1958) Essay towards solving a problem in the doctrine of chances.Biometrika 45:293–315. CrossRef

Boneau CA, Cole JL (1967) Decision theory, the pigeon, and the psycho-physical function 1. Psychol Rev 74:123–135. CrossRef Medline

Borden PY, Ortiz AD, Waiblinger C, Sederberg AJ, Morrissette AE, Forest CR,Jaeger D, Stanley GB (2017) Genetically expressed voltage sensor Ar-cLight for imaging large scale cortical activity in the anesthetized andawake mouse (erratum). Neurophotonics 4:039801. CrossRef Medline

Braun A, Urai AE, Donner TH (2018) Adaptive history biases result fromconfidence-weighted accumulation of past choices. J Neurosci. Advance on-line publication. Retrieved January 25, 2018. doi: 10.1523/JNEUROSCI.2189-17.2017

Busse L, Ayaz A, Dhruv NT, Katzner S, Saleem AB, Scho ML, Zaharia AD,Carandini M (2011) The detection of visual contrast in the behavingmouse J Neurosci 31:11351–11361. CrossRef

Chagas AM, Theis L, Sengupta B, Stuttgen MC, Bethge M, Schwarz C (2013)Functional analysis of ultra high information rates conveyed by rat vibris-sal primary afferents. Front Neural Circuits 7:190. CrossRef Medline

Cook EP, Maunsell JH (2002) Dynamics of neuronal responses in macaqueMT and VIP during motion detection. Nat Neurosci 5:985–994. CrossRefMedline

De Finetti B, Machi A, Smith, A (1993) Theory of Probability: a CriticalIntroductory Treatment. New York, NY: Wiley.

Frund I, Haenel NV, Wichmann FA (2011) Inference for psychometricfunctions in the presence of nonstationary behavior. J Vis 11:16. CrossRefMedline

Frund I, Wichmann FA, Macke JH (2014) Quantifying the effect of intertrialdependence on perceptual decisions. J Vis 14:9. CrossRef Medline

Gold JI, Shadlen MN (2007) The neural basis of decision making. Annu RevNeurosci 30:535–574. CrossRef Medline

Gold JI, Law CT, Connolly P, Bennur S (2008) The relative influences ofpriors and sensory evidence on an oculomotor decision variable duringperceptual learning. J Neurophysiol 100:2653–2668. CrossRef Medline

Green DM, Swets JA (1966) Signal detection theory and psychophysics. So-ciety. New York: Wiley.

Herrnstein RJ (1961) Relative and absolute strength of response as a functionof frequency of reinforcement, 12. J Exp Anal Behav 4:267–272. CrossRefMedline

Jaramillo S, Borges K, Zador AM (2014) Auditory thalamus and auditorycortex are equally modulated by context during flexible categorization ofsounds. J Neurosci 34:5291–5301. CrossRef Medline

Jin L, Han Z, Platisa J, Wooltorton JR a, Cohen LB, Pieribone VA (2012)Single action potentials and subthreshold electrical events imaged in neu-

rons with a fluorescent protein voltage probe. Neuron 75:779 –785.CrossRef Medline

Knopfel T, Díez-García J, Akemann W (2006) Optical probing of neuronalcircuit dynamics: genetically encoded versus classical fluorescent sensors.Trends Neurosci 29:160 –166. CrossRef Medline

Luo TZ, Maunsell JHR (2018) Attentional changes in either criterion orsensitivity are associated with robust modulations in lateral prefrontalcortex. Neuron 97:1382–1393.e7. CrossRef Medline

Nassar MR, Wilson RC, Heasly B, Gold JI (2010) An approximately Bayes-ian delta-rule model explains the dynamics of belief updating in a chang-ing environment. J Neurosci 30:12366 –12378. CrossRef Medline

Nevin JA, Olson K, Mandell C, Yarensky P (1975) Differential reinforcementand signal detection. J Exp Anal Behav 24:355–367. CrossRef Medline

Nienborg H, Cumming BG (2009) Decision-related activity in sensory neu-rons reflects more than a neuron’s causal effect. Nature 459:89 –92.CrossRef Medline

Ollerenshaw DR, Bari BA, Millard DC, Orr LE, Wang Q, Stanley GB (2012)Detection of tactile inputs in the rat vibrissa pathway. J Neurophysiol108:479 – 490. CrossRef Medline

Ollerenshaw DR, Zheng HJV, Millard DC, Wang Q, Stanley GB (2014) Theadaptive trade-off between detection and discrimination in cortical rep-resentations and behavior. Neuron 81:1152–1164. CrossRef Medline

Pouget A, Beck JM, Ma WJ, Latham PE (2013) Probabilistic brains: knownsand unknowns. Nat Neurosci 16:1170 –1178. CrossRef Medline

Reynolds GS (1961) Behavioral contrast. J Exp Anal Behav 4:57–71. CrossRefMedline

Ritt JT, Andermann ML, Moore CI (2008) Embodied information process-ing: vibrissa mechanics and texture features shape micromotions in ac-tively sensing rats. Neuron 57:599 – 613. CrossRef Medline

Romo R, Salinas E (2003) Flutter discrimination: neural codes, perception,memory and decision making. Nat Rev Neurosci 4:203–218. CrossRef Medline

Schwarz C, Hentschke H, Butovas S, Haiss F, Stuttgen MC, Gerdjikov TV,Bergner CG, Waiblinger C (2010) The head-fixed behaving rat: proce-dures and pitfalls. Somatosens Mot Res 27:131–148. CrossRef Medline

Shadmehr R, Smith MA, Krakauer JW (2010) Error correction, sensory pre-diction, and adaptation in motor control. Annu Rev Neurosci 33:89 –108.CrossRef Medline

Stuttgen MC, Ruter J, Schwarz C (2006) Two psychophysical channels ofwhisker deflection in rats align with two neuronal classes of primaryafferents. J Neurosci 26:7933–7941. CrossRef Medline

Stuttgen MC, Yildiz A, Gunturkun O (2011) Adaptive Criterion Setting inPerceptual Decision Making. J Exp Anal Behav 96:155–176. CrossRefMedline

Stuttgen MC, Kasties N, Lengersdorf D, Starosta S, Gunturkun O, Jakel F(2013) Suboptimal criterion setting in a perceptual choice task withasymmetric reinforcement. Behav Processes 96:59 –70. CrossRef Medline

Swets JA (1961) Is there a sensory threshold? Science 134:168 –177.CrossRef Medline

Teichert T, Ferrera VP (2010) Suboptimal integration of reward magnitudeand prior reward likelihood in categorical decisions by monkeys. FrontNeurosci 4:1–13. CrossRef Medline

Van Horn KS (2003) Constructing a logic of plausible inference: a guide toCox’s theorem. Int J Approx Reason 34:3–24. CrossRef

Waiblinger C, Brugger D, Whitmire CJ, Stanley GB, Schwarz C (2015) Sup-port for the slip hypothesis from whisker-related tactile perception of ratsin a noisy environment. Front Integr Neurosci 9:53. CrossRef Medline

Waiblinger C, Whitmire CJ, Sederberg A, Stanley GB, Schwarz C (2018)Primary tactile thalamus spiking reflects cognitive signals. J Neurosci 38:4870 – 4885. CrossRef Medline

Wichmann FA, Hill NJ (2001a) The psychometric function: I. fitting, sam-pling, and goodness of fit. Percept Psychophys 63:1293–1313. CrossRefMedline

Wichmann FA, Hill NJ (2001b) The psychometric function: II. bootstrap-based confidence intervals and sampling. Percept Psychophys 63:1314 –1329. CrossRef Medline

Wilson RC, Nassar MR, Gold JI (2013) A mixture of delta-rules approxima-tion to bayesian inference in change-point problems. PLoS Comput Biol 9:e1003150. CrossRef

Wolfe J, Hill DN, Pahlavan S, Drew PJ, Kleinfeld D, Feldman DE (2008)Texture coding in the rat whisker system: slip-stick versus differentialresonance. PLoS Biol 6:e215. CrossRef Medline

Waiblinger et al. • Perceptual Changes in the Awake Behaving Rat J. Neurosci., February 6, 2019 • 39(6):1088 –1099 • 1099