Bi156 lecture 1/11/12

Reward and drug effects

•  The neurotransmitter dopamine (DA) has a crucial role in motivational control—in learning what things in the world are good and bad, and in choosing actions to gain the good things and avoid the bad things.

•  “Sex and drugs and rock n’ roll, •  Are all my brain and body need.” •  -Ian Dury and the Blockheads, 1980s

Reward circuits •  Reward is a brain response that produces sensations that

are perceived as pleasurable or positive, as a consequence of the activation of specific brain areas.

•  Natural rewards include food, liquids, and sex. •  A hungry rodent will repeatedly press a bar that causes

delivery of food pellets. •  A rat will also press a bar that causes direct electrical

stimulation of neurons in reward areas. •  One can thus define circuits involved in reward by finding

the brain areas that will sustain such electrical self-stimulation.

•  Expectation of reward directly generates the motor response (bar press).

Dopamine neurons in reward pathways •  Some brain areas that will sustain electrical self-

stimulation are are part of a circuit containing dopaminergic (DA) neurons.

A key dopamine circuit involved in reward

•  The circuit we will consider involves the shell region of the nucleus accumbens, which receives a DA projection from the ventral tegmental area and glutamatergic projections from the prefrontal cortex.

•  The major output of the nucleus accumbens is to the ventral pallidum, which in turn connects to other motor nuclei.

•  This ‘motive circuit’ allows the animal to produce the appropriate motor response to the reward (or expectation of a reward).

•  The nucleus accumbens may integrate DA and glu input to generate the appropriate response.

•  The projection from the nucleus accumbens to the ventral pallidum is inhibitory (GABAergic).

Dopamine receptors •  There are at least 6 DA receptors; these are all 7-helix

(GPCR) receptors (no DA-gated ion channels are known), so their effects are slow.

•  They usually modulate excitatory (glu) synaptic transmission.

•  Presynaptic DA autoreceptors also regulate DA release from DA neurons.

•  The D1 receptor couples to Gs adenylyl cyclase activation elevated cAMP

•  The D2 receptor couples to Gi. Giα AC inhibition decreased cAMP; Giβγ directly opens K+ channels.

•  Other DA receptors couple to phospholipase C and cause release of Ca 2+ from intracellular stores.

Dopamine metabolism

•  DA is synthesized from tyrosine in presynaptic terminals. •  DA is removed from the synaptic cleft by the DA

transporter (DAT), which catalyzes reuptake of DA into presynaptic terminals.

•  DA acts at a longer distance than does Glu. It is not always released into a single synaptic cleft, and can diffuse before reuptake so as to to affect multiple neurons.

DA metabolism and signaling at a synapse

Requirement for DA in response to brain stimulation reward

•  This experiment shows that rats, which normally press the lever 100 times/min to receive direct electrical stimulation, extinguish this response rapidly when DA receptors are blocked.

What does DA release in the nucleus accumbens do?

•  The simplest hypothesis would be that DA signals the presence of a reward; that is, whenever the animal experiences a reward DA is released from VTA terminals in the NAc, and this leads to reinforcement of the behavior that generates the reward (e.g. lever pressing in rats).

•  Blockage of DA receptors by neuroleptics (DA antagonists) causes decreased responsiveness to the reward, indicating that DA is necessary for perception of reward.

•  Also, some drugs of abuse (psychostimulants), which may be stronger versions of natural rewards, elevate DA levels through inhibition of uptake via DA transporters.

•  However, as we’ll see, DA does not appear to signal simply the presence or absence of a reward, but has much more complex roles.

Is the DA system simply a higher-order motor system?

•  In rats, it is impossible to distinguish between action (lever-press) and the subjective experience of a reward.

•  Some researchers argue that DA generates the motivation to produce a motor action to receive a reward, and does not reflect the experience of the reward itself.

•  The DA system is in fact involved in movement, as Parkinson’s patients have problems in initiating any motor action.

•  These investigators believe that you can’t make a distinction between DA “motor” areas like the caudate putamen (substantia nigra in humans) and DA “reward” areas like the VTA; in both cases the role of DA is to allow execution of a motor action.

•  DA-deficient rodents can still exhibit reward responses to drug and food stimuli if their motor abilities are restored.

Human case studies argue that DA directly affects the perception of stimuli as rewarding

•  There are quite a few cases like this one, where treatment with DA agonists, especially those acting on the DA3 receptor, produce compulsive gambling and other altered responses to rewards.

DA release signals whether a reward is better or worse than predicted

Cortical and subcortical responses to expected and unexpected rewards

•  DA transmission does not simply signal the presence of a rewarding stimulus, but whether that stimulus is unpredicted, or if predicted is better or worse than the prediction.

•  Experience of a reward of the predicted level does not cause DA release, and does not induce Fos (an IEG) in ventral tegmental area DA neurons.

•  An unexpected reward, or one which exceeds expectations, does activate DA release and also induces Fos in VTA neurons.

•  DA release may ensure that the behavioral response to the unexpected reward is greater than that to an expected reward.

‘Actor-critic’ model for DA’s function in reward

•  DA transmission is like a ‘gate’ which regulates the behavioral consequences of environmental stimuli.

•  The magnitude of DA release would represent an ‘error signal’ which would indicate whether the reward is of the expected magnitude, and therefore would influence motivation to repeat the behavior that generates the reward.

Cellular/molecular implementation of actor-critic model

Circuits involved in DA gating of reward responses •  The ‘actor’ involves excitatory (glu) pathways from

prefrontal cortex and other areas to nucleus accumbens spiny projection (output) neurons. These encode perceptions of rewarding events.

•  DA synapses of ventral tegmental area neurons are located on the spiny projections neurons’ dendrites, proximal to Glu synapses.

•  The DA synapses made by VTA neurons in the NAc, which are responding to signals from cortex about the amount of reward obtained from the event, function as the ‘critic’, regulating the strength of the effects of the Glu synapses on firing of the NAc output neurons.

•  DA transmission would thus determine the firing rate output from NAc in response to the reward signal from cortex.

Effects of DA signaling on output from the nucleus accumbens, contd.

•  The effects of DA release onto nucleus accumbens neurons are complex because these neurons express multiple DA receptors mediating opposing signals.

•  Also, the effects of DA on excitation depends on the amount of stimulation (excitatory tone) impinging on the nucleus accumbens projection neurons.

How does reception of rewards cause VTA neuron firing and DA release in the

NAc?

•  Natural rewards include food, sex, liquids, and exercise. •  These may act by causing transient release of endogenous

opioid peptides (endorphins and others). •  These peptides hyperpolarize GABAergic interneurons in

the ventral tegmental area. •  This in turn causes disinhibition of A10 DA neurons that

project to the nucleus accumbens.

Acute drug effects and reward circuits •  Drugs are likely to be stronger versions of natural rewards. •  Some drugs are known to stimulate DA reward pathways. •  Dependence on drugs may be a consequence of this

activation of reward pathways.

Assaying drug reward

•  Self-stimulation: rats will voluntarily press a bar to inject themselves with a drug.

•  Conditioned place-preference: if rats are given drug in one environment and saline in another environment, they choose to spend more time in the drug-associated environment.

Psychostimulants: cocaine and amphetamines

•  Both cocaine and amphetamines inhibit DAT (and also the serotonin (5HT) transporter).

•  Thus, these drugs increase the amount of time that DA is present after release, and therefore amplify the effects produced in the NAc by firing of ventral tegmental area DA neurons.

•  The distinction in drug reward vs. natural reward might reflect the levels of DA attained in the NAc, simply due to decreased uptake.

•  However, drugs also induce increased DA release by stimulating firing of VTA DA neurons.

VTA DA neurons exhibit LTP

•  Stimulation through multiple inputs produces long-term potentiation (LTP) in VTA DA neurons.

•  LTP induction might increase firing probability and therefore increase DA release in the NAc target area.

Psychostimulants induce LTP in DA neurons

•  Amphetamine produces an increase in AMPAR currents, reflecting LTP-mediated induction of recruitment of AMPARs from intracellular vesicles to the cell surface.

•  This increase would produce long-lasting increases in DA release in the NAc.

Summary of drug effects on synaptic plasticity at inputs to VTA DA neurons

Timecourse of cocaine-induced DA elevation in the NAc

•  Cocaine self-administration (blue bar) produces a rapid rise in DA.

•  This is much too fast to be due to a biochemical effect on the transporter (or probably to any other biochemical effect).

•  This DA increase thus likely reflects an expectation of reward.

•  There is also a small rise in DA as the rat’s paw approaches the lever (peak before red arrowhead, which indicates final approach).

•  Black arrowhead indicates actual lever depression.

•  Green bar is an audiovisual cue given along with the cocaine.

The audiovisual cue paired with cocaine can also elevate DA.

•  After pairing of cocaine and the cue, the cue alone now induces DA elevation in the absence of a bar press (upper traces).

•  In animals that did not experience pairing, the cue had no effect (lower traces).

•  The cue might be ‘expected’, but it has been paired with a stimulus (cocaine) that is stronger than a natural stimulus, so perhaps this is why it alone can produce DA elevation.

DA may induce drug reward-

seeking behavior •  DA seems to be involved in

inducing behavior leading to reward as well, however, as electrically evoked DA release in the NAc (produced by stimulating VTA neurons) stimulates lever pressing for cocaine.

•  This may mimic the rise in DA that occurs just before a lever press seen in the earlier slide (in b here, gray arrowhead in upper right is electrical stimulation, and the maximal lever pressing rate is at 5-15 sec. after stimulation).

Psychostimulant effects in DA transporter knockouts

•  The dopamine transporter (DAT) is the major target for the locomotor effects of cocaine and amphetamines.

•  The DAT-KO mouse is hyperactive and has a 300X increase in the lifetime of DA at a synapse.

•  Cocaine and amphetamine have no effect on motor activity in this mouse.

Relevance of DA system to drug effects in humans

•  Following infusions of cocaine or amphetamine, the ventral tegmental area showed an immediate but transient activation that correlated with the subjective experience of ‘rush’.

•  Also, the timecourse of DAT blockade by cocaine (as measured by positron emission tomography (PET)) was correlated with the temporal profile of cocaine euphoria.

Opiates •  Opiates are long-lived analogs of endogenous opioid

peptides. •  Most opiate drugs of abuse (heroin, morphine, methadone,

etc.) act on the µ opioid receptor. •  The µ receptor couples to Gi cAMP reduction and

opening of K+ channels, thus usually inhibiting neuronal firing.

•  cAMP changes also lead to effects on CREB and other transcription factors, reprogramming gene expression.

•  Animals will bar-press to deliver opioid peptides into the the ventral tegmental area or the nucleus accumbens.

Opiate inhibition of interneurons causes DA release by VTA neurons

DA and opiate reinforcement •  Opiate self-administration is abolished by deletion of the µ

receptor gene. •  However, lesioning of the DA projections from the ventral

tegmental area to the nucleus accumbens does not eliminate opiate self-administration.

•  This suggests that there are DA-independent opiate targets in the nucleus accumbens or elsewhere that can drive self-administration.

Long-term drug effects, tolerance, and addiction

•  Prolonged drug exposure causes desensitization of responses and adjustment of signaling pathways.

•  This often means that higher levels and/or more frequent administration of drug is required to achieve euphoria.

•  This is called tolerance. •  Tolerance and addiction are related concepts. The desensitization

produced by chronic exposure to drugs modulates brain reward regions, rendering the individual unable to attain sufficient feelings of reward in the absence of drug.

•  In an addicted individual, when drug ingestion is stopped these signaling pathway changes produce anti-euphoria (withdrawal), which can only be counteracted by drugs.

CREB, cAMP, and cocaine tolerance •  Chronic cocaine increases CREB transcription factor

activity in nucleus accumbens neurons, possibly through DA signaling effects on cAMP levels.

•  CREB is switched on by PKA phosphorylation, and PKA is activated by cAMP.

•  Chronic cocaine use elevates cAMP levels, which would activate CREB.

•  Acute cocaine effects, however, are correlated with D2 receptor activation, which decreases cAMP and would thus tend to decrease CREB activity.

•  The elevation of cAMP and CREB activity caused by chronic cocaine use would act in opposition to the acute effects of cocaine and may thus represent a mechanism of tolerance.

Experimental manipulation of CREB activity alters cocaine effects

•  Elevation of CREB activity in the nucleus accumbens by injection of an HSV-CREB viral vector reduces the acute rewarding effects of cocaine and makes low levels aversive.

•  Conversely, dominant-negative CREB increases reward. •  This is consistent with a model in which upregulation of CREB by

long-term cocaine produces a state in which the effects of hedonic DA inputs that act through cAMP downregulation via D2 receptors are reduced.

•  To reduce CREB activity to the level generated by cocaine consumption in a naïve individual (acute cocaine reduces CREB activity by lowering cAMP), an individual exposed to long-term cocaine might have to consume cocaine in larger quantities,since he starts out with a higher level of CREB.

ΔFosB and the cocaine response •  Two IEG proteins, Fos and Jun, heterodimerize to form the

transcription factor AP-1. •  IEGs are rapidly induced in the NAc in response to drug

administration. •  Delta-FosB is a long-lived form of Fos which is induced

only after chronic drug exposure. •  Its overexpression in the NAc acts in an opposite manner

to CREB, in that ΔFosB overexpressing animals prefer drug, while CREB overexpressors find it to be aversive.

•  Delta FosB (because it’s much more stable) gradually accumulates with each drug exposure.

•  This produces long-lasting AP-1 (Fos/Jun) complexes, which alter gene regulation.

Opiate tolerance

•  This also involves cAMP signaling. •  Binding of opiates to µ receptors causes opening of K+

channels (causing hyperpolarization) and a decrease in cAMP levels.

•  Endogenous opioids are short-lived, and the receptor is quickly endocytosed after activation, so that the response returns to normal.

•  Synthetic opiates are long-lived, so to recover to the original state the signaling pathway desensitizes and adjusts, causing elevation of cAMP after chronic opiate exposure.

Effects of chronic opiates on the locus coeruleus.

•  cAMP elevation in response to chronic opiates occurs in the locus coeruleus (LC).

•  The LC is an area involved in attentional states, and is the major noradrenergic nucleus of the brain.

•  Acute opiates decrease firing of LC neurons. •  cAMP elevation in response to chronic opiates causes

opening of a nonselective cation channel, thus increasing the intrinsic firing rate of LC neurons.

Associative opiate tolerance •  Tolerance can be divided into two components: •  In non-associative tolerance the animal becomes tolerant to

opiates in all environments. This can be induced by implantation of morphine pellets.

•  Associative tolerance is a learning process. Here the animal is given drug in one environment and saline in another. It learns to tolerate morphine in the morphine-associated environment, but is not morphine-tolerant in the saline environment.

Associative tolerance in humans •  Associative tolerance is a major factor in human heroin

overdoses. •  If an individual commonly uses heroin in one environment

(i.e. a particular room) and employs specific equipment, he/she becomes tolerant to heroin under these environmental conditions.

•  The same dose of heroin administered in a different room with different equipment can then cause a fatal overdose, because the individual is much less tolerant in the environment that is not normally associated with drug consumption.

Mechanisms of associative tolerance

•  Associative tolerance may be a learning process that occurs in the amygdala and is mediated by the release of the neuropeptide cholecystokinin (CCK) .

•  Repeated morphine exposures increases CCK mRNA expression in the amygdala.

•  There are two CNS CCK receptors, A and B; B appears to be relevant to associative tolerance.

CCK and associative tolerance: experimental evidence

•  Associative tolerance can be assayed by loss of morphine-induced analgesia in the morphine-associated (but not in the saline-associated) environment.

•  Non-associative tolerance was induced by implanting morphine pellets.

•  Infusion of a CCK-B receptor antagonist (L365,260) into the amygdala blocks associative tolerance, but has no effect on non-associative tolerance.

•  Infusion of a CCK-A receptor antagonist (MK-329) has no effect on either type of tolerance.

Reviews on reward, drugs, and addiction •  *Dopamine in Motivational Control: Rewarding,

Aversive, and Alerting. E. S. Bromberg-Martin et al., Neuron 68, 813-834 (2010).

•  *Transcriptional and epigenetic mechanisms of addiction. A. J. Robison and E. J. Nestler, Nature Reviews Neurosci. 12, 623-637 (2011).

•  Personality, addiction, dopamine: insights from Parkinson’s disease. Dagher, A., and Robbins, T.W., Neuron 61, 502-510 (2009).

•  *Required reading

Papers for student presentations •  1. A Role for Repressive Histone Methylation in Cocaine-

Induced Vulnerability to Stress. H. E. Covington, III, et al., Neuron 71, 656-670 (2011).

•  2. Writing Memories with Light-Addressable Reinforcement Circuitry. A. Claridge-Chang et al., Cell 139, 405-415 (2009).

•  3. Excitatory transmission from the amygdala to nucleus accumbens facilitates reward seeking G. D. Stuber et al., Nature 475, 377-380 (2011).