Different populations of subthalamic neurons encode cocaine vs ...

J Neurophysiol 110: 1497–1510, 2013. First published July 17, 2013; doi:10.1152/jn.00160.2013.

Different populations of subthalamic neurons encode cocaine vs. sucrose reward and predict future error Sylvie Lardeux,1 Dany Paleressompoulle,1 Remy Pernaud,1 Martine Cador,2 and Christelle Baunez1,3 1

Laboratoire de Neurobiologie de la Cognition, Unité Mixte de Recherche 6155, Centre National de la Recherche Scientifique & Aix-Marseille Université Marseille, France; 2Institut de Neuroscience Cognitive et Intégrative d’Aquitaine, Unité Mixte de Recherche 5287, Centre National de la Recherche Scientifique & Université Bordeaux I, Bordeaux, France; and 3Institut de Neurosciences de la Timone, Unité Mixte de Recherche 7289, Centre National de la Recherche Scientifique and Aix-Marseille Université, Marseille, France Submitted 5 March 2013; accepted in final form 11 July 2013

Lardeux S, Paleressompoulle D, Pernaud R, Cador M, Baunez C. Different populations of subthalamic neurons encode cocaine vs. sucrose reward and predict future error. J Neurophysiol 110: 1497–1510, 2013. First published July 17, 2013; doi:10.1152/jn.00160.2013.—The search for treatment of cocaine addiction raises the challenge to find a way to diminish motivation for the drug without decreasing it for natural rewards. Subthalamic nucleus (STN) inactivation decreases motivation for cocaine while increasing motivation for food, suggesting that STN can dissociate different rewards. Here, we investigated how rat STN neurons respond to cues predicting cocaine or sucrose and to reward delivery while rats are performing a discriminative stimuli task. We show that different neuronal populations of STN neurons encode cocaine and sucrose. In addition, we show that STN activity at the cue onset predicts future error. When changing the reward predicted unexpectedly, STN neurons show capacities of adaptation, suggesting a role in reward-prediction error. Furthermore, some STN neurons show a response to executive error (i.e., “oops neurons”) that is specific to the missed reward. These results position the STN as a nexus where natural rewards and drugs of abuse are coded differentially and can influence the performance. Therefore, STN can be viewed as a structure where action could be taken for the treatment of cocaine addiction. electrophysiology; basal ganglia; motivation; neurons; incentive cue ONE MAJOR CHALLENGE IN THE field of addiction research is to understand how the brain differentiates different types of rewards, as the goal for the treatment of addiction is to decrease the motivation for drugs without decreasing the motivation for natural rewards. However, since any reward acts on the so-called “reward circuit” involving the mesocorticolimbic dopaminergic pathway, the neurobiological effects arising from natural rewards or drugs of abuse cannot be separated easily. The subthalamic nucleus (STN) is the only structure that appears to have opposite roles in terms of motivation for natural reward and drugs of abuse (Baunez et al. 2005; Rouaud et al. 2010). Indeed, STN lesions or high-frequency stimulation increase motivation for food but decrease motivation for cocaine (Baunez et al. 2002, 2005; Rouaud et al. 2010). In addition, the STN modulates motivation depending on reward preference, since STN lesions increase motivation for alcohol in outbred rats, showing a preference for alcohol, whereas

Address for correspondence and present address: C. Baunez, Institut de Neurosciences de la Timone, UMR 7289, CNRS and Aix-Marseille Université, Campus Santé Timone, 27 Boulevard Jean Moulin, 13385 Marseille Cedex 5, France (e-mail: [email protected]). www.jn.org

decrease it in rats that show a lower alcohol preference (Lardeux and Baunez 2008). Like the striatum, the STN is an input structure of the basal ganglia, receiving direct projections from the cortex and the thalamus (Afsharpour 1985; Sugimoto et al. 1983) but also from the striatum, the dorsal and ventral pallidum, and the midbrain dopaminergic nuclei (Parent and Hazrati 1995). The STN is thus in a key position within the cortico-basal gangliathalamocortical loops to modulate basal ganglia outflow and influence motor actions (Maurice et al. 1998a, b). It also modulates cortical activity directly (Degos et al. 2008), and its involvement in cognitive and motivational functions may be even more prominent than its classical role in mediating basal ganglia outflow (Absher et al. 2000; Baunez and Lardeux 2011; Baunez and Robbins 1997; Baunez et al. 2011; Eagle and Baunez 2010; Eagle et al. 2008; Temel et al. 2006; Trillet et al. 1995; Winstanley et al. 2005; Witjas et al. 2005). We know that STN neurons respond to incentive cues and reward delivery (Darbaky et al. 2005; Matsumura et al. 1992; Teagarden and Rebec 2007), and we have shown that STN neurons differentially encode the salience of natural rewards when two different sucrose concentrations are available (Lardeux et al. 2009). The present study aimed at establishing how STN neurons encode a natural reward and a drug of abuse (given in a quantity that does not model addiction). We recorded STN neurons in rats performing discriminative stimuli tasks (Hauber et al. 2000) using two rewards differing by their nature: cocaine vs. sucrose. Our results show that natural rewards and drugs of abuse are differentially encoded in the STN. Sucrose seems to be preferred by the animals and is also encoded more strongly in the STN. This provides a neurobiological basis for the opposite behavioral effects of STN inactivation on motivation for food vs. cocaine, confirming the STN as an interesting target for the treatment of cocaine addiction (Baunez et al. 2005; Rouaud et al. 2010). MATERIALS AND METHODS

Animals Male Long-Evans rats (n ⫽ 8; Janvier, Saint Berthevin Cedex, France), weighing 350 – 400 g at the time of their surgery, were maintained on a 12-h light-dark cycle at an ambient temperature of 21°C. During the entire experiment, rats were kept at 85% of their

0022-3077/13 Copyright © 2013 the American Physiological Society

1497

1498

STN NEURONS AND COCAINE VS. SUCROSE REWARD

free-feeding weight. Water was provided ad libitum, except during testing sessions. All procedures were conducted in accordance with and approved by the French Agriculture and Forestry Ministry (decree 87-849). Surgery After training in the behavioral task for 3 mo, rats were anesthetized with a mixture of ketamine (100 mg/kg im) and medetomidine (250 ␮g/kg im). Reversal of the anesthesia was obtained with Atipamezole (75 mg/kg im). Bilateral, multiwire tetrodes were positioned

stereotaxically into the STN. Coordinates for the aimed site were (in mm; tooth bar set at ⫺3.3): anteroposterior, ⫺3.7; lateral, ⫾2.4 from bregma; dorsoventral, ⫺8.35 from skull (Paxinos and Watson 2005). The electrode assemblies were anchored to the skull with four stainless-steel screws and dental cement. Given the small size of the STN, a drivable device is not appropriate, as it would allow only a small number of recording sessions. Rats were surgically implanted with a chronic intravenous silicone catheter into the right jugular vein, as described previously (Rouaud et al. 2010). Behavioral testing began 15 days after surgery. Apparatus

A Lever

Training and recording sessions took place in a custom-built Plexiglas operant box (Med Associates, St. Albans, VT). A retractable lever and two cue lights, one on either side of the lever, were located along one wall. A magazine equipped with two cup receptacles was located on the opposite wall. Sucrose solution (0.1 ml) was delivered over 3 s to one of the cups. A 10-ml syringe fixed to a pump (Med Associates) and connected to each cup allowed sucrose delivery. During behavioral sessions, rats’ catheters were connected by tubing through a fluid swivel to a syringe containing a solution of cocaine placed outside of the chamber. Cocaine was delivered over 3 s by the activation of a pump. An interface (MED-PC) and a computer controlled the session and collected data. A one-tone generator (3.5 kHz) provided an auditory stimulus.

Lever release

Light

press

400 ms 100 ms

Reward delivery

500 ms

Tone

ON OFF

Behavioral Procedures Behavioral training before surgery. The aim was to record rats performing a discriminative stimuli task. At the beginning of the session, the house light was turned on and the lever extended. Rats were trained to press the lever for 1 s. During this period, one of the two cue lights was illuminated randomly for 100 ms, 400 ms after the start of the lever press. A trigger tone was delivered 500 ms after the extinction of the light (Fig. 1A), indicating that the rat could release the lever. Each cue light (right or left of the lever) was associated with a specific reward. One-half of the rats was trained as follows: the left cue light indicated that 4% sucrose was the reward, and the right cue light indicated that the reward was 32% sucrose. The other rats were trained with the opposite association. Immediately after the rat’s paw was withdrawn from the lever, it was retracted, and the pump was activated so that both rewards were delivered when the animal was standing in the receptacle. The detection of the animal’s head entry in the magazine after a correct lever release started a 5-s intertrial interval. Anticipatory lever releases (release before the trigger tone, i.e., error trials) were not rewarded and led to the retraction of the

C

B 500 mV 0.1 ms

D Distribution of firing rate

80 60 40 20 0

0-1 1-2 2-3 3-4 4-5 5-6 6-7 78- 8 9-1 9 10- 0 1 11- 1 1 12- 2 1 13- 3 1 14- 4 1 15- 5 1 16- 6 1 17- 7 1 18- 8 1 19- 9 2 20- 0 25 253 >30 0

Number of neurons

100

Frequency (Hz)

150

1.0

##

125 100

0.8

*

0.6

**

0.4 0.2 0

Movement Time (s)

Reaction Time (s)

1.2

S4

S32

75

##

50 25 0

S4

S32

Time in the magazine (s)

E Reaction time F Movement time G Magazine time 6

Sucrose Cocaine

5 4 3 2 1 0

## S4

## S32

Fig. 1. Task, histology, electrophysiological properties, and behavioral results. A: schematic representation of the operant task. This schematic diagram illustrates the time elapsed (400 ms) after the lever press when the light is illuminated (100 ms) and the time elapsed (500 ms) before the lever is released. Reward is only delivered if the rat had held the lever down for the entire 1-s duration. Each cue-light stimulus (at the right or at the left of the lever) predicted a specific reward (either cocaine or sucrose). B: example of the waveform of a representative subthalamic nucleus (STN) neuron. C: frontal section stained with cresyl violet at the level of the STN, showing the track of a representative bundle of tetrodes placed within the STN. Dashed lines outline the STN. D: distribution of firing rates of the 471 STN neurons (mean: 6.19 ⫾ 0.2). E: mean reaction time (RT; in ms; ⫾SE) to release the lever after the tone onset for the 4% sucrose group (S4; left) or the 32% sucrose group (S32; right) for sucrose (white bars) and cocaine (black bars). F: movement time (in ms; ⫾SE) to reach the magazine for the S4 (left) or S32 (right) group for sucrose (white bars) and cocaine (black bars). G: mean time spent into the magazine after a correct trial with delivery of 4% (S4; left) or 32% (S32; right) sucrose (white bars) and cocaine (black bars). *P ⬍ 0.05, **P ⬍ 0.001 compared with the S4 group; ##P ⬍ 0.001 compared with sucrose.

J Neurophysiol • doi:10.1152/jn.00160.2013 • www.jn.org


lever and to the extinction of the house light for 2 s. Each session ended after 120 trials (60 trials with each reward distributed randomly) or if 30 min had elapsed. The time taken to withdraw the paw from the lever after the tone onset, called reaction time (RT), was measured, as well as the time taken to reach the magazine once the lever released, called movement time (MT). Postsurgery behavioral testing. After surgery, rats were retrained for 4 days with the same training procedure. Then, one of the sucrose solutions was replaced by intravenous cocaine self-administration (0.25 mg/90 ␮l per injection), keeping the same rules regarding cue lights. One-half of the rats would therefore receive cocaine and 32% sucrose as rewards (S32 group), and the other half, cocaine and 4% sucrose (S4 group). With regard to the cue light, one-half of the rats in each group had the cocaine cue on the left and the other half, on the right. Cue lights were illuminated in pseudo-random order to limit the number of cocaine injections in a too-short period. The cue light associated with cocaine could not be illuminated for more than three consecutive trials nor if it were already illuminated three times in the last five trials. The number of consecutive illuminations associated with sucrose was limited to five. The cocaine infusion began when the rat’s head was detected in the magazine where the sucrose solution was delivered. Detection of the animal’s head entry in the magazine after a correct lever release started a 20-s intertrial interval. The duration of the daily sessions was then increased so that each session ended after 25 cocaine trials (the number of sucrose trials was unlimited) or when 90 min had elapsed. The animals were subjected to this task for 20 standard recording sessions (except for two rats that lost their electrodes after 10 and 15 sessions, respectively, and one for which the catheter was blocked after seven sessions). Challenges: changes in reward magnitude. For the challenge sessions, the total duration of the session remained 90 min for standard sessions. The rats were thus first subjected to the standard task for 45 min; the session was paused; cocaine was then replaced by 0.9% sodium chloride (i.e., saline; challenge 1), or sucrose (4% in the S4 group and 32% in the S32 group) was replaced by 10% sucrose (challenge 2); and the session started for another 45 min, allowing recording of the same neuron in baseline and challenge conditions. The challenge was performed twice on consecutive days and followed by a session in standard conditions. Electrophysiology Each multiwire electrode bundle comprised two tetrodes made with four formvar-insulated, 25-␮m nichrome electrodes. The tetrodes were threaded through a 26-gauge stainless-steel cannula that served as the ground. Rats were placed in the operant chamber and connected to a multichannel commutator (Crist Instrument, Hagerstown, MD) via a flexible cable. Electrophysiological signals were transmitted to a preamplifier, to an amplifier (Neuralynx, Bozeman, MT), and to data-acquisition hardware (DataWave Technologies, Loveland, CO). All waveforms exceeding an amplitude threshold (2⫻ above noise level) were recorded. Unit discriminations were performed offline using Sciworks clustering software (DataWave Technologies) and Offline Sorter (Plexon, Dallas, TX). In many cases, more than one waveform shape could be isolated on a single tetrode. When these waveforms could not be separated easily, they were discarded from the analysis. Autocorrelograms were then constructed for each unit using NeuroExplorer (Nex Technologies, Madison, AL), and units without well-defined refractory periods were rejected or re-sorted. Although a sample of several hundred units was recorded, it is likely that some signals were recorded more than once over the course of the experiment for different sessions. To avoid analyzing the same neuron twice in consecutive sessions, waveforms of neurons recorded on the same electrode were compared according to the Grossman et al. (2008) analysis. Briefly, multivariate ANOVAs allowed the comparison of the waveform recorded in the same electrode between two

1499

consecutive sessions. If the waveforms had a probability ⬍0.001 to be different, the two neurons were considered different; otherwise, they were considered as the same neurons, and the second recording of this neuron was discarded from the analysis. Furthermore, if two neurons—recorded on the same wire during two consecutive sessions and considered different—responded to the same event in these consecutive sessions, the second neuron was also discarded from the analysis. Data Analysis Behavioral analyses. The behavioral performance with regard to the two different rewards—anticipatory responses and time spent in the magazine [RT (i.e., latency to release the lever after the tone) and MT (i.e., time elapsing between the lever release and the magazine entry)]—was recorded and compared with a two-way ANOVA, with reward (sucrose vs. cocaine) and group (S4 vs. S32) as between factors and session as within factors, using StatView (SAS Institute, Cary, NC). When a significant effect was found, post hoc comparisons (Fisher’s protected least significant difference) were performed using a simple main-effect analysis. The behavioral performances of one group (S4 or S32) in the standard and challenge conditions were compared using paired t-tests. Electrophysiological analyses. Analyses were based on binned peri-event firing rates (50 ms bins), as described previously (Lardeux et al. 2009). As there were many events in a short period of time in this task, it was necessary to compare the neuronal activity after the event with the basal neuronal activity just before to demonstrate the neuronal responses to the specific event. Each response to an event was analyzed for 500 ms after the event and compared with the activity during the 400 ms preceding this event. The baseline lasted 400 ms, because it was the shortest period between two consecutive events. As there was often an anticipatory response before the magazine entry, the basal activity taken was between 600 ms and 200 ms before the event. Since the cocaine infusion lasted 3 s and did not have an immediate effect, long-lasting responses after magazine entry were also analyzed with 500-ms bins for a period of 8 s after magazine entry. Each perievent bin was expressed as a z score based on the mean and SD of the firing rate during the respective basal period of this event. Three or more consecutive bins (⬎150 ms) with z scores ⬎1.64 SDs (95% confidence interval), away from the baseline mean firing rate, were considered a significant response. For the long-lasting responses after magazine entry, one or more 500-ms bins with z scores ⬎1.64 SDs (95% confidence interval), during the 3 s after magazine entry (infusion time), were considered a significant response. The activity of neurons that responded to one event for either reward and the activity of neurons that responded to one event in the standard or challenge conditions were compared with a t-test on the normalized data using StatView (SAS Institute). The responses of neuronal populations to one event were analyzed with a two-way ANOVA, with reward and group as between factor. When a significant effect was found, post hoc comparisons were performed using a simple main-effect analysis. The responses of neuronal population in the two groups (S4 and S32) were compared with ANOVA as well. Results of these two groups were presented only if there were a difference between groups. The proportion of neuronal populations was compared with a ␹2 test. Histology At the end of the experiment, all rats were decapitated, and the brains were removed, frozen, and cut in the coronal section with a cryostat. Frontal, 40 ␮m-thick sections of the STN were stained with cresyl violet for assessment of the electrode placement.


1500


RESULTS

Histology The tip of the electrodes was primarily in the medial part of the STN, as illustrated in Fig. 1C. Out of the 16 sets of tetrodes implanted, four were located outside of the STN area, and data collected from them were therefore discarded. Behavioral Results Rats obtained a mean of 39.3 (⫾3.6) rewards/session, i.e., 14.7 (⫾1.2) cocaine infusions and 24.7 (⫾2.1) sucrose rewards. The percentage of primed error (anticipatory lever release following the cue light and before the tone) was higher for cocaine than for sucrose [37.6% ⫾ 6.0 vs. 25.1% ⫾ 3.7; F(1,6) ⫽ 6.757; P ⬍ 0.05]. As shown in Fig. 1, E–G, rats were faster to perform when highly motivated (i.e., for 32% sucrose; S32 group) than when working for either 4% sucrose or cocaine (S4 group). They exhibited shorter RT [group effect: F(1,6) ⫽ 23.351; P ⬍ 0.001] and a trend toward shorter MT. Whereas RT for cocaine did not differ from that for sucrose, whatever the concentration of sucrose (Fig. 1E), MT was significantly shorter for the sucrose (mean 11.1 s ⫾ 4.2) than for the cocaine [mean 74.7 s ⫾ 23.7; Fig. 1F; reward effect: F(1,6) ⫽ 9.206; P ⬍ 0.001] reward. These results are in line with our previous data showing a lack of difference between the RT for 4% and 32% sucrose but a shorter MT for 32% sucrose when rats work with these two rewards (Lardeux et al. 2009). Taken together, these results indicate that in this particular behavioral task, the MT is affected by the reward predicted, whereas the RT is not. The time spent in the magazine (between the first magazine entry and the following first exit) was significantly shorter for the cocaine (mean 0.703 s ⫾ 0.013) than for the sucrose [mean 3.949 s ⫾ 0.047; Fig. 1G; reward effect: F(1,6) ⫽ 63.415; P ⬍ 0.001] reward. The shorter MT in the sucrose trial, as well as the higher percentage of successfully completed sucrose trials and the lower error rate, seems to indicate that the rats were able to discriminate

the two cue lights and the two different rewards and that the animals prefer sucrose over cocaine. This could also indicate that animals were self-regulating the cocaine infusions by delaying their entrance in the magazine with a longer MT and a higher error rate. Neural Coding in the STN As the aim of the present study was to compare cocaine with sucrose, the electrophysiological data from all animals were pooled (those from S4 and S32 groups) for analysis. Further analyses regarding the influence of the sucrose concentration used will be detailed later. We recorded a total of 471 neurons (S4: n ⫽ 220 and S32: n ⫽ 251; see Fig. 1B), out of which 90.7% (427/471) responded to at least one event of the task (S4: n ⫽ 202 and S32: n ⫽ 225), and 30.4% (143/471) responded to only one event. The mean firing rate for the entire session was 6.19 ⫾ 0.2 Hz (Fig. 1D), in line with what was reported in recent studies carried out in freely moving rats (Lardeux et al. 2009; Shi et al. 2004; Teagarden and Rebec 2007). For each event (i.e., cue light, tone, lever release, and magazine entry), neurons were classified in different populations, according to their response (Table 1). The neurons responding exclusively for one reward at a particular event were classified as “exclusive” neurons (i.e., “exclusive sucrose” or “exclusive cocaine”). The neurons exhibiting a response for both rewards but with a response of higher magnitude for one reward (t-test, P ⬍ 0.05) were classified as “selective” neurons. The neurons responding for both rewards in the same manner (t-test, P ⬎ 0.05) were classified as “similar” neurons. As both activation and inhibition were recorded (Table 2), we also compared these populations and analyzed them separately; these results are reported only if they differ from the general results and/or if they differ from each other. When one neuron was responsive for more than one event, we also looked at whether it kept the same specificity for one

Table 1. Percentage of the different populations responsive to each event Both Groups Event Correct trials: Cue light Tone Lever release Magazine entry, short responses Magazine entry, long responses Incorrect trials: Cue light Lever release

Sucrose 4% Group

Sucrose 32% Group

Cocaine

Sucrose

Selective

Similar

Cocaine

Sucrose

Selective

Similar

Cocaine

Sucrose

Selective

Similar

45.3 34/75 26 38/146 32.3 32/99 12.7 33/259 23.6 69/293

44.0 33/75 40.4 59/146 48.5 48/99 59.5* 154/259 57.7* 169/293

5.3 4/75 11.6 17/146 8.1 8/99 15.1 39/259 12.6 37/293

5.3 4/75 21.9 32/146 11.1 11/99 12.7 33/259 6.1 18/293

45.7 21/46 19.2 15/78 31.6 18/57 21.8 22/101 26.6 33/123

43.5 20/46 28.5 30/78 45.6 26/57 64.4 65/101 57.7 71/123

6.5 3/46 16.7 13/78 8.8 5/57 9.9 10/101 10.6 13/123

4.3 2/46 25.6 20/78 14 8/57 4 4/101 4.9 6/123

44.8 13/29 33.8 23/68 33.4 14/42 7 11/158 21.2 36/170

44.8 13/29 42.7 29/68 52.4 22/42 56.3 89/158 57.6 98/170

3.5 1/29 5.9 4/68 7.1 3/42 18.4 29/158 14.1 24/170

6.9 2/29 17.7 12/68 7.1 3/42 18.4 29/158 7.1 12/170

48.8 35/61 36.2 21/58

44.2 23/6 1 60.3 35/58

2.3 1/61 1.7 1/58

4.7 2/61 1.7 1/58

38.5 22/41 32.1 9/28

53.9 17/41 64.3 18/28

3.8 1/41 3.6 1/28

3.8 1/41 0 0/28

64.7 13/20 40 12/30

29.4 6/20 56.7 17/30

0 0/20 0 0/30

5.9 1/20 3.3 1/30

The neurons in the sucrose and cocaine columns are “exclusive” neurons, responsive only when sucrose or cocaine is, respectively, predicted. The “selective” neurons are reactive to both rewards but with a higher magnitude for 1 reward. The “similar” neurons exhibited no differential response as a function of the reward. The incorrect trials correspond to the trials during which the rat had released the lever before the trigger tone and after the cue light. For each event, the 1st line gives the percentage of neurons of each category. The 2nd line gives the number of neurons/total number of responsive neurons. *P ⬍ 0.001 compared with the proportion of the exclusive cocaine neuron. J Neurophysiol • doi:10.1152/jn.00160.2013 • www.jn.org


Table 2. Percentage of activation and inhibition for each event Event

Total

Activation

A

B

17.6 (75/427) 34.2 (146/427) 23.2 (99/427)

29.3 (22/75) 93.2 (136/146) 67.7 (67/99)

70.7 (43/75) 6.8 (10/146) 32.3 (32/99)

60.7 (259/427)

95.0 (246/259)

5.0 (13/259)

68.6 (293/427)

85.7 (251/293)

14.3 (42/293)

10.1 (43/427) 13.6 (58/527)

72.1 (31/43) 72.4 (42/58)

27.9 (12/43) 27.6 (16/58)

The incorrect trials correspond to the trials during which the rat had released the lever before the trigger tone and after the cue light. In parentheses: number of neurons/total number of responsive neurons.

given reward. Overall, the neurons that responded to a sucrose event kept responding to sucrose or responded to both rewards in later events (sucrose: 57.3%, both: 26.2%, cocaine: 16.5%). On the contrary, neurons that responded to a cocaine event responded mostly to subsequent events for sucrose or both (sucrose: 50.7%, both: 34.3%, cocaine: 14.9%). Furthermore, neurons that responded to both rewards in an event responded mostly to both or to sucrose only in subsequent events (sucrose: 34.6%, both: 50%, cocaine: 15.4%). This response pattern was more pronounced in the S32 group than in the S4 group. This suggests that STN neurons encode the information regarding the predicted outcome at cue light and then, mostly respond for the preferred reward for subsequent events. STN Neurons Differentially Encode Cocaine and Sucrose Population coding. Different neuronal populations responded during the cocaine trials or during the sucrose trials. Analyses of the neuronal responses to the cue light were first performed, as it was the most-relevant event to assess reward expectation. The neuronal populations responding to cues predicting sucrose (n ⫽ 41 neurons) and cocaine (n ⫽ 42 neurons) were segregated almost totally and of similar size (Fig. 2A; see also Table 1). As illustrated in Fig. 2, B and C, most neurons responded exclusively to one of the cue lights (44% and 45.3% for sucrose or cocaine, respectively). The neuronal response after the trigger tone and the lever release may be modulated by the predicted reward. This

A

B 5.3

Exclusive-sucrose Exclusive-cocaine 11.1 Selective 8.1 Similar

Inhibition 21.9

Correct trials: Cue light Tone Lever release Magazine entry, short responses Magazine entry, long responses Incorrect trials: Cue light Lever release

Cocaine

Sucrose

Cue Light

Cue Light

C

26

48.5

40.4

Fig. 3. Neuronal population repartition at the tone and lever release. Composition in percentage of the neuronal population responding after the tone (A) and the lever release (B).

modulation is thought to reflect a difference in the expectation of each reward, as shown previously in the striatum and the orbitofrontal cortex (OFC) (Cromwell and Schultz 2003; Hassani et al. 2001; Pasquereau et al. 2007; Schoenbaum et al. 1998; van Duuren et al. 2007). The neuronal populations responding to the events following the cue light (i.e., tone, lever release, and magazine entry) were segregated mostly between exclusive cocaine and exclusive sucrose neurons. Interestingly, for these events, the exclusive sucrose neuronal population was larger than the exclusive cocaine population (Fig. 3, A and B, and 4, A and B, 40.4% vs. 26% at tone, 48.5% vs. 32.3% at lever release, and 59.5% vs. 12.4% at magazine entry; ␹2: P ⬍ 0.05; see also Table 1). Whereas a majority of neurons was inhibited after the cue light (70.7%), a majority of activation was recorded after the tone, the lever release, or the magazine entry (93.2% at tone, 67.7% at lever release, and 95% at magazine entry; Table 2). Furthermore, more neurons were inhibited after the cue predicting sucrose than after the cue predicting cocaine (80% vs. 61.9%; ␹2: P ⬍ 0.001). The cue light was illuminated while the rat was holding its paw on the lever so that the motor activity of the animals was controlled and already integrated in the neuronal response at the cue-light onset. Furthermore, analysis with regard to the position of the cue light indicated that there were no systematic specific responses toward the contralateral side, ruling out a total sensory or spatial response of STN neurons to the light. The neuronal responses measured are thus more likely related to the prediction of the reward. It was also necessary to avoid different possible movement directions for the two available rewards, as movement direction can modulate neuronal response (shown, for example, in the OFC) (Feierstein et al.

Cocaine

Sucrose

Cue Light

Cue Light

45.4

Exclusive-sucrose Exclusive-cocaine Selective Similar

Spikes/s

44 10

8

8

6

6

4

4

2

2

0

0 -1 -0.5 LP

0

0.5 T

1

15

15

10

10

5

5

0 -1 -0.5 0 LP

0.5

1

-1 -0.5 0

T

LP

32.3

11.6

5.3

10

1501

0.5 T

1

0

-1 -0.5 0 LP

0.5 T

Time (s)


1

Fig. 2. STN response to predictive cue lights in correct trials. A: distribution in percentage of the neuronal population responding to the cue light. B and C: example of the firing pattern of 2 STN neurons [1, “exclusive cocaine” (B), and 1, “exclusive sucrose” (C)] to cue light predicting cocaine (left) and to cue light predicting sucrose (right). Rasters are centered on the occurrence of the cue light [time (t) ⫽ 0] that lasted 100 ms. Top: raster plot of spike firing on each trial, with the top row of dots corresponding to the 1st trial. Bottom: mean firing rate across all trials; bin size is 50 ms. The arrows indicate the time of the lever press (LP) and the tone (T).

1502


A

B

59.5

Spikes/s

C

57.7

Cocaine

Sucrose

Magazine entry

Magazine entry

40

40

30

30

20

20

10

10

D

-1

-0.5

0

0.5

1

Sucrose Magazine entry

20

20

15

15

10

10

5

5

-1

-0.5

0

0.5

1

Time (s)

2006). The required movement following lever release was therefore the same regardless of the reward; magazine entry detection was required for delivery of both sucrose and cocaine. Very few neurons responded after both the tone and lever release, ruling out that the increase in firing rate observed after the tone is the same as that at lever release. Nevertheless, it is not possible to totally rule out the fact that these responses could be related to the motor preparation of the lever release. Rate coding. When comparing sucrose-encoding neurons on sucrose trial with cocaine-encoding neurons on cocaine trials, there was no significant difference between the firing rates of the neuronal population at cue-light [F(1,80) ⫽ 0.61; P ⫽ 0.44], tone [F(1,193) ⫽ 0.16; P ⫽ 0.69], or lever-release [F(1,116) ⫽ 0.41; P ⫽ 0.52] events. After magazine entry, short and long responses were analyzed (Figs. 4 and 5). For both short and long responses, the firing rate of the sucrose-responsive neuronal population was higher than that of the cocaine-responsive population [Fig. 5, A and B; reward effect: short responses, F(1,328) ⫽ 20.415; long responses, F(1,346) ⫽ 20.336; P ⬍ 0.01]. During the 500-ms analysis phase of the short responses, the cocaine infusion just began. There is thus no effect of the cocaine itself, but since the behavioral results showed that the animals were in the magazine during this period—whichever reward was predicted (Fig. 1G)—the neuronal responses recorded cannot be related to major movements in the box. The phasic responses recorded during the first 500 ms spent in the magazine (59.5% neurons exclusive sucrose, 12.7% neurons exclusive cocaine; Fig. 4A) were thus more likely related to expectation and confirmation, as the reward delivered matched what was predicted. Furthermore, even if the rewards’ route of administration was different between sucrose and cocaine, the neuronal response latencies were the same for both rewards (Fig. 5G). To measure the

Cocaine

Magazine entry

0

0

0

0

23.6

12.6


15.1

Fig. 4. STN response to reward access. A and B: distribution in percentage of the neuronal population responding after magazine entry for short (A) and long (B) responses. C and D: examples of STN firing pattern to cocaine (left) and sucrose (right) delivery for “exclusive” neurons. C: example of a short response for an exclusive cocaine neuron. D: example of a long response for an exclusive sucrose neuron. Rasters are centered on the magazine entry (t ⫽ 0). Top: raster plot of spike firing on each trial with the top row of dots corresponding to the 1st trial. Bottom: mean firing rate across all trials; bin size is 50 ms.

6.1

12.7

12.7

0

2

4

6

8

10

0

2

4

6

8

10

Time (s)

neuronal response induced by cocaine after its delivery in the bloodstream, the long-lasting responses after magazine entry were analyzed. Although cocaine is now perceived during this long interval, the firing rate for sucrose remains higher than that for cocaine throughout the long response (Fig. 5B). These differences may reflect the different behaviors related to each reward, in terms of licking or staying still in the magazine. The lower firing rate for cocaine may be explained partially by the lower percentage of neurons that is activated than for sucrose (68.66% vs. 90.63%; ␹2: P ⬍ 0.001). However, when analyzed separately, the firing rate of the population activated after magazine entry is also higher for sucrose than for cocaine and thus for both short and long responses (Fig. 5, C and D), whereas there is no differences between the firing rate of the neuronal population inhibited after magazine entry [Fig. 5, E and F; activation, short responses, reward effect: F(1,104) ⫽ 5.806; long responses, reward ⫻ time interactions: F(15,1,410) ⫽ 4.268; P ⬍ 0.01]. Interestingly, the firing rate of the neuronal population activated during the long response is higher for sucrose during the first 3 s, and then it is higher for cocaine. This effect may reflect the reward perception, whereas the inhibitions may reflect a more general arousal effect. Modulation of Neuronal Encoding by Reward Value Influence of the sucrose concentration on board. Since rats performed the task with either 32% sucrose (S32 group) or 4% sucrose (S4 group) vs. cocaine, we then investigated whether the magnitude of the sucrose solution had influenced the STN neuronal responses. It is first interesting to note that more neurons responded to magazine entry in the S32 group (n ⫽ 158/225) than in the S4 group (n ⫽ 101/202; ␹2: P ⬍ 0.001) but that more exclusive neurons were recorded in the S4 group than in the S32 group (␹2: P ⬍ 0.001; Table 1). Furthermore,



A

Magazine entry

4

**

Sucrose Cocaine

4

z-score

B

All neurons 6

Magazine entry

**

3 2

2

1 0

0 -1

-2

C

D

Activation

4

6

** 3

2

2

0

1

-2

0

z-score

4

E

**

Fig. 5. Neuronal population response to reward access. Mean firing rate of all neurons (A and B) and neurons showing an activation (C and D) and an inhibition (E and F) in response to sucrose (gray lines) and cocaine (black lines) for short (A, C, and E) and long (B, D, and F) responses after magazine entry. G: neuronal response latencies at magazine entry: cumulative sum of the percentage of neurons responding after magazine entry for each 50-ms bin for sucrose (suc; gray line) and cocaine (coc; black line). **P ⬍ 0.001 compared with the mean firing rate of cocaine-responsive neurons.

F

Inhibition

1

2 1

z-score

1503

0

0

-1 -1

-2

-2 -3

-3

-600

-400

-200

0

200

400

0

1

2

Time (ms)

G

4

5

6

7

8

Neuronal response latencies 100

Percentage of neurons

3

Time (s)

Magazine entry suc

80

coc

60 40 20 0

-200 -100

0

100

200

300

400

Times (ms)

the firing rate during short and long responses after magazine entry was higher in the S32 group than in the S4 group, regardless of the reward on board [sucrose or cocaine; Fig. 6, A and B; group effect: short response, cocaine F(1,102) ⫽ 18.931, sucrose F(1,224) ⫽ 5.316; long response, cocaine F(1,122) ⫽ 12.619, sucrose F(1,222) ⫽ 5.666; P ⬍ 0.001]. Since we have shown previously that when rats work for either sucrose 4% or 32%, there is a higher firing rate for sucrose 4%, whereas there is less licking activity (Lardeux et al. 2009), these differences in firing rate here between S4 and S32 groups are unlikely to be related to a licking difference. In contrast, the firing rate following the trigger tone was higher in the S4 group than in the S32 group

[Fig. 6C; group effect: cocaine, F(1,85) ⫽ 19.942; sucrose: F(1,106) ⫽ 10.968; P ⬍ 0.001], regardless of the expected reward. The differences between the firing rates after the trigger tone in the two groups could be related to the differences observed for RT measures, revealing shorter RTs in the S32 group. This would suggest that increased STN neuronal activity results in a reduced ability to release the lever. This is in line with the classical view of the functioning of the basal ganglia motor loop. Influence of a change in reward value/reward-prediction error. To assess whether STN neurons could encode an unexpected change in reward value, cocaine was first replaced by


1504


A

Cocaine 6

4

4

**

3

3

2

2

1

1 0

-1 -600

z-score

Fig. 6. Neuronal responses in the S4 and S32 groups. Mean firing rate of the neuronal responses in the S4 (gray lines) and S32 (black lines) groups after magazine entry [short (A) and long (B) responses] and after the tone (C) for cocaine (left) and sucrose (right). **P ⬍ 0.001 compared with the mean firing rate of the S4 group.

-400

-200

0

200

-1 -600

400

z-score

-400

-200

0

400

200

Time (ms) Magazine entry

Magazine entry 4

4

2

**

2

**

0

0

-2

C

**

5

0

B

Magazine entry

6

S32 S4

5

z-score

Sucrose

Magazine entry

-2 0

1

2

3

4

5

6

7

8

0

1

2

3

4

5

6

7

8

Time (s) Tone

8 6

6

4

4

**

2 0

-2 -400

Tone

8

**

2 0

-200

0

200

400

-2 -400

-200

0

200

400

Time (ms)

saline (challenge 1). Then, 4% or 32% sucrose was replaced by a 10% sucrose solution (challenge 2). The neuronal responses of the same neurons under standard and challenge conditions could then be compared, since the change was operated during the course of a regular session. Challenge 1: saline replaces cocaine. BEHAVIORAL EFFECT. The number of rewards obtained was significantly higher under challenge conditions [44.2 vs. 22.9; challenge effect: F(1,4) ⫽ 29.483; P ⬍ 0.05], showing that the rats noticed the change and tried to compensate for the lack of cocaine by working harder, initiating more trials. In the S32 group, the RT was increased during saline trials when compared with cocaine trials [Fig. 7A; 0.661 s vs. 0.47 s; challenge effect: F(1,3) ⫽ 10.288; P ⬍ 0.05], suggesting that the lower reinforcing properties of saline were noticed and decreased the motivation to release the lever quickly. This also confirms further that cocaine acts as a reinforcer in the task under standard conditions. In contrast, RT for sucrose remained unaffected by this challenge. In the S4 group, no significant change was observed. Since rats were trained previously under the same procedure with 4% vs. 32% sucrose, it is possible that the reward was devalued when the rats in the S4 group ended up with cocaine instead of 32% sucrose after surgery. If we consider thus that the standard condition is a devaluation situation for animals of

the S4 group, the lack of effect induced by replacing cocaine by saline may be due to the low level of motivation of the animals. ELECTROPHYSIOLOGICAL EFFECT. During this challenge, 53 out of the 65 neurons recorded responded to at least one event under baseline conditions (S4: n ⫽ 27 and S32: n ⫽ 26), and 58 responded under challenge conditions (S4: n ⫽ 31 and S32: n ⫽ 27). Most of the recorded neurons responded to a given event only in baseline or in the challenge condition (Fig. 7C and Table 3). Only few neurons adapted their response to the same event under both conditions (cocaine or saline; ␹2: P ⬍ 0.001) by changing their selectivity, demonstrating their ability to adapt to the new condition or to code reward-prediction error. The number of responsive neurons after magazine entry allowed further analysis. As shown in Fig. 7B, in the S4 group, fewer neurons responded exclusively to sucrose in the challenge condition compared with the standard condition after magazine entry (␹2: P ⬍ 0.05). This suggests that when the difference between the rewards decreases (from 4% sucrose vs. cocaine to 4% sucrose vs. saline), there is a lower encoding of the relative reward values. In contrast, in the S32 group, more neurons responded selectively at magazine entry in the challenge condition than in the standard condition (␹2: P ⬍ 0.001).



Challenge 1: cocaine

A

S4 group 1.5

1

1

0.5

0.5


0

Sucrose Cocaine/saline

saline Cocaine

Saline

Cue Light

Cue Light

§§

0 Standard Challenge

Standard Challenge

100

100

80

80

60

60

40

40

20

20

Similar Selective Exclusive Cocaine Exclusive Sucrose

20

20

15

15

10

10

5

5

Spikes/s

Reaction Time (s)

1.5

B

C

S32 group

0

0


0 -1

Standard Challenge

-0.5

0

LP

Challenge 2: sucrose


S4 group 2

2

1.5

1.5

1

1

0.5 0

§§ §

Standard Challenge

100

100

80

80

60

60

40

40

20

20

0

Standard Challenge

1

-1

T

-0.5

0

LP

0.5

1

T

Sucrose 4%

Sucrose 10%

Magazine entry

Magazine entry

Sucrose/10% sucrose Cocaine

0.5 0

0.5

10%sucrose

F

S32 group

** *

Standard Challenge

Similar Selective Exclusive Cocaine Exclusive Sucrose

Spikes/s

Reaction Time (s)

D

E

1505

40

40

30

30

20

20

10

10

0


Fig. 7. Challenge sessions: behavioral results and neuronal responses. A and D: RT during correct trials for 4% (S4 group, left) or 32% (S32 group, right) sucrose (white bars) and cocaine (black bars) when saline replaced cocaine (A: challenge 1) and when 10% sucrose replaced 32% or 4% sucrose (D: challenge 2). B and E: effect of challenge (B: challenge 1; E: challenge 2) on the proportion of exclusive, “selective,” and “similar” neurons at magazine entry in the S4 and S32 groups (left and right, respectively). C: example of an exclusive neuron responsive to cocaine in the standard condition at cue-light presentation (left) and no longer responsive when cocaine was replaced by saline (right). The arrows indicate the time of the lever press and the tone. F: example of an exclusive neuron that did not respond to 4% sucrose in the standard condition at magazine entry (left) and that showed an increased activity when 4% sucrose was substituted by 10% sucrose (right). *P ⬍ 0.05, **P ⬍ 0.001 compared with the S4 group; §P ⬍ 0.05, §§P ⬍ 0.001 compared with the “standard” condition.

0 -1

-0.5

0

0.5

1

-1

-0.5

0

0.5

1

Time (s)

This suggests that when the difference between the rewards increases (from 32% sucrose vs. cocaine to 32% sucrose vs. saline), there is a higher selectivity in encoding the relative value of the rewards on board. Furthermore, the firing rate tends to decrease during the challenge in the S32 group for saline compared with what it was for cocaine [challenge effect: F(1,182) ⫽ 3.829; P ⫽ 0.07], whereas it did not change in the S4 group. On the contrary, the firing rate was higher for sucrose during the challenge than during the baseline in the S4 group challenge effect [F(1,299) ⫽ 6.041; P ⬍ 0.05] but did not change in the S32 group. Challenge 2: 10% sucrose replaces 4% or 32% sucrose. BEHAVIORAL EFFECT. As shown in Fig. 7D, replacing 4% by 10% sucrose decreased the RT in the S4 group for both cocaine and sucrose rewards (t-test; cocaine: 1.05 s vs. 0.35 s, P ⬍ 0.05; sucrose: 0.965 s vs. 0.405 s, P ⬍ 0.001), showing that increasing the sucrose concentration enhances motivation to perform the task in general. In contrast, replacing 32% by 10% sucrose had no behavioral consequence within the group. However, it abolished the RT difference between the two groups measured under baseline

conditions [baseline: group effect, sucrose, F(1,3) ⫽ 42.278, P ⬍ 0.001; after a group ⫻ sessions interaction, cocaine, F(1,3) ⫽ 10.707, P ⬍ 0.05; challenge 2: group effect, sucrose, F(1,3) ⫽ 0.290, cocaine: F(1,3) ⫽ 0.379, P ⬎ 0.05]. ELECTROPHYSIOLOGICAL EFFECT. During challenge 2, 53 neurons were recorded; 47 responded to at least one event under baseline conditions (S4: n ⫽ 24 and S32: n ⫽ 23), and 43 responded under challenge 2 conditions (S4: n ⫽ 21 and S32: n ⫽ 22). As for challenge 1, most of the recorded neurons responded to a given event only in baseline or in the challenge condition (␹2: P ⬍ 0.001; Fig. 7F and Table 3), a form of reward-prediction error. Interestingly, after magazine entry in the S4 group, more exclusive sucrose neurons were found than exclusive cocaine under the challenge condition compared with baseline conditions (␹2: P ⬍ 0.001). This may reflect the rats’ higher motivation to work for 10% sucrose than for 4% sucrose. No changes were noticed in the S32 group in line with the absence of behavioral effects (Fig. 7E). The firing rate was not different during the challenge compared with the baseline. These challenges show that the selectivity for one reward is influenced by the second reward available. During the chal-


1506


Table 3. Number of neurons responsive during the challenge Challenge 1: Saline Replaces Cocaine

Challenge 2: 10% Sucrose Replaces 32% or 4% Sucrose

Both

Cocaine Cue light Cue light, incorrect Trigger tone Lever release ALR Magazine Total cocaine Sucrose Cue light Cue light, incorrect Trigger tone Lever release ALR Magazine Total sucrose Total

Both

Baseline

Challenge

Stable

Adaptative

Baseline

Challenge

Stable

Adaptative

3 1 1 4 2 3 14

1 2 3 2 1 7 16

0 0 2 2 0 5 9

0 0 2 2 0 0 4

4 1 3 3 1 5 17

4 5 6 0 3 6 24

0 1 2 1 0 1 5

0 0 0 0 0 0 0

2 4 4 0 3 2 15 29

3 4 4 4 5 11 31 47

0 0 4 2 3 17 26 35

0 0 1 1 0 7 9 13

3 3 8 3 1 13 31 48

4 5 2 11 1 9 32 56

1 0 1 0 0 7 9 14

0 0 1 1 0 4 6 6

For each event, the number of neurons reactive only in the baseline condition, only in challenge condition, and in both conditions is indicated for cocaine (top) and sucrose (bottom). The neurons responsive in both conditions could exhibit the same response in standard and challenge conditions (“stable” neurons), or they could exhibit different responses in the 2 conditions (“adaptive” neurons). Cue light, incorrect, response after the cue light when the rat released the lever before the trigger tone in error trials; ALR, anticipatory lever release; when the rat released the lever before the trigger tone and after the cue light in error trials.

lenges, animals receive a different reward than the one predicted; it is thus possible that the differences observed between the baseline and the challenges conditions are a signal of reward-prediction error (Schultz and Dickinson 2000). STN Neurons Respond Specifically to Executive Error, and Activity at Cue Onset Can Predict Future Error As mentioned earlier, STN has been shown to be involved in encoding error (Brown et al. 2006; Lardeux et al. 2009). Therefore, the neuronal responses to the cue and the lever release during error trials were investigated, especially when the premature lever release occurred after the cue light, after the sucrose or cocaine reward had been signaled. As for correct trials, most neurons responded exclusively to one cue light either for cocaine trials (48.8%) or sucrose trials (44.2%; Fig. 8A and Table 2). Interestingly, the firing rate of the neurons responding to the cue light was higher during incorrect trials than during correct trials [trials effect: cocaine, F(1,66) ⫽ 22.209; sucrose, F(1,60) ⫽ 7.359; P ⬍ 0.001], suggesting that the firing rate at the cue-light onset can predict future errors (Fig. 8, B and C). Indeed, most neurons were activated during error trials, whereas in contrast, most neurons were inhibited at cue light in correct trials (␹2: P ⬍ 0.001). These differences might reflect that the firing rate promotes the anticipated lever release in the error trials. Another hypothesis is that the animals were not expecting the particular cue in these trials and that these firing rates reflect the unexpected occurrence of the cue. Furthermore, during these error trials, the firing rate after the cue light is higher in the cocaine trials compared with the sucrose trials [Fig. 8, B and C; reward effect: F(1,46) ⫽ 4.261; P ⬍ 0.01]. This effect is related to a higher percentage of neurons activated in the cocaine trials than in the sucrose trials (84.62% vs. 63.64%; ␹2: P ⬍ 0.001), which was also the case for correct trials. When looking at neuronal responses at anticipated lever release, we have identified “oops” neurons (Lardeux et al.

2009), responding specifically at lever release during error trials (58/427) but not during correct trials (Fig. 9, B and C, for an example of an “oops cocaine neuron”). It is possible that these neurons respond only in incorrect trials, because the lever-release movement is then different than the movement in correct trials. However, this can be ruled out, since these “oops neurons” respond exclusively to one missed reward, with a majority of them responding to sucrose (Fig. 9A; sucrose: 60.3%, cocaine: 36.2%; ␹2: P ⬍ 0.001), suggesting that the information related to the expected reward plays a role at that stage. DISCUSSION

We report here that most STN neurons respond to cocaine or sucrose, indicating that the STN encodes reward value via different subpopulations. This result is in line with our previous study showing that different populations encoded 4% and 32% sucrose. However, here, we also observed a preferential encoding that we did not observe when both rewards were of the same nature. The shorter MT in sucrose trials and the higher percentage of completed sucrose trials suggest that cocaine is a lower reinforcer than sucrose, in line with the study by Lenoir et al. (2007), showing that rats prefer sucrose over cocaine, especially, maybe, when cocaine is only given in a limited number of injections/session that do not model addiction. However, the fact that RT was not significantly different for sucrose and cocaine suggests that cocaine is still rewarding enough for the animals to release the lever as quickly as possible. The S32 group exhibited shorter RTs than the S4 group, suggesting that cocaine may not be reinforcing enough to compensate for low motivation driven by 4% sucrose. Since rats were trained previously under the same procedure with 4% vs. 32% sucrose, it is also possible that reward was devalued when the rats in the S4 group ended up with cocaine instead of 32% sucrose. In



A

4.7

2.3

48.8

44.2

B

Cocaine

Cue Light

4

**

z-score

0

-2 -400 -300 -200 -100 LP

Sucrose

0

100 200 300 400 500

Cue Light

4

**

2

z-score

##

Correct Incorrect

2

C


0

-2 -400 -300 -200 -100 LP

0

100

200 300 400 500

Time (ms)

Fig. 8. STN response in anticipation of executive error. A: composition in percentage of the neuronal population responding at the cue light in error/ incorrect trials. B and C: mean firing rate of the neuronal responses after the cue light in correct (black lines) and incorrect (gray lines) trials for cocaine (B) and sucrose (C). The arrows indicate the time of the lever press. **P ⬍ 0.001 compared with the mean firing rate during correct trials; ##P ⬍ 0.001 compared with the mean firing rate during incorrect sucrose trials.

contrast, for the rats of the S32 group, cocaine replaced 4% sucrose, and it was less devaluating. Substituting 4% by 10% sucrose during the challenges increased the motivation of the S4 group to work, whereas replacing cocaine by saline had no effect, suggesting that the cocaine-reinforcing effect is weak and does not provide a big contrast with a low concentration of sucrose. This supports the idea that cocaine has a low rewarding efficacy when it is contrasted with sucrose. This is the first study to record the responses of STN neurons to cues announcing two rewards of different types: one, natural (sucrose) and one, drug of abuse (cocaine). We show that segregated neuronal populations responded for each reward, which confirms our previous results (Lardeux et al. 2009). Interestingly, the population responding for sucrose was larger than that responding for cocaine and had a higher firing rate. As discussed above, if sucrose is the preferred reward, then STN seems to encode more strongly the preferred reward. We

1507

showed previously that separate populations encode two natural rewards differing in their salience (32% and 4% sucrose), but we did not observe any preferential encoding (Lardeux et al. 2009). Here, we show a larger population encoding sucrose with a higher firing rate, suggesting a stronger encoding of the preferred reward when rewards differ by their nature. However, it is possible that intravenous cocaine infusion may not be associated properly with the cue light, as the same cue was formerly associated with a sucrose solution. If we consider then that cocaine trials are extinction trials, the results observed here suggest that STN neurons can encode the reward value, and in this case, the value of cocaine is very low. The classical hypothesis of STN function suggests that activation of the output nuclei of the basal ganglia by the STN suppresses undesired movements (Isoda and Hikosaka 2008; Mink 1996), which would have predicted an activation of STN neurons in our task during maintenance of the lever press. Since we recorded mainly activation at lever press (data not shown) but inhibition after the cue light, the neuronal activity observed is more likely due to reward-related processes than to motorrelated processes. It is thus possible that the reward-related processes override the motor processes or are more prevalent. The differences observed between S4 and S32 groups suggest that the neuronal response to a given reward in the STN depends on the alternative reward available. Indeed, in the S32 group, more neurons responded at reward access and with a higher firing rate than in the S4 group, suggesting a stronger reward encoding when rats worked for two supposedly highvalue rewards. Another possibility is that the higher motivational/arousal state that the animals are on while working for 32% sucrose induces this higher neuronal-responding firing rate within the STN. However, the higher number of exclusive neurons in the S4 group at reward access suggests that the neuronal specialization is higher when the contrast between the two available rewards is supposedly high. The effect of the alternative reward on neuronal specialization was also confirmed when saline replaced cocaine, as the S4 group—then having access to either 4% sucrose or saline—showed less specialization than previously with cocaine, thus confirming our previous results (Lardeux et al. 2009). The adaptability of the neuronal responses during the challenges confirms that reward encoding within the STN neurons depends on the rewards available. The modifications of the neuronal signal in challenge conditions may also be a reward-prediction error, as the animal had a reward that was not the one predicted (Schultz and Dickinson 2000). The firing rate at cue onset is predictive of future error. Indeed, increased activity at cue onset preceded an error, whereas we observed decreased activity for correct trials. This indicates that STN neurons contribute to the encoding of motor performance and error, as shown previously in the human STN (Brown et al. 2006). These responses might be related to motor-control processes (impulse control) known to be under the influence of STN (Aron and Poldrack 2006; Baunez et al. 1995; Eagle and Baunez 2010; Eagle et al. 2008) or to a difference in encoding with respect to the trial contingency; however, future studies are needed to address this issue directly. These responses, predictive of an error, were also specific to the predicted reward, as separate populations responded for sucrose and cocaine.


1508


A

B

Incorrect trials Cocaine

Sucrose

Lever release

Lever release

Correct trials

C

Cocaine

Sucrose

Lever release

Lever release

1.7 1.7

36.2


Spikes/s

60.3 30

30

30

30

25

25

25

25

20

20

20

20

15

15

15

15

10

10

10

10

5

5

5

5

0

0

0

-1

-0.5

0

0.5

1

-1

-0.5

0

0.5

1

0 -1

-0.5

0

0.5

1

-1

-0.5

0

0.5

1

Time (s)

Fig. 9. STN response during error. A: composition in percentage of the neuronal population responding at the lever release in error trials. B and C: example of an “oops” cocaine STN neuron. Firing pattern to cocaine (left) and sucrose (right) delivery for the same neuron at lever release, following the cue light, is represented in incorrect (B) and correct (C) trials. Rasters are centered on the occurrence of the lever release (t ⫽ 0). Top: raster plot of spike firing on each trial with the top row of dots corresponding to the 1st trial. Bottom: mean firing rate across all trials; bin size is 50 ms.

Some studies in the prefrontal cortex (PFC), in the globus pallidus, and in the nucleus accumbens (NAc) have reported the existence of neurons responding specifically during behavioral errors (Amiez et al. 2005; Arkadir et al. 2004; Taha et al. 2007; Watanabe 1989). Interestingly, the error activity in the anterior cingulate cortex of the monkey depends on reward prediction (Amiez et al. 2005) and may thus—as well as the midbrain dopaminergic system (Schultz and Dickinson 2000)— drive the neuronal responses observed in the STN during error trials via the hyperdirect pathway, linking the PFC to the STN. Many studies have shown neuronal modulation for different natural rewards in the PFC, NAc, striatum, pallidum, and dopaminergic neurons (Arkadir et al. 2004; Cromwell and Schultz 2003; Hassani et al. 2001; Hosokawa et al. 2004; Miyazaki et al. 2004; Roesch et al. 2007; Roesch and Olson 2004; Schultz 2002; Setlow et al. 2003; Tindell et al. 2006; Tremblay and Schultz 1999; van Duuren et al. 2007; Wallis and Miller 2003). In the striatum and in the OFC, it has also been shown that neurons can fire more strongly for the cue associated with the preferred reward (Cromwell and Schultz 2003; Hassani et al. 2001; Padoa-Schioppa and Assad 2006; Setlow et al. 2003; Tremblay and Schultz 1999). It might well be possible that encoding the relative value of the reward observed here at the level of the STN is under the influence of either the traditional, indirect pathway—indirectly linking the striatum/NAc to the STN— or the cortex via the hyperdirect pathway (Nambu et al. 2002). The importance of the hyperdirect pathway is in line with evidence of several commonalities between OFC and STN (Eagle and Baunez 2010). Neurons encoding cocaine reward have been shown in both the NAc and the PFC (Carelli and Deadwyler 1994; Chang et al. 1994, 1996, 1997; Peoples et al. 1997; Rebec and Sun 2005). Both structures may thus promote STN encoding of cocaine reward. Furthermore, separate populations responding to sucrose and cocaine have been found in the NAc (Bowman et al. 1996; Carelli and Deadwyler 1994; Carelli et al. 2000).

This would support the idea that differential encoding of the nature of the reward in the STN is under the influence of the NAc activity, possibly via the ventral pallidum, forming different microcircuits within the basal ganglia that encode natural rewards and drugs of abuse. The stronger encoding of the sucrose reward may explain why inactivation of the STN has opposite effects on the motivation for sucrose or for cocaine (Baunez et al. 2005; Rouaud et al. 2010). Indeed, a higher activation of the STN elicits an activation of the output nuclei of the basal ganglia, leading to an inhibition of the thalamus. In the normal state, an increase of STN neuron activation, when rats work for sucrose compared with cocaine, could thus lead to an inhibition to take sucrose. The STN inactivation would disrupt this “hyperactivation” and then result in an increased motivation for food (Baunez et al. 2002). On the contrary, the poor activation of the STN elicited by cocaine would diminish the effect of the lesion on motivation for cocaine. All of these findings reveal very important functions of STN neurons that are critical to better understand their role in motivational processes and reward circuit. With specialized populations for various types of reward, STN is an interesting structure, where motivation for different rewards can be dissociated, as confirmed by behavioral data (Baunez et al. 2005; Rouaud et al. 2010). This is a critical aspect for the treatment of addiction. ACKNOWLEDGMENTS Drs. Saleem Nicola, Frederic Ambroggi, and Sabrina Ravel are greatly acknowledged for helpful comments on the manuscript and corrections of the English formulation. Present address of S. Lardeux: Dept. of Psychiatry and Behavioral Science, Albert Einstein College of Medicine, Bronx, NY 10461. GRANTS Support for this work has been provided by grants from the Centre National de la Recherche Scientifique (CNRS), the Université de Provence, and the


STN NEURONS AND COCAINE VS. SUCROSE REWARD Agence Nationale pour la Recherche (ANR-05-JC05_48262 and ANR-09MNPS-028-01 to C. Baunez). S. Lardeux was supported by a grant from the Ministry of Education and Research. DISCLOSURES The authors declare no competing financial interests. AUTHOR CONTRIBUTIONS Author contributions: S.L., D.P., R.P., and C.B. conception and design of research; S.L. performed experiments; S.L. analyzed data; S.L. and C.B. interpreted results of experiments; S.L. prepared figures; S.L. drafted manuscript; S.L. and C.B. edited and revised manuscript; S.L., M.C., and C.B. approved final version of manuscript. REFERENCES Absher JR, Vogt BA, Clark DG, Flowers DL, Gorman DG, Keyes JW, Wood FB. Hypersexuality and hemiballism due to subthalamic infarction. Neuropsychiatry Neuropsychol Behav Neurol 13: 220 –229, 2000. Afsharpour S. Topographical projections of the cerebral cortex to the subthalamic nucleus. J Comp Neurol 236: 14 –28, 1985. Amiez C, Joseph JP, Procyk E. Anterior cingulate error-related activity is modulated by predicted reward. Eur J Neurosci 21: 3447–3452, 2005. Arkadir D, Morris G, Vaadia E, Bergman H. Independent coding of movement direction and reward prediction by single pallidal neurons. J Neurosci 24: 10047–10056, 2004. Aron AR, Poldrack RA. Cortical and subcortical contributions to stop signal response inhibition: role of the subthalamic nucleus. J Neurosci 26: 2424 – 2433, 2006. Baunez C, Amalric M, Robbins TW. Enhanced food-related motivation after bilateral lesions of the subthalamic nucleus. J Neurosci 22: 562–568, 2002. Baunez C, Dias C, Cador M, Amalric M. The subthalamic nucleus exerts opposite control on cocaine and ‘natural’ rewards. Nat Neurosci 8: 484 – 489, 2005. Baunez C, Lardeux S. Frontal cortex-like functions of the subthalamic nucleus. Front Syst Neurosci 5: 83, 2011. Baunez C, Nieoullon A, Amalric M. In a rat model of parkinsonism, lesions of the subthalamic nucleus reverse increases of reaction time but induce a dramatic premature responding deficit. J Neurosci 15: 6531– 6541, 1995. Baunez C, Robbins TW. Bilateral lesions of the subthalamic nucleus induce multiple deficits in an attentional task in rats. Eur J Neurosci 9: 2086 –2099, 1997. Baunez C, Yelnik J, Mallet L. Six questions on the subthalamic nucleus: lessons from animal models and from stimulated patients. Neuroscience 198: 193–204, 2011. Bowman EM, Aigner TG, Richmond BJ. Neural signals in the monkey ventral striatum related to motivation for juice and cocaine rewards. J Neurophysiol 75: 1061–1073, 1996. Brown P, Chen CC, Wang S, Kuhn AA, Doyle L, Yarrow K, Nuttin B, Stein J, Aziz T. Involvement of human basal ganglia in offline feedback control of voluntary movement. Curr Biol 16: 2129 –2134, 2006. Carelli RM, Deadwyler SA. A comparison of nucleus accumbens neuronal firing patterns during cocaine self-administration and water reinforcement in rats. J Neurosci 14: 7735–7746, 1994. Carelli RM, Ijames SG, Crumling AJ. Evidence that separate neural circuits in the nucleus accumbens encode cocaine versus “natural” (water and food) reward. J Neurosci 20: 4255– 4266, 2000. Chang JY, Paris JM, Sawyer SF, Kirillov AB, Woodward DJ. Neuronal spike activity in rat nucleus accumbens during cocaine self-administration under different fixed-ratio schedules. Neuroscience 74: 483– 497, 1996. Chang JY, Sawyer SF, Lee RS, Woodward DJ. Electrophysiological and pharmacological evidence for the role of the nucleus accumbens in cocaine self-administration in freely moving rats. J Neurosci 14: 1224 –1244, 1994. Chang JY, Sawyer SF, Paris JM, Kirillov A, Woodward DJ. Single neuronal responses in medial prefrontal cortex during cocaine self-administration in freely moving rats. Synapse 26: 22–35, 1997. Cromwell HC, Schultz W. Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J Neurophysiol 89: 2823–2838, 2003. Darbaky Y, Baunez C, Arecchi P, Legallet E, Apicella P. Reward-related neuronal activity in the subthalamic nucleus of the monkey. Neuroreport 16: 1241–1244, 2005.

1509

Degos B, Deniau JM, Le Cam J, Mailly P, Maurice N. Evidence for a direct subthalamo-cortical loop circuit in the rat. Eur J Neurosci 27: 2599 –2610, 2008. Eagle DM, Baunez C. Is there an inhibitory-response-control system in the rat? Evidence from anatomical and pharmacological studies of behavioral inhibition. Neurosci Biobehav Rev 34: 50 –72, 2010. Eagle DM, Baunez C, Hutcheson DM, Lehmann O, Shah AP, Robbins TW. Stop-signal reaction-time task performance: role of prefrontal cortex and subthalamic nucleus. Cereb Cortex 18: 178 –188, 2008. Feierstein CE, Quirk MC, Uchida N, Sosulski DL, Mainen ZF. Representation of spatial goals in rat orbitofrontal cortex. Neuron 51: 495–507, 2006. Grossman SE, Fontanini A, Wieskopf JS, Katz DB. Learning-related plasticity of temporal coding in simultaneously recorded amygdala-cortical ensembles. J Neurosci 28: 2864 –2873, 2008. Hassani OK, Cromwell HC, Schultz W. Influence of expectation of different rewards on behavior-related neuronal activity in the striatum. J Neurophysiol 85: 2477–2489, 2001. Hauber W, Bohn I, Giertler C. NMDA, but not dopamine D(2), receptors in the rat nucleus accumbens are involved in guidance of instrumental behavior by stimuli predicting reward magnitude. J Neurosci 20: 6282– 6288, 2000. Hosokawa T, Kato K, Inoue M, Mikami A. Neurons in the orbitofrontal cortex code both visual shapes and reward types. Neuroreport 15: 1493– 1496, 2004. Isoda M, Hikosaka O. Role for subthalamic nucleus neurons in switching from automatic to controlled eye movement. J Neurosci 28: 7209 –7218, 2008. Lardeux S, Baunez C. Alcohol preference influences the subthalamic nucleus control on motivation for alcohol in rats. Neuropsychopharmacology 33: 634 – 642, 2008. Lardeux S, Pernaud R, Paleressompoulle D, Baunez C. Beyond the reward pathway: coding reward magnitude and error in the rat subthalamic nucleus. J Neurophysiol 102: 2526 –2537, 2009. Lenoir M, Serre F, Cantin L, Ahmed SH. Intense sweetness surpasses cocaine reward. PLoS One 2: e698, 2007. Matsumura M, Kojima J, Gardiner TW, Hikosaka O. Visual and oculomotor functions of monkey subthalamic nucleus. J Neurophysiol 67: 1615– 1632, 1992. Maurice N, Deniau JM, Glowinski J, Thierry AM. Relationships between the prefrontal cortex and the basal ganglia in the rat: physiology of the corticosubthalamic circuits. J Neurosci 18: 9539 –9546, 1998a. Maurice N, Deniau JM, Menetrey A, Glowinski J, Thierry AM. Prefrontal cortex-basal ganglia circuits in the rat: involvement of ventral pallidum and subthalamic nucleus. Synapse 29: 363–370, 1998b. Mink JW. The basal ganglia: focused selection and inhibition of competing motor programs. Prog Neurobiol 50: 381– 425, 1996. Miyazaki K, Miyazaki KW, Matsumoto G. Different representation of forthcoming reward in nucleus accumbens and medial prefrontal cortex. Neuroreport 15: 721–726, 2004. Nambu A, Tokuno H, Takada M. Functional significance of the corticosubthalamo-pallidal ‘hyperdirect’ pathway. Neurosci Res 43: 111–117, 2002. Padoa-Schioppa C, Assad JA. Neurons in the orbitofrontal cortex encode economic value. Nature 441: 223–226, 2006. Parent A, Hazrati LN. Functional anatomy of the basal ganglia. II. The place of subthalamic nucleus and external pallidium in basal ganglia circuitry. Brain Res Rev 20: 128 –154, 1995. Pasquereau B, Nadjar A, Arkadir D, Bezard E, Goillandeau M, Bioulac B, Gross CE, Boraud T. Shaping of motor responses by incentive values through the basal ganglia. J Neurosci 27: 1176 –1183, 2007. Paxinos G, Watson C. The Rat Brain in Stereotaxic Coordinates. Sydney, Australia: Academic, 2005. Peoples LL, Uzwiak AJ, Gee F, West MO. Operant behavior during sessions of intravenous cocaine infusion is necessary and sufficient for phasic firing of single nucleus accumbens neurons. Brain Res 757: 280 –284, 1997. Rebec GV, Sun W. Neuronal substrates of relapse to cocaine-seeking behavior: role of prefrontal cortex. J Exp Anal Behav 84: 653– 666, 2005. Roesch MR, Calu DJ, Schoenbaum G. Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat Neurosci 10: 1615–1624, 2007. Roesch MR, Olson CR. Neuronal activity related to reward value and motivation in primate frontal cortex. Science 304: 307–310, 2004. Rouaud T, Lardeux S, Panayotis N, Paleressompoulle D, Cador M, Baunez C. Reducing the desire for cocaine with subthalamic nucleus deep brain stimulation. Proc Natl Acad Sci USA 107: 1196 –1200, 2010.


1510


Schoenbaum G, Chiba AA, Gallagher M. Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat Neurosci 1: 155–159, 1998. Schultz W. Getting formal with dopamine and reward. Neuron 36: 241–263, 2002. Schultz W, Dickinson A. Neuronal coding of prediction errors. Annu Rev Neurosci 23: 473–500, 2000. Setlow B, Schoenbaum G, Gallagher M. Neural encoding in ventral striatum during olfactory discrimination learning. Neuron 38: 625– 636, 2003. Shi LH, Luo F, Woodward DJ, Chang JY. Neural responses in multiple basal ganglia regions during spontaneous and treadmill locomotion tasks in rats. Exp Brain Res 157: 303–314, 2004. Sugimoto T, Hattori T, Mizuno N, Itoh K, Sato M. Direct projections from the centre median-parafascicular complex to the subthalamic nucleus in the cat and rat. J Comp Neurol 214: 209 –216, 1983. Taha SA, Nicola SM, Fields HL. Cue-evoked encoding of movement planning and execution in the rat nucleus accumbens. J Physiol 584: 801– 818, 2007. Teagarden MA, Rebec GV. Subthalamic and striatal neurons concurrently process motor, limbic, and associative information in rats performing an operant task. J Neurophysiol 97: 2042–2058, 2007. Temel Y, Kessels A, Tan S, Topdag A, Boon P, Visser-Vandewalle V. Behavioural changes after bilateral subthalamic stimulation in advanced Parkinson disease: a systematic review. Parkinsonism Relat Disord 12: 265–272, 2006.

Tindell AJ, Smith KS, Pecina S, Berridge KC, Aldridge JW. Ventral pallidum firing codes hedonic reward: when a bad taste turns good. J Neurophysiol 96: 2399 –2409, 2006. Tremblay L, Schultz W. Relative reward preference in primate orbitofrontal cortex. Nature 398: 704 –708, 1999. Trillet M, Vighetto A, Croisile B, Charles N, Aimard G. [Hemiballismus with logorrhea and thymo-affective disinhibition caused by hematoma of the left subthalamic nucleus]. Rev Neurol (Paris) 151: 416 – 419, 1995. van Duuren E, Escamez FA, Joosten RN, Visser R, Mulder AB, Pennartz CM. Neural coding of reward magnitude in the orbitofrontal cortex of the rat during a five-odor olfactory discrimination task. Learn Mem 14: 446 – 456, 2007. Wallis JD, Miller EK. Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task. Eur J Neurosci 18: 2069 –2081, 2003. Watanabe M. The appropriateness of behavioral responses coded in post-trial activity of primate prefrontal units. Neurosci Lett 101: 113– 117, 1989. Winstanley CA, Baunez C, Theobald DE, Robbins TW. Lesions to the subthalamic nucleus decrease impulsive choice but impair autoshaping in rats: the importance of the basal ganglia in Pavlovian conditioning and impulse control. Eur J Neurosci 21: 3107–3116, 2005. Witjas T, Baunez C, Henry JM, Delfini M, Regis J, Cherif AA, Peragut JC, Azulay JP. Addiction in Parkinson’s disease: impact of subthalamic nucleus deep brain stimulation. Mov Disord 20: 1052–1055, 2005.
