On habits and addiction: an associative analysis of compulsive drug seeking

doi:10.1016/j.ddmod.2009.07.004

Drug Discovery Today: Disease Models

Volume 5, Issue 4, Winter 2008, Pages 235-245

https://doi.org/10.1016/j.ddmod.2009.07.004 Get rights and content

The processes that underlie the pathological pursuit of drugs in addiction and that support the transition from casual drug taking to their compulsive pursuit have recently been proposed to reflect the interaction of two action control processes that mediate the goal-directed and habitual control of actions for natural rewards. Here we describe the evidence for these learning processes, their associate structure and the motivational mechanisms through which their operation is translated into performance. Finally, we describe the potential changes in the interaction between habitual and goal-directed processes induced by drug addiction that subserve compulsive drug pursuit; that is the increase in habit learning and reduction in the regulation of habits induced by changes in the circuitry that mediates goal-directed action.

Section editors:

Nigel Maidment and Niall Murphy – Hatos Center for Neuropharmacology, UCLA, Los Angeles, CA, USA

M. Foster Olive – Center for Drug and Alcohol Programs, Medical University of South Carolina, Charleston, SC, USA

Section snippets

What is habit learning?

In the field of learning theory, the term habit tends to be reserved for the product of stimulus–response learning [5, 6]. Consider an instrumental conditioning experiment in which a hungry rat is rewarded with food pellets for pressing a lever. In general, theories of stimulus–response learning assume that these food pellet deliveries strengthen, or reinforce, an association between prevailing stimuli (e.g. the experimental context or other more punctate stimuli such as the sight of the lever)

Tests of goal-directed action selection

Some time ago, on the basis of analyses of goal-directed behavior derived from human action theory (e.g. [8]), Dickinson and Balleine [9] argued that, to be classified as goal-directed, the performance of an action must be shown to depend on two things: (i) the causal relationship between that action and its outcome (i.e. its action–outcome contingency) and (ii) the current motivational value of that outcome. It turns out that under certain training conditions instrumental lever pressing in

Pavlovian–instrumental interactions

As with the contingency degradation effect, the outcome-selectivity of outcome devaluation allows one to rule out alternative interpretations of the sensitivity of instrumental performance to the devaluation manipulation. To understand why this is the case, it is important to consider another role that contextual cues could play in instrumental conditioning (i.e. apart from their putative role as discriminative, or eliciting, stimuli in habit learning). In a typical instrumental conditioning

Habit formation

As discussed above, the sensitivity of instrumental performance to manipulations of outcome value and action–outcome contingency degradation indicates that rats can apply a goal-directed strategy to control the performance of instrumental actions. Although this might be taken to imply that it is the R–O and not the S–R learning process that dominates instrumental learning, this is not always the case. Certain types of training are known to generate performance that is relatively insensitive to

Modeling compulsive drug seeking in animals

It is instructive at this point to reconsider the habit learning account of drug addiction in light of the behavioral findings described above. Indeed, on the basis of these findings, it seems improbable that drugs produce compulsive behavior merely by facilitating the rate of habit formation. Although this might address some of the inflexibility of drug-seeking behavior, given what we know about the habitual control of action selection in both rodents and humans this position should anticipate

Conclusion

During the development of addiction the pursuit of drugs of abuse becomes progressively less goal-directed and progressively more habitual coming under the control of internal and external states and stimuli. Understandably, therefore, recent theory and research on addiction has begun to focus on the habit learning process and its behavioral and neural bases. It is important, however, to distinguish habitual drug seeking from other forms of habitual behavior. Under normal conditions, habit

References (59)

A. Dickinson et al.
Reinforcer specificity of the suppression of instrumental performance on a non-contingent schedule
Behav. Process.
(1989)
R.M. Colwill et al.
Associative structures in instrumental learning
B.W. Balleine et al.
Goal-directed instrumental action: contingency and incentive learning and their cortical substrates
Neuropharmacology
(1998)
T.W. Robbins
Limbic–striatal interactions in reward-related processes
Neurosci. Biobehav. Rev.
(1989)
J.M. Kruse
Pavlovian conditioned stimulus effects upon instrumental choice behavior are reinforcer specific
Learn. Motiv.
(1983)
L.H. Corbit et al.
The role of prelimbic cortex in instrumental conditioning
Behav. Brain Res.
(2003)
R.C. Bolles
Comparisons of stimulus learning and response learning in a punishment situation
Learn. Motiv.
(1980)
H. Samson et al.
Oral ethanol self-administration in the rat: effect of naloxone
Pharmacol. Biochem. Behav.
(1985)
B.J. Everitt
Associative processes in addiction and reward. The role of amygdala–ventral striatal subsystems
Ann. N. Y. Acad. Sci.
(1999)
B.J. Everitt et al.
Neural systems of reinforcement for drug addiction: from actions to habits to compulsion
Nat. Neurosci.
(2005)

A. Redish

A unified framework for addiction: vulnerabilities in the decision process

Behav. Brain Sci.

(2008)

S. Tiffany

What can dependence theories tell us about assessing the emergence of tobacco dependence?

Addiction

(2004)

A. Dickinson

Actions and habits: the development of behavioural autonomy

Philos. Trans. Roy. Soc. Lond. B

(1985)

A. Dickinson

Instrumental conditioning

C.D. Adams et al.

Instrumental responding following reinforcer devaluation

Q. J. Exp. Psychol.

(1981)

M. Frese et al.

Goal Directed Behavior: The Concept of Action in Psychology

(1985)

A. Dickinson et al.

Actions and responses: the dual psychology of behaviour

L.J. Hammond

The effect of contingency upon appetitive conditioning of free operant behavior

J. Exp. Anal. Behav.

(1980)

J. Davis et al.

Differential reinforcement of other behavior (DRO): a yoked-control comparison

J. Exp. Anal. Behav.

(1971)

B.W. Balleine

Incentive processes in instrumental conditioning

A. Dickinson et al.

Motivational control of goal-directed action

Anim. Learn. Behav.

(1994)

B.W. Balleine et al.

The role of incentive learning in instrumental outcome revaluation by specific satiety

Anim. Learn. Behav.

(1998)

B.W. Balleine

Taste, disgust and value: taste aversion learning and outcome encoding in instrumental conditioning

B.W. Balleine et al.

Still at the choice point: action selection and initiation in instrumental conditioning

Ann. N. Y. Acad. Sci.

(2007)

R.A. Rescorla

Response–outcome versus outcome–response associations in instrumental learning

Anim. Learn. Behav.

(1992)

R.A. Rescorla et al.

Two-process learning theory: relationships between Pavlovian conditioning and instrumental learning

Psychol. Rev.

(1967)

M.A. Trapold et al.

The second learning process in instrumental conditioning

L.H. Corbit

The role of the nucleus accumbens in instrumental conditioning: evidence of a functional dissociation between accumbens core and shell

J. Neurosci.

(2001)

A. Dickinson

Bidirectional instrumental conditioning

Q. J. Exp. Psychol. B

(1996)

Cited by (105)

Changes in dorsomedial striatum activity during expression of goal-directed vs. habit-like cue-induced cocaine seeking
2024, Addiction Neuroscience
A preclinical model of cue exposure therapy, cue extinction, reduces cue-induced cocaine seeking that is goal-directed but not habit-like. Goal-directed and habitual behaviors differentially rely on the dorsomedial striatum (DMS) and dorsolateral striatum (DLS), but the effects of cue extinction on dorsal striatal responses to cue-induced drug seeking are unknown. We used fiber photometry in rats trained to self-administer cocaine paired with an audiovisual cue to examine how dorsal striatal intracellular calcium and extracellular dopamine activity differs between goal-directed and habit-like cue-induced cocaine seeking and how it is impacted by cue extinction. After minimal fixed-ratio training, rats showed enhanced DMS and DLS calcium responses to cue-reinforced compared to unreinforced lever presses. After rats were trained on goal-promoting fixed ratio schedules or habit-promoting second-order schedules of reinforcement, different patterns of dorsal striatal calcium and dopamine responses to cue-reinforced lever presses emerged. Rats trained on habit-promoting second-order schedules showed reduced DMS calcium responses and enhanced DLS dopamine responses to cue-reinforced lever presses. Cue extinction reduced calcium responses during subsequent drug seeking in the DMS, but not in the DLS. Therefore, cue extinction may reduce goal-directed behavior through its effects on the DMS, whereas habit-like behavior and the DLS are unaffected.
Punishment resistance for cocaine is associated with inflexible habits in rats
2024, Addiction Neuroscience
Addiction is characterized by continued drug use despite negative consequences. In an animal model, a subset of rats continues to self-administer cocaine despite footshock consequences, showing punishment resistance. We sought to test the hypothesis that punishment resistance arises from failure to exert goal-directed control over habitual cocaine seeking. While habits are not inherently permanent or maladaptive, continued use of habits under conditions that should encourage goal-directed control makes them maladaptive and inflexible. We trained male and female Sprague Dawley rats on a seeking-taking chained schedule of cocaine self-administration. We then exposed them to four days of punishment testing in which footshock was delivered randomly on one-third of trials. Before and after punishment testing (four days pre-punishment and ≥ four days post-punishment), we assessed whether cocaine seeking was goal-directed or habitual using outcome devaluation via cocaine satiety. We found that punishment resistance was associated with continued use of habits, whereas punishment sensitivity was associated with increased goal-directed control. Although punishment resistance for cocaine was not predicted by habitual responding pre-punishment, it was associated with habitual responding post-punishment. In parallel studies of food self-administration, we similarly observed that punishment resistance was associated with habitual responding post-punishment but not pre-punishment in males, although it was associated with habitual responding both pre- and post-punishment in females, indicating that punishment resistance was predicted by habitual responding in food-seeking females. These findings indicate that punishment resistance is related to habits that have become inflexible and persist under conditions that should encourage a transition to goal-directed behavior.
Modulation of cue value and the augmentation of heroin seeking in chronically food-restricted male rats under withdrawal
2023, Pharmacology Biochemistry and Behavior
Food restriction augments drug seeking in abstinent rats. The underlying motivational mechanisms, however, remain unclear. We hypothesized that caloric restriction enhances the incentive value attributed to drug-associated cues and, in turn, augments drug seeking. Male rats were trained to lever-press for heroin, and then moved to the animal colony for a forced-abstinence period. Rats were maintained on free access to food (Sated) or subjected to 14 days of food restriction (FDR). In a series of experiments, we assessed the effect of food-restriction on the incentive value of heroin-associated cues. Tests included performance under a progressive ratio (PR) schedule of reinforcement maintained by heroin-associated cues, acquisition of a novel operant response reinforced by drug-associated cues, effect of food-restriction on operant response reinforced by neutral cues, acquisition of a novel operant response reinforced by drug-associated or neutral cues, and the effect of food-restriction on operant response reinforced by drug-associated or neutral cues, under a discrete choice procedure.
Food-restriction did not change breakpoints in PR maintained by heroin-associated cues. FDR rats acquired the novel response at a greater level compared to the Sated group. Food-restriction-induced increase in novel-response rate was observed for both heroin-paired and the neutral cue. Responding for a heroin-associated cue was greater than for the neutral cue in both Sated and FDR groups. Response rate for the neutral cue, however, was greater in the FDR versus Sated group.
Our findings suggest that food restriction increases the conditioned motivational properties of environmental stimuli, including, but not exclusive to, heroin-paired cues.
Immp2l knockdown in male mice increases stimulus-driven instrumental behaviour but does not alter goal-directed learning or neuron density in cortico-striatal circuits in a model of Tourette syndrome and autism spectrum disorder
2023, Behavioural Brain Research
Cortico-striatal neurocircuits mediate goal-directed and habitual actions which are necessary for adaptive behaviour. It has recently been proposed that some of the core symptoms of autism spectrum disorder (ASD) and Gilles de la Tourette syndrome (GTS), such as tics and other repetitive behaviours, may emerge because of imbalances in these neurocircuits. We have recently developed a model of ASD and GTS by knocking down Immp2l, a mitochondrial gene frequently associated with these disorders. The current study sought to determine whether Immp2l knockdown (KD) in male mice alters flexible, goal- or cue- driven behaviour using procedures specifically designed to examine response-outcome and stimulus-response associations, which underlie goal-directed and habitual behaviour, respectively. Whether Immp2l KD alters neuron density in cortico-striatal neurocircuits known to regulate these behaviours was also examined. Immp2l KD mice and wild type-like mice (WT) were trained on Pavlovian and instrumental learning procedures where auditory cues predicted food delivery and lever-press responses earned a food outcome. It was demonstrated that goal-directed learning was not changed for Immp2l KD mice compared to WT mice, as lever-press responses were sensitive to changes in the value of the food outcome, and to contingency reversal and degradation. There was also no difference in the capacity of KD mice to form habitual behaviours compared to WT mice following extending training of the instrumental action. However, Immp2l KD mice were more responsive to auditory stimuli paired with food as indicated by a non-specific increase in lever response rates during Pavlovian-to-instrumental transfer. Finally, there were no alterations to neuron density in striatum or any prefrontal cortex or limbic brain structures examined. Thus, the current study suggests that Immp2l is not necessary for learned maladaptive goal or stimulus driven behaviours in ASD or GTS, but that it may contribute to increased capacity for external stimuli to drive behaviour. Alterations to stimulus-driven behaviour could potentially influence the expression of tics and repetitive behaviours, suggesting that genetic alterations to Immp2l may contribute to these core symptoms in ASD and GTS. Given that this is the first application of this battery of instrumental learning procedures to a mouse model of ASD or GTS, it is an important initial step in determining the contribution of known risk-genes to goal-directed versus habitual behaviours, which should be more broadly applied to other rodent models of ASD and GTS in the future.
Intermittent cocaine self-administration has sex-specific effects on addiction-like behaviors in rats
2023, Neuropharmacology
Intermittent access (IntA) models of cocaine self-administration were developed to better model in rodents how cocaine is used by human drug users. Compared to traditional continuous access (ContA) models, IntA has been shown to enhance several pharmacological and behavioral effects of cocaine, but few studies have examined sex differences in IntA. Moreover, no one has examined the efficacy of cue extinction to reduce cocaine seeking in the IntA model, which has previously shown to be ineffective in other models that promote habit-like cocaine seeking. Therefore, rats were implanted with jugular vein catheters and dorsolateral striatum (DLS) cannulae and trained to self-administer cocaine paired with an audiovisual cue with ContA or IntA. In subsets of rats, we evaluated: the ability of Pavlovian cue extinction to reduce cue-induced drug seeking; motivation for cocaine using a progressive ratio procedure; punishment-resistant cocaine taking by pairing cocaine infusions with footshocks; and dependence of drug-seeking on DLS dopamine (a measure of habit-like behavior) with the dopamine antagonist cis-flupenthixol. Overall, cue extinction reduced cue-induced drug seeking after ContA or IntA. Compared to ContA, IntA resulted in increased motivation for cocaine exclusively in females, but IntA facilitated punished cocaine self-administration exclusively in males. After 10 days of IntA training, but not fewer, drug-seeking was dependent on DLS dopamine most notably in males. Our results suggest that IntA may be valuable for identifying sex differences in the early stages of drug use and provide a foundation for the investigation of the mechanisms involved.
Restoring the youthful state of striatal plasticity in aged mice re-enables cognitive control of action
2023, Current Biology
Multidisciplinary evidence suggests that the control of voluntary action arbitrates between two major forms of behavioral processing: cognitively guided (or goal directed) and autonomously guided (or habitual). Brain-state irregularities affecting the striatum—such as aging—commonly shift control toward the latter, although the responsible neural mechanisms remain unknown. Combining instrumental conditioning with cell-specific mapping and chemogenetics in striatal neurons, we explored strategies that invigorate goal-directed capacity in aged mice. We found that, under conditions favoring goal-directed control, aged animals resiliently expressed autonomously guided behavior, a response that was underpinned by a characteristic one-to-one functional engagement of the two main neuronal populations in the striatum—D1- and D2-dopamine receptor-expressing spiny projection neurons (SPNs). Chemogenetically induced desensitization of D2-SPN signaling in aged transgenic mice recapitulated the striatal plasticity state observed in young mice, an effect that shifted behavior toward vigorous, goal-directed action. Our findings contribute to the understanding of the neural bases of behavioral control and propose neural system interventions that enhance cognitive functioning in habit-prone brains.

View all citing articles on Scopus

View full text

Animal models of addictionOn habits and addiction: an associative analysis of compulsive drug seeking

Section editors:

Section snippets

What is habit learning?

Tests of goal-directed action selection

Pavlovian–instrumental interactions

Habit formation

Modeling compulsive drug seeking in animals

Conclusion

Behav. Process.

Neuropharmacology

Neurosci. Biobehav. Rev.

Learn. Motiv.

Behav. Brain Res.

Learn. Motiv.

Pharmacol. Biochem. Behav.

Associative processes in addiction and reward. The role of amygdala–ventral striatal subsystems

Ann. N. Y. Acad. Sci.

Neural systems of reinforcement for drug addiction: from actions to habits to compulsion

Nat. Neurosci.

A unified framework for addiction: vulnerabilities in the decision process

Behav. Brain Sci.

What can dependence theories tell us about assessing the emergence of tobacco dependence?

Addiction

Actions and habits: the development of behavioural autonomy

Philos. Trans. Roy. Soc. Lond. B

Instrumental conditioning

Instrumental responding following reinforcer devaluation

Q. J. Exp. Psychol.

Goal Directed Behavior: The Concept of Action in Psychology

Actions and responses: the dual psychology of behaviour

The effect of contingency upon appetitive conditioning of free operant behavior

J. Exp. Anal. Behav.

Differential reinforcement of other behavior (DRO): a yoked-control comparison

J. Exp. Anal. Behav.

Incentive processes in instrumental conditioning

Motivational control of goal-directed action

Anim. Learn. Behav.

The role of incentive learning in instrumental outcome revaluation by specific satiety

Anim. Learn. Behav.

Taste, disgust and value: taste aversion learning and outcome encoding in instrumental conditioning

Still at the choice point: action selection and initiation in instrumental conditioning

Ann. N. Y. Acad. Sci.

Response–outcome versus outcome–response associations in instrumental learning

Anim. Learn. Behav.

Two-process learning theory: relationships between Pavlovian conditioning and instrumental learning

Psychol. Rev.

The second learning process in instrumental conditioning

The role of the nucleus accumbens in instrumental conditioning: evidence of a functional dissociation between accumbens core and shell

J. Neurosci.

Bidirectional instrumental conditioning

Q. J. Exp. Psychol. B

Animal models of addiction
On habits and addiction: an associative analysis of compulsive drug seeking