Preprint August 15, 2011 Huys et al. Neurobiological understanding and computational models of addictive behavior. To appear in Mishara et al. (Ed.): Phenomenological Neuropsychiatry: Bridging the Clinic and Clinical Neuroscience. 1 Neurobiological understanding and computational models of addictive behavior Quentin JM Huys 1,2 , Anne Beck 3 , Peter Dayan 1 and Andreas Heinz 3 1 Gatsby Computational Neuroscience Unit, University College London, UK 2 Wellcome Trust Centre for Neuroimaging, University College London, UK 3 Department of Psychiatry and Psychotherapy, Charité - Universitätsmedizin Berlin, Germany Abstract An increasing wealth of experimental detail is available about the development and nature of addiction. Critical issues such as the varying vulnerabilities of different subjects are being illuminated at many levels of psychological and neurobiological detail. Furthermore, a rich theoretical understanding is starting to evolve in the field of neural reinforcement learning. In this chapter, we consider some of the currently most pressing issues in the interface between experiment and theory, notably the so-called “compulsive” phase of drug taking. Introduction People suffering from addictive disorders show impairments in decision making that are central to the conceptualization of substance dependence and consequently are enshrined in both DSM IV- TR (American Psychiatric Association, 1994) and ICD-10 (World Health Organization, 1990). The development of addiction is characterized by several key features (Beck et al., 2011; Gelder et al., 2006; Heinz et al., 2009a; Koob, 2003). These include the development of tolerance to the effects of the drug of abuse, which is accompanied by adaptations in central neurotransmission. Strikingly, tolerance does not decrease, but rather seems to coincide with an increase in, the lure of drugs. Once established, addictions are accompanied by intense cravings for the drug and by elaborate behavioral distortions aimed at obtaining the drug. Recent work has particularly emphasized how normal reinforcers lose the ability to control behavior: people with substance dependences continue to take drugs despite apparently understanding and acknowledging the devastating effect this may have on their life and despite expressed desires (and frequent attempts) to abstain. Though largely preclinical, these recent studies have resulted in an increasingly detailed description of how initially harmless drug-taking for hedonic reasons is transformed into a maladaptive and “compulsive” pattern of drug-taking resistant to negative outcomes (Vanderschuren and Everitt, 2004; Everitt and Robbins, 2005). Finally, a key feature of addiction is the tendency for relapse, both early and years or decades later. The dominant accounts of decision making come from the evolving field of neural reinforcement learning (Sutton and Barto, 1998), which links computational notions of optimal control (Puterman, 2005), psychological data on conditioning (Sutton and Barto, 1990), together with the neural substrate in the striatum, and its associated neuromodulatory, amygdala and cortical inputs (Niv, 2009; Daw and Doya, 2006). There is now extensive evidence that addictive substances modulate and derail adaptive decision-making in part by influencing the function of dopamine.