Although, there are considerable works on the neural mechanisms of reward-based learning and decision making, and most of them mention that addiction can be explained by malfunctioning in these cognitive processes, there are very few computational models. This paper focuses on nicotine addiction, and a computational model for nicotine addiction is proposed based on the neurophysiological basis of addiction. The model compromises different levels ranging from molecular basis to systems level, and it demonstrates three different possible behavioral patterns which are addict, nonaddict, and indecisive. The dynamical behavior of the proposed model is investigated with tools used in analyzing nonlinear dynamical systems, and the relation between the behavioral patterns and the dynamics of the system is discussed. 1. Introduction The value of an experience or an action is imposed by the reward gained afterwards. An action inducing a greater reward is sensed as a superior action, and thus the successive occurrences of this type of actions are rewarded more frequently [1]. In the case of addiction, the abusive substance (nicotine, drugs, etc.) has a value greater than other forms of reward imposing actions. Some persistent modifications in the synaptic plasticity in the neural subsystems of the brain are believed to lead to addiction; thus, it is claimed that addiction is a disorder in the mesolimbic system which develops by the modification of responses to rewarding actions [1–4]. Mislead by the overemphasized reward perception, the addicts compulsively seek for the substance they are addicted to. Since the reward mechanism has been persistently changed, addicts usually cannot be completely cured, and they often relapse into drug use after treatment [2]. Considering the negative social and physical impacts of addiction, any attempt to understand the neural mechanisms underlying this phenomenon is valuable. So, in this study, a computational model of addiction is given. We propose a model for nicotine addiction which is composed of a cortico-striato-thalamic action selection circuit and a dopamine signaling unit operating according to reinforcement learning. The proposed model is based on the interaction of neural structures known to be taking role in addiction and the effect of neurotransmitters on these structures. The proposed model focuses on the neurophysiological basis of addiction and the results obtained by the model show that the model is capable of demonstrating different behavioral patterns. (See the Supplementary Material containing the source codes
References
[1]
S. E. Hyman, R. C. Malenka, and E. J. Nestler, “Neural mechanisms of addiction: the role of reward-related learning and memory,” Annual Review of Neuroscience, vol. 29, pp. 565–598, 2006.
[2]
R. Spanagel and M. Heilig, “Addiction and its brain science,” Addiction, vol. 100, no. 12, pp. 1813–1822, 2005.
[3]
B. J. Everitt, J. A. Parkinson, M. C. Olmstead, M. Arroyo, P. Robledo, and T. W. Robbins, “Associative processes in addiction and reward. The role of amygdala-ventral striatal subsystems,” Annals of the New York Academy of Sciences, vol. 877, pp. 412–438, 1999.
[4]
R. N. Cardinal and B. J. Everitt, “Neural and psychological mechanisms underlying appetitive learning: links to drug addiction,” Current Opinion in Neurobiology, vol. 14, no. 2, pp. 156–162, 2004.
[5]
S. N. Haber, J. L. Fudge, and N. R. McFarland, “Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum,” Journal of Neuroscience, vol. 20, no. 6, pp. 2369–2382, 2000.
[6]
R. L. Solomon and J. D. Corbit, “An opponent-process theory of motivation,” The American Economic Review, vol. 68, no. 6, pp. 12–24, 1978.
[7]
G. F. Koob and M. Le Moal, “Neurobiological mechanisms for opponent motivational processes in addiction,” Philosophical Transactions of the Royal Society B, vol. 363, no. 1507, pp. 3113–3123, 2008.
[8]
P. Dayan, “Dopamine, reinforcement learning, and addiction,” Pharmacopsychiatry, vol. 42, pp. S56–S65, 2009.
[9]
S. Peele and B. K. Alexander, “The Meaning of Addiction,” chapter 3, Theories of Addiction, http://www.peele.net/lib/moa3.html.
[10]
M. R. Delgado, “Reward-related responses in the human striatum,” Annals of the New York Academy of Sciences, vol. 1104, pp. 70–88, 2007.
[11]
B. S. Gutkin, S. Dehaene, and J. P. Changeux, “A neurocomputational hypothesis for nicotine addiction,” Proceedings of the National Academy of Sciences of the United States of America, vol. 103, no. 4, pp. 1106–1111, 2006.
[12]
N. S. Sengor, O. Karabacak, and U. Steinmetz, “A computational model of cortico-striato-thalamic circuits in goal-directed behaviour,” Lecture Notes in Computer Science, Proceedings of ICANN, vol. 5164, no. 2, pp. 328–337, 2008.
[13]
S. Metin and N. S. Sengor, “Dynamical system approach in modeling addiction,” in Proceedings of the Brain-Inspired Cognitive Systems Conference (BICS '10), 2010.
[14]
S. Metin and N. S. Sengor, “A neurocomputational model of nicotine addiction based on reinforcement learning,” in Proceedings of the 20th International Conference on Artificial Neural Networks (ICANN '10), pp. 15–18, 2010.
[15]
S. H. Ahmed, M. Graupner, and B. Gutkin, “Computational approaches to the neurobiology of drug addiction,” Pharmacopsychiatry, vol. 42, supplement 1, pp. S144–S152, 2009.
[16]
R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, MIT Press, Cambridge, Mass, USA, 2nd edition, 1999.
[17]
A. D. Redish, “Addiction as a computational process gone awry,” Science, vol. 306, no. 5703, pp. 1944–1947, 2004.
[18]
S. Grossberg and W. E. Gutowski, “Neural dynamics of decision making under risk: affective balance and cognitive-emotional interactions,” Psychological Review, vol. 94, no. 3, pp. 300–318, 1987.
[19]
S. Grossberg and N. A. Schmajuk, “Neural dynamics of attentionally modulated Pavlovian conditioning-conditioned reinforcement, inhibition, and opponent processing_Grossberg,” Psychobiology, vol. 15, no. 3, pp. 195–240, 1987.
[20]
N. D. Daw, S. Kakade, and P. Dayan, “Opponent interactions between serotonin and dopamine,” Neural Networks, vol. 15, no. 4-6, pp. 603–616, 2002.
[21]
A. Dezfouli, P. Piray, M. M. Keramati, H. Ekhtiari, C. Lucas, and A. Mokri, “A neurocomputational model for cocaine addiction,” Neural computation, vol. 21, no. 10, pp. 2869–2893, 2009.
[22]
N. D. Daw, Reinforcement learning models of the dopamine system and their behavioral implications [Ph.D. thesis], Carnegie Mellon University, 2003.
[23]
M. Guthrie, C. E. Myers, and M. A. Gluck, “A neurocomputational model of tonic and phasic dopamine in action selection: a comparison with cognitive deficits in Parkinson's disease,” Behavioural Brain Research, vol. 200, no. 1, pp. 48–59, 2009.
[24]
M. Graupner and B. Gutkin, “Nicotine control of dopamine signaling in the Ventral Tegmental Area,” Neuroscience. In press.
[25]
M. Graupner and B. Gutkin, “Modeling nicotinic neuromodulation from global functional and network levels to nAChR based mechanisms,” Acta Pharmacologica Sinica, vol. 30, no. 6, pp. 681–693, 2009.
[26]
A. M. Oster and B. S. Gutkin, “A reduced model of DA neuronal dynamics that displays quiescence, tonic firing and bursting,” Journal of Physiology Paris, vol. 105, no. 1–3, pp. 53–58, 2011.
[27]
K. P. Mayhew, B. R. Flay, and J. A. Mott, “Stages in the development of adolescent smoking,” Drug and Alcohol Dependence, vol. 59, supplement 1, pp. 61–81, 2000.
[28]
http://medical-dictionary.thefreedictionary.com/.
[29]
P. W. Kalivas, “Neurocircuitry of addiction,” in Neuropsychopharmacology: The Fifth Generation of Progress, chapter 95, pp. 1357–1366, Lippincott, Williams & Wilkins, 2002.
[30]
G. F. Koob and M. Le Moal, “Drug addiction, dysregulation of reward, and allostasis,” Neuropsychopharmacology, vol. 24, no. 2, pp. 97–129, 2001.
[31]
D. Saal, Y. Dong, A. Bonci, and R. C. Malenka, “Drugs of abuse and stress trigger a common synaptic adaptation in dopamine neurons,” Neuron, vol. 37, no. 4, pp. 577–582, 2003.
[32]
D. W. Self, “Neural substrates of drug craving and relapse in drug addiction,” Annals of Medicine, vol. 30, no. 4, pp. 379–389, 1998.
[33]
G. Di Chiara, “Role of dopamine in the behavioural actions of nicotine related to addiction,” European Journal of Pharmacology, vol. 393, no. 1-3, pp. 295–314, 2000.
[34]
H. D. Mansvelder and D. S. McGehee, “Cellular and synaptic mechanisms of nicotine addiction,” Journal of Neurobiology, vol. 53, no. 4, pp. 606–617, 2002.
[35]
A. A. Grace, S. B. Floresco, Y. Goto, and D. J. Lodge, “Regulation of firing of dopaminergic neurons and control of goal-directed behaviors,” Trends in Neurosciences, vol. 30, no. 5, pp. 220–227, 2007.
[36]
A. J. Yu and P. Dayan, “Uncertainty, neuromodulation, and attention,” Neuron, vol. 46, no. 4, pp. 681–692, 2005.
[37]
M. R. Dranias, S. Grossberg, and D. Bullock, “Dopaminergic and non-dopaminergic value systems in conditioning and outcome-specific revaluation,” Brain Research, vol. 1238, pp. 239–287, 2008.
[38]
E. J. Nestler, “Common molecular and cellular substrates of addiction and memory,” Neurobiology of Learning and Memory, vol. 78, no. 3, pp. 637–647, 2002.
[39]
N. R. Taylor and J. G. Taylor, “Hard-wired models of working memory and temporal sequence storage and generation,” Neural Networks, vol. 13, no. 2, pp. 201–224, 2000.
[40]
J. G. Taylor and N. R. Taylor, “Analysis of recurrent cortico-basal ganglia-thalamic loops for working memory,” Biological Cybernetics, vol. 82, no. 5, pp. 415–432, 2000.
[41]
K. Gurney, T. J. Prescott, and P. Redgrave, “A computational model of action selection in the basal ganglia. I. A new functional anatomy,” Biological Cybernetics, vol. 84, no. 6, pp. 401–410, 2001.
[42]
K. Gurney, T. J. Prescott, and P. Redgrave, “A computational model of action selection in the basal ganglia. II. Analysis and simulation of behaviour,” Biological Cybernetics, vol. 84, no. 6, pp. 411–423, 2001.
