Post-traumatic stress disorder (PTSD) symptoms include behavioral avoidance which is acquired and tends to increase with time. This avoidance may represent a general learning bias; indeed, individuals with PTSD are often faster than controls on acquiring conditioned responses based on physiologically-aversive feedback. However, it is not clear whether this learning bias extends to cognitive feedback, or to learning from both reward and punishment. Here, male veterans with self-reported current, severe PTSD symptoms (PTSS group) or with few or no PTSD symptoms (control group) completed a probabilistic classification task that included both reward-based and punishment-based trials, where feedback could take the form of reward, punishment, or an ambiguous “no-feedback” outcome that could signal either successful avoidance of punishment or failure to obtain reward. The PTSS group outperformed the control group in total points obtained; the PTSS group specifically performed better than the control group on reward-based trials, with no difference on punishment-based trials. To better understand possible mechanisms underlying observed performance, we used a reinforcement learning model of the task, and applied maximum likelihood estimation techniques to derive estimated parameters describing individual participants’ behavior. Estimations of the reinforcement value of the no-feedback outcome were significantly greater in the control group than the PTSS group, suggesting that the control group was more likely to value this outcome as positively reinforcing (i.e., signaling successful avoidance of punishment). This is consistent with the control group’s generally poorer performance on reward trials, where reward feedback was to be obtained in preference to the no-feedback outcome. Differences in the interpretation of ambiguous feedback may contribute to the facilitated reinforcement learning often observed in PTSD patients, and may in turn provide new insight into how pathological behaviors are acquired and maintained in PTSD.
References
[1]
Hoge C, Castro C, Messer S, McGurk D, Cotting D, et al. (2004) Combat duty in Iraq and Afghanistan, mental health problems, and barriers to care. New England Journal of Medicine 351: 13–22.
[2]
Dohrenwend BP, Turner JB, Turse NA, Adams BG, Koenen KC, et al. (2006) The psychological risks of Vietnam for U.S. veterans: A revisit with new data and methods. Science 313: 979–982.
[3]
Jakupcak M, Conybeare D, Phelps L, Hunt S, Holmes HA, et al. (2007) Anger, hostility, and aggression among Iraq and Afghanistan War veterans reporting PTSD and subthreshold PTSD. Journal of Traumatic Stress 20: 945–954.
[4]
Mylle J, Maes M (2004) Partial posttraumatic stress disorder revisited. Journal of Affective Disorders 78: 37–48.
[5]
Weiss DS, Marmar CR, Schlenger WE, Fairbank JA, Jordan BK, et al. (1992) The prevalence of lifetime and partial post-traumatic stress in vietnam theater veterans. Journal of Traumatic Stress 5: 365–376.
[6]
Yarvis JS, Schiess L (2008) Subthreshold posttraumatic stress disorder (PTSD) as a predictor of depression, alcohol use, and health problems in veterans. Journal of Workplace Behavioral Health 23: 395–424.
[7]
Solomon Z, Horesh D, Ein-Dor T (2009) The longitudinal course of posttraumatic stress disorder symptom clusters among war veterans. Journal of Clinical Psychiatry 70: 837–843.
[8]
O'Donnell ML, Holmes AC, Creamer MC, Ellen S, Judson R, et al. (2009) The role of post-traumatic stress disorder and depression in predicting disability after injury. Medical Journal of Australia 190: S71–74.
Pitman RK, Shalev AY, Orr SP (2000) Posttraumatic stress disorder: Emotion, conditioning and memory. In: Corbetta MD, Gazzaniga MS, editors. The New Cognitive Neurosciences. New York: Plenum Press. pp. 687–700.
[11]
Myers CE, VanMeenen K, McAuley JD, Beck KD, Pang KCH, et al. (2012a) Facilitated acquisition of eyeblink conditioning in veterans with high behavioral inhibition, a risk factor for post-traumatic stress disorder (PTSD). Stress 15: 31–44.
[12]
Norrholm SD, Jovanovic T, Olin IW, Sands LA, Karapanou I, et al. (2011) Fear extinction in traumatized civilians with posttraumatic stress disorder: Relation to symptom severity. Biological Psychiatry 69: 556–563.
[13]
Orr SP, Metzger LJ, Lasko NB, Macklin ML, Peri T, et al. (2000) De novo conditioning in trauma-exposed individuals with and without posttraumatic stress disorder. Journal of Abnormal Psychology 109: 290–298.
[14]
Wessa M, Flor H (2007) Failure of extinction of fear responses in posttraumatic stress disorder: Evidence from second-order conditioning. American Journal of Psychiatry 164: 1684–1692.
[15]
Ayers ED, White J, Powell DA (2003) Pavlovian eyeblink conditioning in combat veterans with and without post-traumatic stress disorder. Integrative Physiological and Behavioral Science 38: 230–247.
[16]
Burriss L, Ayers E, Powell DA (2007) Combat veterans show normal discrimination during differential trace eyeblink conditioning but increased responsivity to the conditioned and unconditioned stimulus. Journal of Psychiatric Research 41: 785–794.
[17]
Milad MR, Orr SP, Lasko NB, Chang Y, Rauch SL, et al. (2008) Presence and acquired origin of reduced recall for fear extinction in PTSD: Results of a twin study. Journal of Psychiatric Research 42: 515–520.
[18]
Neylan TC, Lenoci M, Rothlind J, Metzler TJ, Schuff N, et al. (2004) Attention, learning, and memory in posttraumatic stress disorder. Journal of Traumatic Stress 17: 41–46.
[19]
Bódi N, Kéri S, Nagy H, Moustafa A, Myers CE, et al. (2009) Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinson's patients. Brain 132: 2385–2395.
[20]
Frank MJ, Moustafa AA, Haughey HM, Curran T, Hutchison KE (2007) Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proceedings of the National Academy of Sciences USA 104: 16311–16316.
[21]
Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD (2006) Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442: 1042–1045.
[22]
Palminteri S, Justo D, Jauffret C, Pavlicek B, Dauta A, et al. (2012) Critical roles for anterior insula and dorsal striatum in punishment-based avoidance learning. Neuron 76: 998–1009.
[23]
Cools R, Robinson OJ, Sahakian B (2008) Acute tryptophan depletion in healthy volunteers enhances punishment prediction but does not affect reward prediction. Neuropsychopharmacology 33: 2291–2299.
[24]
Seymour B, Daw ND, Roiser JP, Dayan P, Dolan R (2012) Serotonin selectively modulates reward value in human decision-making. Journal of Neuroscience 32: 5833–5842.
[25]
Palminteri S, Clair AH, Mallet L, Pessiglione M (2012) Similar improvement of reward and punishment learning by serotonin reuptake inhibitors in obsessive-compulsive disorder. Biological Psychiatry 72: 244–250.
[26]
Biederman J, Rosenbaum JF, Bolduc-Murphy EA, Faraone SV, Chaloff J, et al. (1993) A 3-year follow-up of children with and without behavioral inhibition. J Am Acad Child Adolesc Psychiatry 32: 814–821.
[27]
Hirshfeld DR, Rosenbaum JF, Biederman J, Bolduc EA, Faraone SV, et al. (1992) Stable behavioral inhibition and its association with anxiety disorder. J Am Acad Child Adolesc Psychiatry 31: 103–111.
[28]
Rosenbaum JF, Biederman J, Hirshfeld DR, Bolduc EA, Faraone SV, et al. (1991) Further evidence of an association between behavioral inhibition and anxiety disorders: results from a family study of children from a non-clinical sample. J Psychiatr Res 25: 49–65.
[29]
Sheynin J, Shikari S, Gluck MA, Moustafa AA, Servatius RJ, et al. (2013) Enhanced avoidance learning in behaviorally-inhibited young men and women. Stress 16: 289–299.
[30]
Myers CE, VanMeenen KM, Servatius RJ (2012b) Behavioral inhibition and PTSD symptoms in veterans. Psychiatry Research 196: 271–276.
[31]
Barto AG, Sutton RS, Anderson CW (1983) Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Transactions on Systems, Man and Cybernetics SMC-13: 834–846.
[32]
Sutton R (1988) Learning to predict by the methods of temporal differences. Machine Learning 3: 9–44.
[33]
Montague P, Dayan P, Sejnowski T (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. Journal of Neuroscience 16: 1936–1947.
[34]
Schultz W (1998) Predictive reward signal of dopamine neurons. Journal of Neurophysiology 80: 1–27.
[35]
Schultz W, Dayan P, Montague P (1997) A neural substrate of prediction and reward. Science 275: 1593–1599.
[36]
Bray S, O'Doherty J (2007) Neural coding of reward-prediction error signals during classical conditioning with attractive faces. Journal of Neurophysiology 97: 3036–3045.
[37]
Dickerson KC, Li J, Delgado MR (2011) Parallel contributions of distinct human memory systems during probabilistic learning. Neuroimage 55: 266–276.
[38]
O'Doherty J, Dayan P, Friston K, Critchlet H, Dolan R (2003) Temporal difference models and reward-related learning in the human brain. Neuron 28: 329–337.
[39]
O'Doherty J, Dayan P, Schultz J, Deichmann R, Fristen K, et al. (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304: 452–455.
[40]
Rodriguez PF, Aron AR, Poldrack RA (2006) Ventral-striatal/nucleus-accumbens sensitivity to prediction errors during classification learning. Human Brain Mapping 27: 306–313.
[41]
Mattfeld AT, Gluck MA, Stark CEL (2011) Functional specialization within the striatum along both the dorsal.ventral and anterior/posterior axes during associative learning via reward and punishment. Learning and Memory 18: 703–711.
[42]
Chang LJ, Doll BB, van't Wout M, Frank MJ, Sanfey AG (2010) Seeing is believing: Trustworthiness as a dynamic belief. Cognitive Psychology 61: 87–105.
[43]
Doll BB, Jacobs WJ, Sanfey AG, Frank MJ (2009) Instructional control of reinforcement learning: A behavioral and neurocomputational investigation. Brain Research 1299: 74–94.
[44]
Blanchard EB, Jones-Alexander J, Buckley TC, Forneris CA (1996) Psychometric properties of the PTSD checklist (PCL). Behavioral Research and Therapy 34: 669–673.
[45]
Weathers FW, Litz BT, Herman DS, Huska JA, Keane TM (1993) The PTSD checklist (PCL): Reliability, validity, and diagnostic utility. Annual Meeting of the International Society for Traumatic Stress Studies. San Antonio, TX.
[46]
Gladstone GL, Parker G (2005) Measuring a behaviorally inhibited temperament style: Development and initial validation of new self-report measure. Psychiatry Research 135: 133–143.
[47]
Keane TM, Fairbank JA, Caddell JM (1989) Clinical evaluation of a measure to assess combat exposure. Psychological Assessment 1: 53–55.
[48]
Booth-Kewley S, Larson GE, Highfill-McRoy RM, Garland CF, Gaskin TA (2010) Correlates of posttraumatic stress disorder symptoms in Marines back from war. Journal of Traumatic Stress 23: 69–77.
[49]
Orcutt HK, Erickson DJ, Wolfe J (2002) A prospective analysis of trauma exposure: The mediating role of PTSD symptomatology. Journal of Traumatic Stress 15: 259–266.
[50]
Rona RJ, Hooper R, Jones M, Iversen AC, Hull L, et al. (2009) The contribution of prior psychological symptoms and combat exposure to post Iraq deployment mental health in the UK military. J Trauma Stress 22: 11–19.
[51]
Fincham D, Smit J, Carey P, Stein DJ, Seedat S (2008) The relationship between behavioural inhibition, anxiety disorders, depression and CD4 counts in HIV-positive adults: A cross-sectional controlled study. AIDS Care 20: 1279–1283.
[52]
Gladstone GL, Parker GB, Mitchell PB, Wilhelm KA, Malhi GS (2005) Relationship between self-reported childhood behavioral inhibition and lifetime anxiety disorders in a clinical sample. Depression and Anxiety 22: 103–113.
[53]
Ginsberg JP, Ayers E, Burriss L, Powell DA (2008) Disruption of bradycardia associated with discriminative conditioning in combat veterans with PTSD. Neuropsychiatric Disease and Treatment 4: 635–646.
[54]
Daw ND, Doya K (2006) The computational neurobiology of learning and reward. Current Opinion in Neurobiology 16: 199–204.
[55]
Brewin CR, Andrews B, Valentine JD (2000) Meta-analysis of risk factors for posttraumatic stress disorder in trauma-exposed adults. Journal of Consulting and Clinical Psychology 68: 748–766.
Grillon C, Morgan CA (1999) Fear-potentiated startle conditioning to explicit and contextual cues in Gulf War veterans with posttraumatic stress disorder. Journal of Abnormal Psychology 108: 134–142.
[58]
Blechert J, Michael T, Vriends N, Margraf J, Wilhelm FH (2007) Fear conditioning in posttraumatic stress disorder: Evidence for delayed extinction of autonomic, experiential, and behavioural responses. Behaviour Research and Therapy 45: 2019–2033.
[59]
Jovanovic T, Norrholm SD, Blanding NQ, Phifer JE, Weiss T, et al. (2010a) Fear potentiation is associated with hypothalamic-pituitary-adrenal axis function in PTSD. Psychoneuroendocrinology 35: 846–857.
[60]
Peri T, Ben-Shakhar G, Orr SP, Shalev AY (2000) Psychophysiologic assessment of aversive conditioning in posttraumatic disorder. Biological Psychiatry 47: 512–519.
[61]
Levy-Gigi E, Kéri S, Myers CE, Lencovsky Z, Sharvit-Benbaji H, et al. (2012) Individuals with post-traumatic stress disorder show a selective deficit in generalization of associative learning. Neuropsychology 26: 758–767.
[62]
Kahneman D, Tversky A (1984) Choices, values, and frames. American Psychologist 39: 341–350.
[63]
Somlai Z, Moustafa AA, Keri S, Myers CE, Gluck MA (2011) General functioning predicts reward and punishment learning in schizophrenia. Schizophrenia Research 127: 131–136.
[64]
Maguen S, Turcotte DM, Peterson AL, Dremsa TL, Garb HN, et al. (2008) Description of risk and resilience factors among military medical personnel before deployment to Iraq. Military Medicine 173: 1–9.
[65]
Tesechi RG, Calhoun LG (1996) The Posttraumatic Growth Inventory: Measuring the positive legacy of trauma. Journal of Traumatic Stress 9: 455–471.
[66]
Kim H, Shimojo S, O'Doherty JP (2006) Is avoiding an aversive outcome rewarding? Neural substrates of avoidance learning in the human brain. PLoS Biology 4: 1453–1461.
[67]
O'Doherty JP (2004) Reward representations and reward-related learning in the human brain: insights from neuroimaging. Current Opinion in Neurobiology 14: 769–776.
[68]
Delgado M, Nystrom L, Fissell C, Noll D, Fiez J (2000) Tracking the hemodynamic responses to reward and punishment in the striatum. Journal of Neurophysiology 84: 3072–3077.
[69]
Frank M, Seeberger L, O'Reilly R (2004) By carrot or by stick: Cognitive reinforcement learning in Parkinsonism. Science 306: 1940–1943.
[70]
Sch？nberg T, Daw ND, Joel D, O'Doherty JP (2007) Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. Journal of Neuroscience 27: 12860–12867.
[71]
Li J, Daw ND (2011) Signals in human striatum are appropriate for policy update rather than value prediction. Journal of Neuroscience 31: 5504–5511.
[72]
Ranasinghe PD, Levy BR (2007) Prevalence of and sex disparities in posttraumatic stress disorder in an internally displaced Sri Lankan population 6 months after the 2004 Tsunami. Disaster Medicine and Public Health Preparedness 1: 34–41; discussion 41–33.
[73]
Maes M, Delmeire L, Schotte C, Janca A, Creten T, et al. (1998) Epidemiologic and phenomenological aspects of post-traumatic stress disorder: DSM-III-R diagnosis and diagnostic criteria not validated. Psychiatry Research 81: 179–193.
[74]
Hall BJ, Elhai JD, Grubaugh A, Tuerk P, Magruder K (2012) Examining the factor structure of PTSD between male and female veterans in primary care. Journal of Anxiety Disorders 26: 409–415.
[75]
Iverson KM, Hendricks AM, Kimerling R, Krengel M, Meterko M, et al. (2011) Psychiatric diagnoses and neurobehavioral symptom severity among OEF/OIF VA patients with a deployment-related traumatic brain injury: A gender comparison. Womens Health Issues 21: S210–217.
[76]
Myung IJ (2003) Tutorial on maximum likelihood estimation. Journal of Mathematical Psychology 47: 90–100.
