The Effect of Facial Expressions on the Process of Learning, and Memory of Face Images among Healthy Participants and Individuals with Traumatic Brain Injury (TBI): Examination Using Eye Movements
Objective: Context-Dependent Effect (CDE) is a process by which restoring at test the original learning context enhances memory. We examined the CDE of facial expressions through behavioral and eye movement measures. Participants and Method: Twenty-three healthy individuals and 23 patients with moderate-to-severe TBI individuals participated in the current study. Participants were exposed to pictures of faces with neutral expression and were asked to remember them for a subsequent memory test. In the testing session, they were asked to determine whether or not the faces presented to them had appeared before, under two conditions: 1) where the context remains constant (facial expression remained neutral—the Repeat condition); 2) where the context changes (facial expression changed to angry or happy expression— the Re-pair condition). Results: While memory of the individuals with TBI was poorer than that of the control group, both groups exhibited CDE, that is consistency of facial expressions between study and test facilitated face recognition. In both groups, there is a relationship between Dwell Time on the faces and face recognition. This effect appeared in the study and test phases. Conclusions: This research supports previous studies showing evidence for CDE in the TBI group and extends our comprehension of the relationship between eye movements, memory, and context of facial expression.
References
[1]
Barbas, H., & Blatt, G. J. (1995). Topographically Specific Hippocampal Projections Target Functionally Distinct Prefrontal Areas in the Rhesus Monkey. Hippocampus, 5, 511-533. https://doi.org/10.1002/hipo.450050604
[2]
Barton, J. J. S., Radcliffe, N., Cherkasova, M. V., Edelman, J., & Intriligator, J. M. (2006). Information Processing during Face Recognition: The Effects of Familiarity, Inversion, and Morphing on Scanning Fixations. Perception, 35, 1089-1105. https://doi.org/10.1068/p5547
[3]
Bornhofen, C., & Mcdonald, S. (2008). Emotion Perception Deficits Following Traumatic Brain Injury: A Review of the Evidence and Rationale for Intervention. Journal of the International Neuropsychological Society, 14, 511-525. https://doi.org/10.1017/S1355617708080703
[4]
Bruce, V., & Young, A. (1986). Understanding Face Recognition. British Journal of Psychology, 77, 305-327. https://doi.org/10.1111/j.2044-8295.1986.tb02199.x
[5]
Cabeza, R., Locantore, J. K., & Anderson, N. D. (2003). Lateralization of Prefrontal Activity during Episodic Memory Retrieval: Evidence for the Production-Monitoring Hypothesis. Journal of Cognitive Neuroscience, 15, 249-259. https://doi.org/10.1162/089892903321208187
[6]
Chanon, V. W., & Hopfinger, J. B. (2008). Memory’s Grip on Attention: The Influence of Item Memory on the Allocation of Attention. Visual Cognition, 16, 325-340. https://doi.org/10.1080/13506280701459026
[7]
Chen, H. C., & Lee, Y. S. (2015). The Eye Movement Measure of Memory and Its Relationship with Explicit Measures. Consciousness and Cognition, 33, 354-363. https://doi.org/10.1016/j.concog.2015.02.003
[8]
Coad, B. M., Postans, M., Hodgetts, C. J., Muhlert, N., Graham, K. S., Lawrence, A. D. (2020). Structural Connections Support Emotional Connections: Uncinate Fasciculus Microstructure Is Related to the Ability to Decode Facial Emotion Expressions. Neuropsychologia, 145, Article ID: 106562. https://doi.org/10.1016/j.neuropsychologia.2017.11.006
[9]
Davachi, L., & DuBrow, S. (2015). How the Hippocampus Preserves Order: The Role of Prediction and Context. Trends in Cognitive Sciences, 19, 92-99. https://doi.org/10.1016/j.tics.2014.12.004
[10]
Deitcher, Y., Sachar, Y., & Vakil, E. (2020). Effect of Eye Movement Reactivation on Visual Memory among Individuals with Moderate-to-Severe Traumatic Brain Injury (TBI). Journal of Clinical and Experimental Neuropsychology, 42, 208-221. https://doi.org/10.1080/13803395.2019.1704223
[11]
Diana, R. A., Yonelinas, A. P., & Ranganath, C. (2007). Imaging Recollection and Familiarity in the Medial Temporal Lobe: A Three-Component Model. Trends in Cognitive Sciences, 11, 379-386. https://doi.org/10.1016/j.tics.2007.08.001
[12]
George, M. S., Ketter, T. A., Gill, D. S., Haxby, J. V., Ungerleider, L. G., Herscovitch, P., & Post, R. M. (1993). Brain Regions Involved in Recognizing Facial Emotion or Identity: An Oxygen-15 PET Study. The Journal of Neuropsychiatry and Clinical Neurosciences, 5, 384-394. https://doi.org/10.1176/jnp.5.4.384
[13]
Hannula, D. E., Althoff, R. R., Warren, D. E., Riggs, L., Cohen, N. J., & Ryan, J. D. (2010). Worth a Glance: Using Eye Movements to Investigate the Cognitive Neuroscience of Memory. Frontiers in Human Neuroscience, 4, Article 166. https://doi.org/10.3389/fnhum.2010.00166
[14]
Hannula, D. E., Ryan, J. D., Tranel, D. & Cohen, N. J. (2007). Rapid Onset Relational Memory Effects Are Evident in Eye Movement Behavior, but Not in Hippocampal Amnesia. Journal of Cognitive Neuroscience, 19, 1690-1705. https://doi.org/10.1162/jocn.2007.19.10.1690
[15]
Haxby, J. V., Hoffman, E. A., Gobbini, M. I. et al. (2000). The Distributed Human Neural System for Face Perception. Trends in Cognitive Sciences, 4, 223-233. https://doi.org/10.1016/S1364-6613(00)01482-0
[16]
Hayes, S. M., Ryan, L., Schnyer, D. M., & Nadel, L. (2004). An fMRI Study of Episodic Memory: Retrieval of Object, Spatial, and Temporal Information. Behavioral Neuroscience, 118, 885-896. https://doi.org/10.1037/0735-7044.118.5.885
[17]
Heisz, J. J., & Shore, D. I. (2008). More Efficient Scanning for Familiar Faces. Journal of Vision, 8, 1-10. https://doi.org/10.1167/8.1.9
[18]
Heisz, J. J., Pottruff, M. M., & Shore, D. I. (2013). Females Scan More than Males: A Potential Mechanism for Sex Differences in Recognition Memory. Psychological Science, 24, 1157-1163. https://doi.org/10.1177/0956797612468281
[19]
Holm, L., Eriksson, J., & Andersson, L. (2008). Looking As If You Know: Systematic Object Inspection Precedes Object Recognition. Journal of Vision, 8, 1-7. https://doi.org/10.1167/8.4.14
[20]
Hsiao, J. H., & Cottrell, G. (2008). Two Fixations Suffice in Face Recognition. Psychological Science, 19, 998-1006. https://doi.org/10.1111/j.1467-9280.2008.02191.x
[21]
Kafkas, A., & Montaldi, D. (2011). Recognition Memory Strength is Predicted by Pupillary Responses at Encoding While Fixation Patterns Distinguish Recollection from Familiarity. The Quarterly Journal of Experimental Psychology, 64, 1971-1989. https://doi.org/10.1080/17470218.2011.588335
[22]
Langner, O., Dotsch, R., Bijlstra, G., Wigboldus, D. H. J., Hawk, S. T., & van Knippenberg, A. (2010). Presentation and Validation of the Radboud Faces Database. Cognition & Emotion, 24, 1377-1388. https://doi.org/10.1080/02699930903485076
[23]
Lundqvist, D., Flykt, A., & Öhman, A. (1998). The Karolinska Directed Emotional Faces KDEF. CD ROM from Department of Clinical Neuroscience, Psychology Section, Karolinska Institutet.
[24]
Mancuso, M., Magnani, N., Cantagallo, A., Rossi, G., Capitani, D., Galletti, V., Cardamone, G., & Robertson, I. H. (2015). Emotion Recognition Impairment in Traumatic Brain Injury Compared with Schizophrenia Spectrum: Similar Deficits with Different Origins. The Journal of Nervous and Mental Disease, 203, 87-95. https://doi.org/10.1097/NMD.0000000000000245
[25]
McDonald, S., & Saunders. J. C. (2005). Differential Impairment in Recognition of Emotion across Different Media in People with Severe Traumatic Brain Injury. Journal of the International Neuropsychology Society, 11, 392-399. https://doi.org/10.1017/S1355617705050447
[26]
Metoki, A., Alm, K. H., Wang, Y., Ngo, C. T., & Olson, I. R. (2017). Never Forget a Name: White Matter Connectivity Predicts Person Memory. Brain Structure and Function, 222, 4187-4201. https://doi.org/10.1007/s00429-017-1458-3
[27]
Peterson, M. S., & Kramer, A. F. (2001). Attentional Guidance of the Eyes by Contextual Information and Abrupt Onsets. Perception and Psychophysics, 63, 1239-1249. https://doi.org/10.3758/BF03194537
[28]
Rolls, E. T. (2018). The Storage and Recall of Memories in the Hippocampo-Cortical System. Cell Tissue Research, 373, 577-604. https://doi.org/10.1007/s00441-017-2744-3
[29]
Rolls, E. T. (2019). The Orbitofrontal Cortex and Emotion in Health and Disease, Including Depression. Neuropsychologia, 128, 14-43. https://doi.org/10.1016/j.neuropsychologia.2017.09.021
[30]
Rosenberg, H., McDonald, S., Dethier, M., Kessels, R. P. C., & Westbrook, R. F. (2014). Facial Emotion Recognition Deficits following Moderate-Severe Traumatic Brain Injury (TBI): Re-Examining the Valence Effect and the Role of Emotion Intensity. Journal of the International Neuropsychological Society, 20, 994-1003. https://doi.org/10.1017/S1355617714000940
[31]
Ryan, J. D., Hannula, D. E., & Cohen, N. J. (2007). The Obligatory Effects of Memory on Eye Movements. Memory, 15, 508-525. https://doi.org/10.1080/09658210701391022
[32]
Schacter, D. L. (1987). Implicit Memory: History and Current Status. Journal of Experimental Psychology: Learning, Memory and Cognition, 13, 501-518. https://doi.org/10.1037/0278-7393.13.3.501
[33]
Schmitter-Edgecombe, M., & Seelye, A. M. (2012). Recovery of Content and Temporal Order Memory for Performed Activities Following Moderate to Severe Traumatic Brain Injury. Journal of Clinical and Experimental Neuropsychology, 34, 256-268. https://doi.org/10.1080/13803395.2011.633497
[34]
Snodgrass, J. G., & Corwin, J. (1988). Pragmatics of Measuring Recognition Memory: Applications to Dementia and Amnesia. Journal of Experimental Psychology: General, 117, 34-50. https://doi.org/10.1037/0096-3445.117.1.34
[35]
Snow, J., Ingeholm, J., Levy, I., Caravella, R., Case, L., Wallace, G., & Martin, A. (2011). Impaired Visual Scanning and Memory for Faces in High-Functioning Autism Spectrum Disorders: It’s Not Just Eyes. Journal of the International Neuropsychological Society, 17, 1021-1029. https://doi.org/10.1017/S1355617711000981
[36]
Suh, M., Basu, S., Kolster, R., Sarkar, R., McCandliss, B., & Ghajar, J. (2006). Increased Oculomotor Deficits during Target Blanking as an Indicator of Mild Traumatic Brain Injury. Neuroscience Letters, 410, 203-207. https://doi.org/10.1016/j.neulet.2006.10.001
[37]
Trojano, L., Moretta, P., & Estraneo, A. (2009). Communicating Using the Eyes without Remembering It: Cognitive Rehabilitation in a Severely Brain-Injured Patient with Amnesia, Tetraplegia and Anarthria. Journal of Rehabilitation Medicine, 41, 393-396. https://doi.org/10.2340/16501977-0344
[38]
Vakil, E. (2005). The Effect of Moderate to Severe Traumatic Brain Injury (TBI) on Different Aspects of Memory: A Selective Review. Journal of Clinical and Experimental Neuropsychology, 27, 977-1021. https://doi.org/10.1080/13803390490919245
[39]
Vakil, E., & Tweedy, J. R. (1994). Memory for Temporal Order and Spatial Position Information: Tests of the automatic-Effortful Distinction. Neuropsychiatry, Neuropsychology, and Behavioral Neurology, 7, 281-288.
[40]
Vakil, E., Aviv, O., Mishael, M., Schwizer Ashkenazi, S., & Sacher, Y. (2019b). Direct and Indirect Measures of Context in Patients with Mild-to-Severe Traumatic Brain Injury (TBI): The Additive Contribution of Eye Tracking. Journal of Clinical and Experimental Neuropsychology, 6, 644-652. https://doi.org/10.1080/13803395.2019.1604946
[41]
Vakil, E., Blachstein, H., & Hoofien, D. (1991). Automatic Temporal Order Judgment: The Effect of Intentionality of Retrieval on Closed-Head-Injured Patients. Journal of Clinical and Experimental Neuropsychology, 13, 291-298. https://doi.org/10.1080/01688639108401044
[42]
Vakil, E., Jaffe, R., Eluze, S., Groswasser, Z., & Aberbuck, S. (1996). Word Recall versus Reading Speed: Evidence of Preserved Priming in Head-Injured Patients. Brain and Cognition, 31, 75-89. https://doi.org/10.1006/brcg.1996.0026
[43]
Vakil, E., McDonald, S., Allen, S. K., & Vardi-Shapiro, N. (2019a). Facial Expressions Yielding Context-Dependent Effect: The Additive Contribution of Eye Movements. Acta Psychologica, 192, 138-145. https://doi.org/10.1016/j.actpsy.2018.11.008
[44]
Vakil, E., Raz, T., & Levy, D. A. (2007). Multifactorial Context Effects on Visual Recognition Memory. Quarterly Journal of Experimental Psychology, 60, 916-923. https://doi.org/10.1080/17470210701357568
[45]
Vakil, E., Sherf, R., Hoffman, M., & Stern, M. (1998). Direct and Indirect Memory Measures of Temporal Order and Spatial Location: Control versus Closed-Head Injury Participants. Neuropsychiatry, Neuropsychology, and Behavioral Neurology, 11, 212-217. https://doi.org/10.1037/0894-4105.11.4.545
[46]
Vuilleumier, P., & Pourtois, G. (2017). Distributed and Interactive Brain Mechanisms during Emotion Face Perception: Evidence from Functional Neuroimaging. Neuropsychologia, 45, 174-194. https://doi.org/10.1016/j.neuropsychologia.2006.06.003
[47]
Wechsler, D. A. (1981). Wechsler Adult Intelligence Scale—Revised Manual. The Psychological Corporation.
[48]
Winston, J. S., Henson, R. N. A., Fine-Goulden, M. R., & Dolan, R. J. (2004). fMRI-Adaptation Reveals Dissociable Neural Representations of Identity and Expression in Face Perception. Journal of Neurophysiology, 92, 1830-1839. https://doi.org/10.1152/jn.00155.2004
[49]
Wright, M. J., Wong, A. L., Obermeit, L. C., Woo, E., Schmitter-Edgecombe, M., & Fuster. J. M. (2014). Memory for Performed and Observed Activities Following Traumatic Brain Injury. Journal of Clinical and Experimental Neuropsychology, 36, 268-277. https://doi.org/10.1080/13803395.2014.884543
[50]
Yankouskaya, A., Booth, D. A., & Humphreys, G. (2012). Interactions between Facial Emotion and Identity in Face Processing: Evidence Based on Redundancy Gains. Attention, Perception, & Psychophysics, 74, 1692-1711. https://doi.org/10.3758/s13414-012-0345-5
[51]
Zwickel, J., & Müller, H. J. (2010). Observing Fearful Faces Leads to Visuo-Spatial Perspective Taking. Cognition, 117, 101-105. https://doi.org/10.1016/j.cognition.2010.07.004
