全部 标题 作者
关键词 摘要

OALib Journal期刊
ISSN: 2333-9721
费用:99美元

查看量下载量

相关文章

更多...
PLOS ONE  2011 

On the Time Course of Vocal Emotion Recognition

DOI: 10.1371/journal.pone.0027256

Full-Text   Cite this paper   Add to My Lib

Abstract:

How quickly do listeners recognize emotions from a speaker's voice, and does the time course for recognition vary by emotion type? To address these questions, we adapted the auditory gating paradigm to estimate how much vocal information is needed for listeners to categorize five basic emotions (anger, disgust, fear, sadness, happiness) and neutral utterances produced by male and female speakers of English. Semantically-anomalous pseudo-utterances (e.g., The rivix jolled the silling) conveying each emotion were divided into seven gate intervals according to the number of syllables that listeners heard from sentence onset. Participants (n = 48) judged the emotional meaning of stimuli presented at each gate duration interval, in a successive, blocked presentation format. Analyses looked at how recognition of each emotion evolves as an utterance unfolds and estimated the “identification point” for each emotion. Results showed that anger, sadness, fear, and neutral expressions are recognized more accurately at short gate intervals than happiness, and particularly disgust; however, as speech unfolds, recognition of happiness improves significantly towards the end of the utterance (and fear is recognized more accurately than other emotions). When the gate associated with the emotion identification point of each stimulus was calculated, data indicated that fear (M = 517 ms), sadness (M = 576 ms), and neutral (M = 510 ms) expressions were identified from shorter acoustic events than the other emotions. These data reveal differences in the underlying time course for conscious recognition of basic emotions from vocal expressions, which should be accounted for in studies of emotional speech processing.

References

[1]  Friederici A, Alter K (2004) Lateralization of auditory language functions: a dynamic dual pathway model. Brain and Language 89: 267–276.
[2]  Pell MD (2006) Cerebral mechanisms for understanding emotional prosody in speech. Brain and Language 96: 221–234.
[3]  Belin P, Fecteau S, Bedard C (2004) Thinking the voice: neural correlates of voice perception. Trends in Cognitive Sciences 8: 129–135.
[4]  Spreckelmeyer KN, Kutas M, Urbach T, Altenmuller E, Munte TF (2009) Neural processing of vocal emotion and identity. Brain and Cognition 69: 121–126.
[5]  Etcoff NL, Magee JL (1992) Categorical perception of facial expressions. Cognition 44: 227–240.
[6]  Laukka P (2005) Categorical perception of vocal emotion expressions. Emotion 5: 277–295.
[7]  Pell MD (2005) Nonverbal emotion priming: evidence from the ‘facial affect decision task’. Journal of Nonverbal Behavior 29: 45–73.
[8]  Cowie R, Cornelius RR (2003) Describing the emotional states that are expressed in speech. Speech Communication 40: 5–32.
[9]  Ekman P (1994) Strong evidence for universals in facial expressions: a reply to Russell's mistaken critique. Psychological Bulletin 115: 268–287.
[10]  Izard CE (1994) Innate and universal facial expressions: evidence from developmental and cross-cultural research. Psychological Bulletin 115: 288–299.
[11]  Oatley K, Johnson-Laird PN (1987) Towards a cognitive theory of emotions. Cognition and Emotion 1: 29–50.
[12]  Pell MD, Monetta L, Paulmann S, Kotz SA (2009) Recognizing emotions in a foreign language. Journal of Nonverbal Behavior 33: 107–120.
[13]  Scherer KR, Banse R, Wallbott H (2001) Emotion inferences from vocal expression correlate across languages and cultures. Journal of Cross-Cultural Psychology 32: 76–92.
[14]  Thompson W, Balkwill L-L (2006) Decoding speech prosody in five languages. Semiotica 158: 407–424.
[15]  Van Bezooijen R, Otto S, Heenan T (1983) Recognition of vocal expressions of emotion: a three-nation study to identify universal characteristics. Journal of Cross-Cultural Psychology 14: 387–406.
[16]  Pell MD, Paulmann S, Dara C, Alasseri A, Kotz SA (2009) Factors in the recognition of vocally expressed emotions: a comparison of four languages. Journal of Phonetics 37: 417–435.
[17]  Schirmer A, Kotz S (2006) Beyond the right hemisphere: brain mechanisms mediating vocal emotional processing. TRENDS in Cognitive Sciences 10: 24–30.
[18]  Bower GH (1981) Mood and memory. American Psychologist 36: 129–148.
[19]  Bower GH (1987) Commentary on mood and memory. Behavior Research and Therapy 25: 443–455.
[20]  Niedenthal P, Setterlund M, Jones D (1994) Emotional organization of perceptual memory. In: Niedenthal P, Kitayama S, editors. The heart's eye: emotional influences in perception and attention. New York: Academic Press. pp. 87–113.
[21]  Brosch T, Grandjean D, Sander D, Scherer KR (2008) Cross-modal emotional attention: Emotional vocies modulate early stages of visual processing. Journal of Cognitive Neuroscience 21: 1670–1679.
[22]  Kotz SA, Paulmann S (2007) When emotional prosody and semantics dance cheek to cheek: ERP evidence. Brain Research 115: 107–118.
[23]  Paulmann S, Titone D, Pell MD (In Press) How emotional prosody guides your way: evidence from eye movements. Speech Communication. doi:10.1016/j.specom.2011.07.004.
[24]  Vroomen J, Driver J, de Gelder B (2001) Is cross-modal integration of emotional expressions independent of attentional resources. Cognitive, Affective, & Behavioral Neuroscience 1: 382–387.
[25]  Scherer KR (1986) Vocal affect expression: A review and a model for future research. Psychological Bulletin 99: 143–165.
[26]  Banse R, Scherer KR (1996) Acoustic profiles in vocal emotion expression. Journal of Personality and Social Psychology 70: 614–636.
[27]  Pell MD (2001) Influence of emotion and focus location on prosody in matched statements and questions. Journal of the Acoustical Society of America 109: 1668–1680.
[28]  Sobin C, Alpert M (1999) Emotion in speech: the acoustic attributes of fear, anger, sadness, and joy. Journal of Psycholinguistic Research 23: 347–365.
[29]  Juslin P, Laukka P (2003) Communication of emotions in vocal expression and music performance: different channels, same code? Psychological Bulletin 129: 770–814.
[30]  Bachorowski J (1999) Vocal expression and perception of emotion. Current Directions in Psychological Science 8: 53–57.
[31]  Mozziconacci SJL (2001) Modeling emotion and attitude in speech by means of perceptually based parameter values. User Modeling and User-Adapted Interaction 11: 297–326.
[32]  Hammerschmidt K, Jurgens U (2007) Acoustical correlates of affective prosody. Journal of Voice 21: 531–540.
[33]  Pell MD, Jaywant A, Monetta L, Kotz SA (2011) Emotional speech processing: disentangling the effects of prosody and semantic cues. Cognition and Emotion 25: 834–853.
[34]  Pell MD, Skorup V (2008) Implicit processing of emotional prosody in a foreign versus native language. Speech Communication 50: 519–530.
[35]  Cosmides L (1983) Invariances in the acoustic expression of emotion during speech. Journal of Experimental Psychology: Human Perception and Performance 9: 864–881.
[36]  Grandjean D, Sander D, Pourtois G, Schwartz S, Seghier M, et al. (2005) The voices of wrath: brain responses to angry prosody in meaningless speech. Nature Neuroscience 8: 145–156.
[37]  Ohala J (1983) Cross-language use of pitch: an ethological view. Phoetica 40: 1–18.
[38]  Paulmann S, Kotz SA (2008) Early emotional prosody perception based on different speaker voices. NeuroReport 19: 209–213.
[39]  Paulmann S, Pell MD (2010) Contextual influences of emotional speech prosody on face processing: how much is enough? Cognitive, Affective, and Behavioral Neuroscience 10: 230–242.
[40]  Bostanov V, Kotchoubey B (2004) Recognition of affective prosody: continuous wavelet measures of event-related brain potentials to emotional exclamations. Psychophysiology 41: 259–268.
[41]  Paulmann S, Jessen S, Kotz SA (2009) Investigating the multimodal nature of human communication. Journal of Psychophysiology 23: 63–76.
[42]  Pell MD (2005) Prosody-face interactions in emotional processing as revealed by the facial affect decision task. Journal of Nonverbal Behavior 29: 193–215.
[43]  Johnson WF, Emde RN, Scherer KR, Klinnert MD (1986) Recognition of emotion from vocal cues. Archives of General Psychiatry 43: 280–283.
[44]  Kramer E (1964) Elimination of verbal cues in judgments of emotion from voice. Journal of Abnormal and Social Psychology 68: 390–396.
[45]  Scherer KR, Banse R, Wallbott HG, Goldbeck T (1991) Vocal cues in emotion encoding and decoding. Motivation and Emotion 15: 123–148.
[46]  Greasley P, Sherrard C, Waterman M (2000) Emotion in language and speech: methodological issues in naturalistic approaches. Language and Speech 43: 355–375.
[47]  Russell JA (1994) Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. Psychological Bulletin 115: 102–141.
[48]  Murray IR, Arnott JL (1993) Toward the simulation of emotion in synthetic speech: A review of the literature on human vocal emotion. Journal of the Acoustical Society of America 93: 1097–1108.
[49]  Grosjean F (1980) Spoken word recognition processes and the gating paradigm. Perception & Psychophysics 28: 267–283.
[50]  Grosjean F (1985) The recognition of words after their acoustic offset: Evidence and implications. Perception & Psychophysics 38: 299–310.
[51]  Salasoo A, Pisoni D (1985) Interaction of knowledge sources in spoken word identification. Journal of Memory and Language 24: 210–231.
[52]  Warren P, Marslen-Wilson W (1987) Continuous uptake of acoustic cues in spoken word recognition. Perception & Psychophysics 41: 262–275.
[53]  Vieillard S, Peretz I, Gosselin N, Khalfa S (2008) Happy, sad, scary and peaceful musical excerpts for research on emotions. Cognition and Emotion 22: 720–752.
[54]  Audibert N, Auberge V, Rilliard A (2007) When is the emotional information? pp. 2137–2140. A gating experiment for gradient and contours cues. Proceedings of ICPhS XVI Meeting. Saarbrucken, 6–10 August 2007.
[55]  Cornew L, Carver L, Love T (2010) There's more to emotion than meets the eye: A processing bias for neutral content in the domain of emotional prosody. Cognition and Emotion 24: 1133–1152.
[56]  Grosjean F (1996) Gating. Language and Cognitive Processes 11: 597–604.
[57]  Tyler L, Wessels J (1985) Is gating an on-line task? Evidence from naming latency data. Perception & Psychophysics 38: 217–222.
[58]  Pollack I, Rubenstein H, Horowitz A (1960) Communication of verbal modes of expression. Language and Speech 3: 121–130.
[59]  Poeppel D, Idsardi WJ, van Wassenhove V (2008) Speech perception at the interface of neurobiology and linguistics. Philosophical Transactions of the Royal Society B: Biological Sciences 363: 1071–1086.
[60]  Borod JC, Welkowitz J, Alpert M, Brozgold AZ, Martin C, et al. (1990) Parameters of emotional processing in neuropsychiatric disorders: Conceptual issues and a battery of tests. Journal of Communication Disorders 23: 247–271.
[61]  Ekman P, Sorenson ER, Friesen WV (1969) Pan-cultural elements in facial displays of emotion. Science 164: 86–88.
[62]  Ekman P (1992) An argument for basic emotions. Cognition and Emotion 6: 169–200.
[63]  Williams CE, Stevens KN (1981) Vocal correlates of emotional states. In: Darby JK, editor. Speech Evaluation in Psychiatry. New York: Grune & Stratton. pp. 221–240.
[64]  Wagner HL (1993) On measuring performance in category judgment studies of nonverbal behavior. Journal of Nonverbal Behavior 17: 3–28.
[65]  Levitt EA (1964) The relationship between abilities to express emotional meanings vocally and facially. In: Davitz JR, editor. The communication of emotional meanings. New York: McGraw-Hill. pp. 87–100.
[66]  Zuckerman M, Lipets M, Hall Koivumaki J, Rosenthal R (1975) Encoding and decoding nonverbal cues of emotion. Journal of Personality and Social Psychology 32: 1068–1076.
[67]  Paulmann S, Pell MD (2011) Is there an advantage for recognizing multi-modal emotional stimuli? Motivation and Emotion 35: 192–201.
[68]  Juslin P, Laukka P (2001) Impact of intended emotion intensity on cue utilization and decoding accuracy in vocal expression of emotion. Emotion 1: 381–412.
[69]  Simon-Thomas E, Keltner D, Sauter D, Sinicropi-Yao L, Abramson A (2009) The voice conveys specific emotions: evidence from vocal burst displays. Emotion 9: 838–846.
[70]  Hawk ST, van Kleef GA, Fischer AH, van der Schalk J (2009) “Worth a thousand words”: Absolute and relative decoding of nonlinguistic affect vocalizations. Emotion 9: 293–305.
[71]  Palermo R, Coltheart M (2004) Photographs of facial expression: Accuracy, response times, and ratings of intensity. Behavior Research Methods, Instruments, and Computers 36: 634–638.
[72]  Tracy JL, Robins RW (2008) The Automaticity of Emotion Recognition. Emotion 8: 81–95.
[73]  Ekman P, Friesen W, O'Sullivan M, Chan A, Diacoyanni-Tarlatzis I, et al. (1987) Universals and cultural differences in the judgments of facial expressions of emotion. Journal of Personality and Social Psychology 53: 712–717.
[74]  Wildgruber D, Riecker A, Hertrich I, Erb M, Grodd W, et al. (2005) Identification of emotional intonation evaluated by fMRI. NeuroImage 24: 1233–1241.
[75]  Kotz S, Meyer M, Alter K, Besson M, von Cramon Y, et al. (2003) On the lateralization of emotional prosody: an event-related functional MR investigation. Brain and Language 86: 366–376.
[76]  Kreifelts B, Ethofer T, Schiozawa T, Grodd W, Wildgruber D (2009) Cerebral representation of non-verbal emotional perception: fMRI reveals audiovisual integration area between voice- and face- sensitive regions in the superior temporal sulcus. Neuropsychologia 47: 3059–3066.
[77]  Schirmer A, Zysset S, Kotz S, von Cramon DY (2004) Gender differences in the activation of inferior frontal cortex during emotional speech perception. NeuroImage 21: 1114–1123.
[78]  Matsumoto D, Assar M (1992) The effects of language on judgments of universal facial expressions of emotion. Journal of Nonverbal Behavior 16: 85–99.
[79]  Scherer KR (2009) Emotions are emergent processes: they require a dynamic computational architecture. Philosophical Transactions of the Royal Society B 364: 3459–3474.
[80]  Ashley V, Vuilleumier P, Swick D (2004) Time-course and specificity of event-related potentials to emotional expressions. NeuroReport 15: 211–215.
[81]  Eimer M, Holmes A, McGlone FP (2003) The role of spatial attention in the processing of facial expression: an ERP study of rapid brain responses to six basic emotions. Cognitive, Affective, and Behavioral Neuroscience 3: 97–110.
[82]  Paulmann S, Pell MD (2009) Facial expression decoding as a function of emotional meaning status: ERP evidene. NeuroReport 20: 1603–1608.
[83]  Eimer M (2000) Event-related brain potentials distinguish processing stages involved in face perception and recognition. Clinical Neurophysiology 111: 694–705.
[84]  Paulmann S, Pell MD (2010) Contextual influences of emotional speech prosody on face processing: How much is enough? Cognitive, Affective and Behavioral Neuroscience 10: 230–242.
[85]  Ohman A (1987) The psychophysiology of emotion: An evolutionary-cognitive perspective. Advances in Psychophysiology 2: 79–127.
[86]  Pratto F, John OP (1991) Automatic vigilance: The attention-grabbing power of negative social information. Journal of Personality and Social Psychology 61: 380–391.
[87]  Calder AJ, Keane J, Lawrence AD, Manes F (2004) Impaired recognition of anger following damage to the ventral striatum. Brain 127: 1958–1969.
[88]  Calder AJ, Lawrence AD, Young AW (2001) Neuropsychology of fear and loathing. Nature reviews Neuroscience 2: 352–363.
[89]  Phillips ML, Young AW, Scott SK, Calder AJ, Andrew C, et al. (1998) Neural responses to facial and vocal expressions of fear and disgust. Proceedings of Royal Society 265: 1809–1817.
[90]  Leppanen J, Hietanen J (2004) Positive facial expressions are recognized faster than negative facial expressions, but why? Psychological Research 69: 22–29.
[91]  Frick R (1986) The prosodic expression of anger: differentiating threat and frustration. Aggressive Behaviour 12: 121–128.
[92]  Sauter D, Scott SK (2007) More than one kind of happiness: can we recognize vocal expressions of different positive states? Motivation and Emotion 31: 192–199.
[93]  Charash M, McKay D (2002) Attention bias for disgust. Anxiety Disorders 16: 529–541.
[94]  Cisler J, Olatunji B, Lohr J, Williams N (2009) Attentional bias differences between fear and disgust: implications for the role of disgust in disgust-related anxiety disorders. Cognition and Emotion 23: 675–687.
[95]  de Gelder B, Vroomen J (2000) The perception of emotions by ear and by eye. Cognition and Emotion 14: 289–311.
[96]  Wurm LH, Vakoch DA, Strasser MR, Calin-Jageman R, Ross SE (2001) Speech perception and vocal expression of emotion. Cognition and Emotion 15: 831–852.
[97]  Jaywant A, Pell MD (2012) Categorical processing of negative emotions from speech prosody. Speech Communication 54: 1–10.
[98]  Bach DR, Grandjean D, Sander D, Herdener M, Strik WK, et al. (2008) The effect of appraisal level on processing of emotional prosody in meaningless speech. NeuroImage 42: 919–927.
[99]  Ethofer T, Van De Ville D, Scherer K, Vuilleumier P (2009) Decoding of emotional information in voice-sensitive cortices. Current Biology 19: 1028–1033.
[100]  Hot P, Saito Y, Mandai O, Kobayashi T, Sequeira H (2006) An ERP investigation of emotional processing in European and Japanese individuals. Brain Research 1122: 171–178.
[101]  Ishii K, Reyes JA, Kitayama S (2003) Spontaneous attention to word content versus emotional tone: differences among three cultures. Psychological Science 14: 39–46.

Full-Text

Contact Us

service@oalib.com

QQ:3279437679

WhatsApp +8615387084133