|
无声的感知——听障人群视觉节奏加工的表现及机制
|
Abstract:
日常生活中健听者通常能够根据声音的节奏轻松完成动作同步,例如跟随音乐踏步、依据谈话节奏点头回应等。这种对听觉时间线索的依赖,凸显了声音在形成时间知觉时的基石作用。但对于听障人群而言,视觉通道则是他们感知节奏的核心途径。本文从健听者拍子同步视听通道差异出发,以听障人群的视觉节奏加工为核心,结合“功能补偿假说”与“听觉脚手架假说”,系统综述听觉剥夺条件下听障人群视觉节奏加工的行为表现及神经机制。研究表明,听障人群因长期依赖视觉而出现视觉代偿,其动态视觉信息处理能力显著增强,并伴随听觉皮层跨模态重组现象;然而,听觉脚手架的缺失导致其对复杂时间序列结构的解析能力受限。未来研究可整合计算模型与神经干预技术,探索视听代偿的动态平衡机制进一步推动跨模态代偿机制的理论完善与实践应用。
In daily life, healthy listeners can usually easily complete the movement synchronization according to the rhythm of the sound, such as following the music and nodding in response according to the rhythm of the conversation. This reliance on auditory temporal cues highlights the cornerstone role of sound in shaping temporal perception. But for the hearing-impaired people, the visual channel is the core way for them to perceive the rhythm. Starting from the differences of synchronous audio-visual channel of healthy listeners, taking the visual rhythm processing of hearing impaired people as the core, combining the “functional compensation hypothesis” and “auditory scaffolding hypothesis”, we systematically summarize the behavioral performance and neural mechanism of visual rhythm processing of hearing impaired people under the condition of auditory deprivation. The study shows that the dynamic visual information processing ability is significantly enhanced, accompanied by the cross-modal reorganization of the auditory cortex; However, the absence of auditory scaffolding leads to the limited ability to resolve the complex time series structure. Future studies can integrate computational models and neural intervention techniques, explore the dynamic balance mechanism of audio-visual compensation, and further promote the theoretical improvement and practical application of cross-modal compensation mechanism.
[1] | 耿晓丽, 姜一馨, 王悦, 周梁(2023). 探究听力障碍者视觉运动拍子同步能力的表现和发展. 心理科学进展, (31), 62. |
[2] | Auer, E. T., & Bernstein, L. E. (2007). Enhanced Visual Speech Perception in Individuals with Early-Onset Hearing Impairment. Journal of Speech, Language, and Hearing Research, 50, 1157-1165. https://doi.org/10.1044/1092-4388(2007/080) |
[3] | Bengtsson, S. L., Ullén, F., Henrik Ehrsson, H., Hashimoto, T., Kito, T., Naito, E. et al. (2009). Listening to Rhythms Activates Motor and Premotor Cortices. Cortex, 45, 62-71. https://doi.org/10.1016/j.cortex.2008.07.002 |
[4] | Chen, Y., Repp, B. H., & Patel, A. D. (2002). Spectral Decomposition of Variability in Synchronization and Continuation Tapping: Comparisons between Auditory and Visual Pacing and Feedback Conditions. Human Movement Science, 21, 515-532. https://doi.org/10.1016/s0167-9457(02)00138-0 |
[5] | Codina, C., Pascalis, O., Mody, C., Toomey, P., Rose, J., Gummer, L. et al. (2011). Visual Advantage in Deaf Adults Linked to Retinal Changes. PLOS ONE, 6, e20417. https://doi.org/10.1371/journal.pone.0020417 |
[6] | Conway, C. M., Pisoni, D. B., & Kronenberger, W. G. (2009). The Importance of Sound for Cognitive Sequencing Abilities. Current Directions in Psychological Science, 18, 275-279. https://doi.org/10.1111/j.1467-8721.2009.01651.x |
[7] | Cooper, G., & Meyer, L. B. (1960). The Rhythmic Structure of Music. University of Chicago Press. |
[8] | Gan, L., Huang, Y., Zhou, L., Qian, C., & Wu, X. (2015). Synchronization to a Bouncing Ball with a Realistic Motion Trajectory. Scientific Reports, 5, Article 11974. https://doi.org/10.1038/srep11974 |
[9] | Glenberg, A. M., & Jona, M. (1991). Temporal Coding in Rhythm Tasks Revealed by Modality Effects. Memory & Cognition, 19, 514-522. https://doi.org/10.3758/bf03199576 |
[10] | Grahn, J. A. (2012). See What I Hear? Beat Perception in Auditory and Visual Rhythms. Experimental Brain Research, 220, 51-61. https://doi.org/10.1007/s00221-012-3114-8 |
[11] | Grahn, J. A., Henry, M. J., & McAuley, J. D. (2011). FMRI Investigation of Cross-Modal Interactions in Beat Perception: Audition Primes Vision, but Not Vice Versa. NeuroImage, 54, 1231-1243. https://doi.org/10.1016/j.neuroimage.2010.09.033 |
[12] | Guttman, S. E., Gilroy, L. A., & Blake, R. (2005). Hearing What the Eyes See. Psychological Science, 16, 228-235. https://doi.org/10.1111/j.0956-7976.2005.00808.x |
[13] | Hove, M. J., & Keller, P. E. (2010). Spatiotemporal Relations and Movement Trajectories in Visuomotor Synchronization. Music Perception, 28, 15-26. https://doi.org/10.1525/mp.2010.28.1.15 |
[14] | Hove, M. J., Fairhurst, M. T., Kotz, S. A., & Keller, P. E. (2013a). Synchronizing with Auditory and Visual Rhythms: An fMRI Assessment of Modality Differences and Modality Appropriateness. NeuroImage, 67, 313-321. https://doi.org/10.1016/j.neuroimage.2012.11.032 |
[15] | Hove, M. J., Iversen, J. R., Zhang, A., & Repp, B. H. (2013b). Synchronization with Competing Visual and Auditory Rhythms: Bouncing Ball Meets Metronome. Psychological Research, 77, 388-398. https://doi.org/10.1007/s00426-012-0441-0 |
[16] | Iversen, J. R., Patel, A. D., Nicodemus, B., & Emmorey, K. (2015). Synchronization to Auditory and Visual Rhythms in Hearing and Deaf Individuals. Cognition, 134, 232-244. https://doi.org/10.1016/j.cognition.2014.10.018 |
[17] | Neville, H. J., & Lawson, D. (1987). Attention to Central and Peripheral Visual Space in a Movement Detection Task. III. Separate Effects of Auditory Deprivation and Acquisition of a Visual Language. Brain Research, 405, 284-294. https://doi.org/10.1016/0006-8993(87)90297-6 |
[18] | Patel, A. D., Iversen, J. R., Chen, Y., & Repp, B. H. (2005). The Influence of Metricality and Modality on Synchronization with a Beat. Experimental Brain Research, 163, 226-238. https://doi.org/10.1007/s00221-004-2159-8 |
[19] | Pollok, B., Krause, V., Butz, M., & Schnitzler, A. (2009). Modality Specific Functional Interaction in Sensorimotor Synchronization. Human Brain Mapping, 30, 1783-1790. https://doi.org/10.1002/hbm.20762 |
[20] | Proksch, J., & Bavelier, D. (2002). Changes in the Spatial Distribution of Visual Attention after Early Deafness. Journal of Cognitive Neuroscience, 14, 687-701. https://doi.org/10.1162/08989290260138591 |
[21] | Repp, B. H., & Penel, A. (2002). Auditory Dominance in Temporal Processing: New Evidence from Synchronization with Simultaneous Visual and Auditory Sequences. Journal of Experimental Psychology: Human Perception and Performance, 28, 1085-1099. https://doi.org/10.1037/0096-1523.28.5.1085 |
[22] | Stauffer, C. C., Haldemann, J., Troche, S. J, & Rammsayer, T. H. (2012). Auditory and Visual Temporal Sensitivity: Evidence for a Hierarchical Structure of Modality-Specific and Modality-Independent Levels of Temporal Information Processing. Psychological Research, 76, 20-31. https://doi.org/10.1007/s00426-011-0333-8 |
[23] | Su, Y. (2014). Audiovisual Beat Induction in Complex Auditory Rhythms: Point-Light Figure Movement as an Effective Visual Beat. Acta Psychologica, 151, 40-50. https://doi.org/10.1016/j.actpsy.2014.05.016 |
[24] | Thaut, M., Trimarchi, P., & Parsons, L. (2014). Human Brain Basis of Musical Rhythm Perception: Common and Distinct Neural Substrates for Meter, Tempo, and Pattern. Brain Sciences, 4, 428-452. https://doi.org/10.3390/brainsci4020428 |
[25] | Zhou, L., Xing, L., Zheng, C., & Li, S. (2024). Moving Stimuli Enhance Beat Timing and Sensorimotor Coupling in Vision. Journal of Experimental Psychology: Human Perception and Performance, 50, 416-429. https://doi.org/10.1037/xhp0001193 |