A visual-to-auditory sensory substitution device initially developed for the blind is known to allow visual-like perception through sequential exploratory strategies. Here we used functional magnetic resonance imaging (fMRI) to test whether processing the location versus the orientation of simple (elementary) “visual” stimuli encoded into sounds using the device modulates the brain activity within the dorsal visual stream in the absence of sequential exploration of these stimuli. Location and orientation detection with the device induced a similar recruitment of frontoparietal brain areas in blindfolded sighted subjects as the corresponding tasks using the same stimuli in the same subjects in vision. We observed a similar preference of the right superior parietal lobule for spatial localization over orientation processing in both sensory modalities. This provides evidence that the parietal cortex activation during the use of the prosthesis is task related and further indicates the multisensory recruitment of the dorsal visual pathway in spatial processing. 1. Introduction When deprived of its natural input, the “visual” cortex of early-blind subjects is widely activated during a variety of perceptual and cognitive tasks and its cross-modal recruitment may account for the improved perceptual abilities of these subjects in the auditory and tactile modalities. Sensory substitution prostheses translating vision into touch or vision into audition were developed initially for the blind since these devices could take advantage of the cross-modal plasticity changes induced by blindness. It should be noted, however, that cross-modal interactions are increasingly being recognized as playing a vital role in normal perception too. The visual cortex is functionally divided into a ventral/identification and a dorsal/localization stream [1–4]. Interestingly, using a sensory substitution device (SSD) often activates brain areas within both the dorsal and the ventral visual pathways, not only in the blind but also in normally sighted subjects [5–11]. Despite the increasing number of studies on sensory substitution, little is known about the specific versus nonspecific nature of these brain activations as regards to the task performed with the SSD. One may hypothesize that the brain activations elicited during the use of an SSD are partly driven by mechanisms specific to sensory substitution (i.e., the conversion of sounds or tactile sensations into “visual” representations) but also depend on the nature of the task performed with the SSD (e.g., stimulus identification
References
[1]
S. M. Zeki, “Functional specialisation in the visual cortex of the rhesus monkey,” Nature, vol. 274, no. 5670, pp. 423–428, 1978.
[2]
L. G. Ungerleider and M. Mishkin, “Two cortical visual systems,” in Analysis of Visual Behavior, D. J. Ingle, M. A. Goodale, and R. J. W. Mansfield, Eds., pp. 549–586, MIT Press, Cambridge, Mass, USA, 1982.
[3]
J. V. Haxby, C. L. Grady, B. Horwitz et al., “Dissociation of object and spatial visual processing pathways in human extrastriate cortex,” Proceedings of the National Academy of Sciences of the United States of America, vol. 88, no. 5, pp. 1621–1625, 1991.
[4]
M. A. Goodale and A. D. Milner, “Separate visual pathways for perception and action,” Trends in Neurosciences, vol. 15, no. 1, pp. 20–25, 1992.
[5]
P. Arno, A. G. De Volder, A. Vanlierde et al., “Occipital activation by pattern recognition in the early blind using auditory substitution for vision,” NeuroImage, vol. 13, no. 4, pp. 632–645, 2001.
[6]
L. Renier, O. Collignon, C. Poirier et al., “Cross-modal activation of visual cortex during depth perception using auditory substitution of vision,” NeuroImage, vol. 26, no. 2, pp. 573–580, 2005.
[7]
M. Ptito, S. M. Moesgaard, A. Gjedde, and R. Kupers, “Cross-modal plasticity revealed by electrotactile stimulation of the tongue in the congenitally blind,” Brain, vol. 128, no. 3, pp. 606–614, 2005.
[8]
C. Poirier, A. De Volder, D. Tranduy, and C. Scheiber, “Pattern recognition using a device substituting audition for vision in blindfolded sighted subjects,” Neuropsychologia, vol. 45, no. 5, pp. 1108–1121, 2007.
[9]
A. Amedi, W. M. Stern, J. A. Camprodon et al., “Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex,” Nature Neuroscience, vol. 10, no. 6, pp. 687–689, 2007.
[10]
E. Striem-Amit, O. Dakwar, L. Reich, and A. Amedi, “The large-scale organization of “visual” streams emerges without visual experience,” Cerebal Cortex. In press.
[11]
I. Matteau, R. Kupers, E. Ricciardi, P. Pietrini, and M. Ptito, “Beyond visual, aural and haptic movement perception: hMT+ is activated by electrotactile motion stimulation of the tongue in sighted and in congenitally blind individuals,” Brain Research Bulletin, vol. 82, no. 5-6, pp. 264–270, 2010.
[12]
L. Renier and A. G. De Volder, “Cognitive and brain mechanisms in sensory substitution of vision: a contribution to the study of human perception,” Journal of Integrative Neuroscience, vol. 4, no. 4, pp. 489–503, 2005.
[13]
C. Poirier, A. G. De Volder, and C. Scheiber, “What neuroimaging tells us about sensory substitution,” Neuroscience and Biobehavioral Reviews, vol. 31, no. 7, pp. 1064–1070, 2007.
[14]
T. W. James, G. K. Humphrey, J. S. Gati, R. S. Menon, and M. A. Goodale, “Differential effects of viewpoint on object-driven activation in dorsal and ventral streams,” Neuron, vol. 35, no. 4, pp. 793–801, 2002.
[15]
K. F. Valyear, J. C. Culham, N. Sharif, D. Westwood, and M. A. Goodale, “A double dissociation between sensitivity to changes in object identity and object orientation in the ventral and dorsal visual streams: a human fMRI study,” Neuropsychologia, vol. 44, no. 2, pp. 218–228, 2006.
[16]
K. Sathian and A. Zangaladze, “Feeling with the mind's eye: contribution of visual cortex to tactile perception,” Behavioural Brain Research, vol. 135, no. 1-2, pp. 127–132, 2002.
[17]
I. Gauthier, W. G. Hayward, M. J. Tarr, A. W. Anderson, P. Skudlarski, and J. C. Gore, “BOLD activity during mental rotation and viewpoint-dependent object recognition,” Neuron, vol. 34, no. 1, pp. 161–171, 2002.
[18]
E. Eger, J. Ashburner, J. D. Haynes, R. J. Dolan, and G. Rees, “fMRI activity patterns in human LOC carry information about object exemplars within category,” Journal of Cognitive Neuroscience, vol. 20, no. 2, pp. 356–370, 2008.
[19]
G. Pourtois, S. Schwartz, M. Spiridon, R. Martuzzi, and P. Vuilleumier, “Object representations for multiple visual categories overlap in lateral occipital and medial fusiform cortex,” Cerebral Cortex, vol. 19, no. 8, pp. 1806–1819, 2009.
[20]
I. Faillenot, S. Sunaert, P. van Hecke, and G. A. Orban, “Orientation discrimination of objects and gratings compared: an fMRI study,” European Journal of Neuroscience, vol. 13, no. 3, pp. 585–596, 2001.
[21]
J. Talairach and P. Tournoux, Co-Planar Stereotaxic Atlas of the Human Brain, Thieme Medical, New York, NY, USA, 1988.
[22]
C. Capelle, C. Trullemans, P. Arno, and C. Veraart, “A real-time experimental prototype for enhancement of vision rehabilitation using auditory substitution,” IEEE Transactions on Biomedical Engineering, vol. 45, no. 10, pp. 1279–1293, 1998.
[23]
P. Arno, C. Capelle, M. C. Wanet-Defalque, M. Catalan-Ahumada, and C. Veraart, “Auditory coding of visual patterns for the blind,” Perception, vol. 28, no. 8, pp. 1013–1029, 1999.
[24]
L. Renier, C. Laloyaux, O. Collignon et al., “The Ponzo illusion with auditory substitution of vision in sighted and early-blind subjects,” Perception, vol. 34, no. 7, pp. 857–867, 2005.
[25]
L. Renier, R. Bruyer, and A. G. De Volder, “Vertical-horizontal illusion present for sighted but not early blind humans using auditory substitution of vision,” Perception and Psychophysics, vol. 68, no. 4, pp. 535–542, 2006.
[26]
L. Renier and A. G. De Volder, “Vision substitution and depth perception: early blind subjects experience visual perspective through their ears,” Disability and Rehabilitation: Assistive Technology, vol. 5, no. 3, pp. 175–183, 2010.
[27]
K. J. Friston, A. P. Holmes, K. J. Worsley, J. P. Poline, C. D. Frith, and R. S. J. Frackowiak, “Statistical parametric maps in functional imaging: a general linear approach,” Human Brain Mapping, vol. 2, no. 4, pp. 189–210, 1994.
[28]
S. C. Prather, J. R. Votaw, and K. Sathian, “Task-specific recruitment of dorsal and ventral visual areas during tactile perception,” Neuropsychologia, vol. 42, no. 8, pp. 1079–1087, 2004.
[29]
R. W. van Boven, J. E. Ingeholm, M. S. Beauchamp, P. C. Bikle, and L. G. Ungerleider, “Tactile form and location processing in the human brain,” Proceedings of the National Academy of Sciences of the United States of America, vol. 102, no. 35, pp. 12601–12605, 2005.
[30]
M. Zhang, E. Mariola, R. Stilla et al., “Tactile discrimination of grating orientation: fMRI activation patterns,” Human Brain Mapping, vol. 25, no. 4, pp. 370–377, 2005.
[31]
E. C. Roth and J. B. Hellige, “Spatial processing and hemispheric asymmetry: contributions of the transient/magnocellular visual system,” Journal of Cognitive Neuroscience, vol. 10, no. 4, pp. 472–484, 1998.
[32]
G. Jager and A. Postma, “On the hemispheric specialization for categorical and coordinate spatial relations: a review of the current evidence,” Neuropsychologia, vol. 41, no. 4, pp. 504–515, 2003.
[33]
K. O. Bushara, R. A. Weeks, K. Ishii et al., “Modality-specific frontal and parietal areas for auditory and visual spatial localization in humans,” Nature Neuroscience, vol. 2, no. 8, pp. 759–766, 1999.
[34]
J. P. Rauschecker and B. Tian, “Mechanisms and streams for processing of “what” and “where” in auditory cortex,” Proceedings of the National Academy of Sciences of the United States of America, vol. 97, no. 22, pp. 11800–11806, 2000.
[35]
C. Alain, S. R. Arnott, S. Hevenor, S. Graham, and C. L. Grady, ““What” and “where” in the human auditory system,” Proceedings of the National Academy of Sciences of the United States of America, vol. 98, no. 21, pp. 12301–12306, 2001.
[36]
C. L. Reed, R. L. Klatzky, and E. Halgren, “What vs. where in touch: an fMRI study,” NeuroImage, vol. 25, no. 3, pp. 718–726, 2005.
[37]
L. A. Renier, I. Anurova, A. G. De Volder, S. Carlson, J. VanMeter, and J. P. Rauschecker, “Multisensory integration of sounds and vibrotactile stimuli in processing streams for “what” and “where”,” Journal of Neuroscience, vol. 29, no. 35, pp. 10950–10960, 2009.
[38]
B. Stricanne, R. A. Andersen, and P. Mazzoni, “Eye-centered, head-centered, and intermediate coding of remembered sound locations in area LIP,” Journal of Neurophysiology, vol. 76, no. 3, pp. 2071–2076, 1996.
[39]
J. C. Culham and N. G. Kanwisher, “Neuroimaging of cognitive functions in human parietal cortex,” Current Opinion in Neurobiology, vol. 11, no. 2, pp. 157–163, 2001.
[40]
E. Ricciardi, D. Bonino, C. Gentili, L. Sani, P. Pietrini, and T. Vecchi, “Neural correlates of spatial working memory in humans: a functional magnetic resonance imaging study comparing visual and tactile processes,” Neuroscience, vol. 139, no. 1, pp. 339–349, 2006.
[41]
C. Alain, Y. He, and C. Grady, “The contribution of the inferior parietal lobe to auditory spatial working memory,” Journal of Cognitive Neuroscience, vol. 20, no. 2, pp. 285–295, 2008.
[42]
R. A. Weeks, A. Aziz-Sultan, K. O. Bushara et al., “A PET study of human auditory spatial processing,” Neuroscience Letters, vol. 262, no. 3, pp. 155–158, 1999.
[43]
P. P. Maeder, R. A. Meuli, M. Adriani et al., “Distinct pathways involved in sound recognition and localization: a human fMRI study,” NeuroImage, vol. 14, no. 4, pp. 802–816, 2001.
[44]
S. Clarke, A. B. Thiran, P. Maeder et al., “What and where in human audition: selective deficits following focal hemispheric lesions,” Experimental Brain Research, vol. 147, no. 1, pp. 8–15, 2002.
[45]
R. J. Zatorre, M. Bouffard, P. Ahad, and P. Belin, “Where is 'where' in the human auditory cortex?” Nature Neuroscience, vol. 5, no. 9, pp. 905–909, 2002.
[46]
S. R. Arnott, M. A. Binns, C. L. Grady, and C. Alain, “Assessing the auditory dual-pathway model in humans,” NeuroImage, vol. 22, no. 1, pp. 401–408, 2004.
[47]
F. Gougoux, R. J. Zatorre, M. Lassonde, P. Voss, and F. Lepore, “A functional neuroimaging study of sound localization: visual cortex activity predicts performance in early-blind individuals,” PLoS Biology, vol. 3, no. 2, article e27, pp. 0324–0333, 2005.
[48]
C. F. Altmann, C. Bledowski, M. Wibral, and J. Kaiser, “Processing of location and pattern changes of natural sounds in the human auditory cortex,” NeuroImage, vol. 35, no. 3, pp. 1192–1200, 2007.
[49]
C. F. Altmann, M. Henning, M. K. D？ring, and J. Kaiser, “Effects of feature-selective attention on auditory pattern and location processing,” NeuroImage, vol. 41, no. 1, pp. 69–79, 2008.
[50]
J. Lewald, K. A. J. Riederer, T. Lentz, and I. G. Meister, “Processing of sound location in human cortex,” European Journal of Neuroscience, vol. 27, no. 5, pp. 1261–1270, 2008.
[51]
R. Malach, J. B. Reppas, R. R. Benson et al., “Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex,” Proceedings of the National Academy of Sciences of the United States of America, vol. 92, no. 18, pp. 8135–8139, 1995.
[52]
K. Grill-Spector, T. Kushnir, T. Hendler, S. Edelman, Y. Itzchak, and R. Malach, “A sequence of object-processing stages revealed by fMRI in the human occipital lobe,” Human Brain Mapping, vol. 6, no. 4, pp. 316–328, 1998.
[53]
Z. Kourtzi and N. Kanwisher, “Cortical regions involved in perceiving object shape,” Journal of Neuroscience, vol. 20, no. 9, pp. 3310–3318, 2000.
[54]
A. Amedi, R. Malach, T. Hendler, S. Peled, and E. Zohary, “Visuo-haptic object-related activation in the ventral visual pathway,” Nature Neuroscience, vol. 4, no. 3, pp. 324–330, 2001.
[55]
A. Amedi, G. Jacobson, T. Hendler, R. Malach, and E. Zohary, “Convergence of visual and tactile shape processing in the human lateral occipital complex,” Cerebral Cortex, vol. 12, no. 11, pp. 1202–1212, 2002.
[56]
R. Kupers, D. R. Chebat, K. H. Madsen, O. B. Paulson, and M. Ptito, “Neural correlates of virtual route recognition in congenital blindness,” Proceedings of the National Academy of Sciences of the United States of America, vol. 107, no. 28, pp. 12716–12721, 2010.
[57]
A. Vanlierde, A. G. De Volder, M. C. Wanet-Defalque, and C. Veraart, “Occipito-parietal cortex activation during visuo-spatial imagery in early blind humans,” NeuroImage, vol. 19, no. 3, pp. 698–709, 2003.
[58]
M. Zhang, V. D. Weisser, R. Stilla, S. C. Prather, and K. Sathian, “Multisensory cortical processing of object shape and its relation to mental imagery,” Cognitive, Affective and Behavioral Neuroscience, vol. 4, no. 2, pp. 251–259, 2004.
[59]
A. T. Sack, C. Jacobs, F. de Martino, N. Staeren, R. Goebel, and E. Formisano, “Dynamic premotor-to-parietal interactions during spatial imagery,” Journal of Neuroscience, vol. 28, no. 34, pp. 8417–8429, 2008.
[60]
H. D. Zimmer, “Visual and spatial working memory: from boxes to networks,” Neuroscience and Biobehavioral Reviews, vol. 32, no. 8, pp. 1373–1395, 2008.
