The goal of this paper is to address a topic that is rarely investigated in the literature of technology-assisted motor rehabilitation, that is, the integration of auditory feedback in the rehabilitation device. After a brief introduction on rehabilitation robotics, the main concepts of auditory feedback are presented, together with relevant approaches, techniques, and technologies available in this domain. Current uses of auditory feedback in the context of technology-assisted rehabilitation are then reviewed. In particular, a comparative quantitative analysis over a large corpus of the recent literature suggests that the potential of auditory feedback in rehabilitation systems is currently and largely underexploited. Finally, several scenarios are proposed in which the use of auditory feedback may contribute to overcome some of the main limitations of current rehabilitation systems, in terms of user engagement, development of acute-phase and home rehabilitation devices, learning of more complex motor tasks, and improving activities of daily living. 1. Introduction Stroke is the leading cause of movement disability in the USA and Europe [1, 2]. In the EU, there are 200 to 300 stroke cases per 100,000 every year, and about 30% survive with major motor deficits [3]. These impressive numbers are increasing due to aging and lifestyle in developed countries. Improving the outcome of movement therapy after stroke is thus a major societal goal that received a lot of interest in the last decade from many researchers in the medical and engineering fields. After the acute phase, stroke patients require continuous medical care and rehabilitation treatment, the latter being usually delivered as both individual and group therapy. The rationale for doing motor rehabilitation is that the motor system is plastic following stroke and can be influenced by motor training [4]. Motor learning is a complex process and to date there is still a lack of knowledge on how the sensory motor system reorganizes in response to movement training [5]. Motor learning can be described as “a set of processes associated with practice or experience leading to relatively permanent changes in the capability for producing skilled action” [6]. Early after a stroke, the brain can undergo dramatic plastic changes [7, 8] that can be further enhanced by environmental stimulation. Animal studies have shown that an enriched poststroke recovery environment can induce structural plastic changes in the brain such as decreased infarct volume and increased dendritic branching, spine density, neurotrophic
References
[1]
L.-J. Lloyd-Jones, R. Adams, M. Carnethon, et al., “Heart disease and stroke statistics 2009 update: a report from the american heart association statistics committee and stroke statistics subcommittee,” Circulation, vol. 119, pp. e21–e181, 2009.
[2]
R. Rosamond, K. Flegal, G. Friday, et al., “Heart disease and stroke statistics-2007 update: a report from the american heart association statistics committee and stroke statistics subcommittee,” Circulation, vol. 115, no. 5, pp. 69–171, 2010.
[3]
Stroke Prevention and Educational Awareness Diffusion (SPREAD). The Italian Guidelines for Stroke Prevention and Treatment, Hyperphar Group, Milano, Italy, 2003.
[4]
R. J. Nudo, “Postinfarct cortical plasticity and behavioral recovery,” Stroke, vol. 38, no. 2, pp. 840–845, 2007.
[5]
D. J. Reinkensmeyer, P. Bonato, M. L. Boninger et al., “Major trends in mobility technology research and development: overview of the results of the NSF-WTEC European study,” Journal of NeuroEngineering and Rehabilitation, vol. 9, article 22, 2012.
[6]
A. Shumway-Cook and M. Woollacott, Motor Learning and Recovery of Function. Motor Control-Theory and Practical Applications, Lippincott Williams & Wilkins, Philadelphia, Pa, USA, 2001.
[7]
O. W. Witte, “Lesion-induced plasticity as a potential mechanism for recovery and rehabilitative training,” Current Opinion in Neurology, vol. 11, no. 6, pp. 655–662, 1998.
[8]
S. H. Kreisel, H. B？zner, and M. G. Hennerici, “Pathophysiology of stroke rehabilitation: temporal aspects of neurofunctional recovery,” Cerebrovascular Diseases, vol. 21, no. 1-2, pp. 6–17, 2006.
[9]
B. B. Johansson, “Functional and cellular effects of environmental enrichment after experimental brain infarcts,” Restorative Neurology and Neuroscience, vol. 22, no. 3-4, pp. 163–174, 2004.
[10]
J. Nithianantharajah and A. J. Hannan, “Enriched environments, experience-dependent plasticity and disorders of the nervous system,” Nature Reviews Neuroscience, vol. 7, no. 9, pp. 697–709, 2006.
[11]
H. T. Hendricks, J. Van Limbeek, A. C. Geurts, and M. J. Zwarts, “Motor recovery after stroke: a systematic review of the literature,” Archives of Physical Medicine and Rehabilitation, vol. 83, no. 11, pp. 1629–1637, 2002.
[12]
H. S. Jorgensen, H. Nakayama, H. O. Raaschou, J. Vive-Larsen, M. Stoier, and T. S. Olsen, “Outcome and time course of recovery in stroke. Part I: outcome. The Copenhagen stroke study,” Archives of Physical Medicine and Rehabilitation, vol. 76, no. 5, pp. 406–412, 1995.
[13]
G. Rosati, “The place of robotics in post-stroke rehabilitation,” Expert Review of Medical Devices, vol. 7, no. 6, pp. 753–758, 2010.
[14]
R. Sigrist, G. Rauter, R. Riener, and P. Wolf, “Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review,” Psychonomic Bulletin and Review, vol. 20, no. 1, pp. 21–53, 2013.
[15]
F. Malouin, C. L. Richards, B. McFadyen, and J. Doyon, “To walk again after a stroke: new perspectives of locomotor rehabilitation,” Medecine/Sciences, vol. 19, no. 10, pp. 994–998, 2003.
[16]
C. Bütefisch, H. Hummelsheim, P. Denzler, and K.-H. Mauritz, “Repetitive training of isolated movements improves the outcome of motor rehabilitation of the centrally paretic hand,” Journal of the Neurological Sciences, vol. 130, no. 1, pp. 59–68, 1995.
[17]
J. Liepert, H. Bauder, W. H. R. Miltner, E. Taub, and C. Weiller, “Treatment-induced cortical reorganization after stroke in humans,” Stroke, vol. 31, no. 6, pp. 1210–1216, 2000.
[18]
G. Kwakkel, B. J. Kollen, and H. I. Krebs, “Effects of robot-assisted therapy on upper limb recovery after stroke: a systematic review,” Neurorehabilitation and Neural Repair, vol. 22, no. 2, pp. 111–121, 2008.
[19]
D. J. Reinkensmeyer, J. A. Galvez, L. Marchai, E. T. Wolbrecht, and J. E. Bobrow, “Some key problems for robot-assisted movement therapy research: a perspective from the University of California at Irvine,” in Proceedings of the IEEE 10th International Conference on Rehabilitation Robotics (ICORR '07), pp. 1009–1015, Noordwijk, The Netherlands, June 2007.
[20]
R. Riener, L. Lünenburger, S. Jezernik, M. Anderschitz, G. Colombo, and V. Dietz, “Patient-cooperative strategies for robot-aided treadmill training: first experimental results,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 13, no. 3, pp. 380–394, 2005.
[21]
R. Colombo, F. Pisano, S. Micera et al., “Assessing mechanisms of recovery during robot-aided neurorehabilitation of the upper limb,” Neurorehabilitation and Neural Repair, vol. 22, no. 1, pp. 50–63, 2008.
[22]
A. A. Timmermans, H. A. Seelen, R. D. Willmann, and H. Kingma, “Technology-assisted training of arm-hand skills in stroke: concepts on reacquisition of motor control and therapist guidelines for rehabilitation technology design,” Journal of NeuroEngineering and Rehabilitation, vol. 6, no. 1, article 1, 2009.
[23]
M. L. Aisen, H. I. Krebs, N. Hogan, F. McDowell, and B. T. Volpe, “The effect of robot-assisted therapy and rehabilitative training on motor recovery following stroke,” Archives of Neurology, vol. 54, no. 4, pp. 443–446, 1997.
[24]
H. I. Krebs, B. T. Volpe, D. Williams et al., “Robot-aided neurorehabilitation: a robot for wrist rehabilitation,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 15, no. 3, pp. 327–335, 2007.
[25]
E. T. Wolbrecht, V. Chan, D. J. Reinkensmeyer, and J. E. Bobrow, “Optimizing compliant, model-based robotic assistance to promote neurorehabilitation,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 16, no. 3, pp. 286–297, 2008.
[26]
T. Nef, M. Mihelj, G. Kiefer, C. Perndl, R. Müller, and R. Riener, “ARMin—exoskeleton for arm therapy in stroke patients,” in Proceedings of the IEEE 10th International Conference on Rehabilitation Robotics (ICORR '07), pp. 68–74, June 2007.
[27]
S. K. Banala, S. H. Kim, S. K. Agrawal, and J. P. Scholz, “Robot assisted gait training with active leg exoskeleton (ALEX),” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 17, no. 1, pp. 2–8, 2009.
[28]
J. Mehrholz, T. Platz, J. Kugler, and M. Pohl, “Electromechanical and robot-assisted arm training for improving arm function and activities of daily living after stroke,” Cochrane Database of Systematic Reviews, no. 4, Article ID CD006876, 2008.
[29]
G. B. Prange, M. J. A. Jannink, C. G. M. Groothuis-Oudshoorn, H. J. Hermens, and M. J. Ijzerman, “Systematic review of the effect of robot-aided therapy on recovery of the hemiparetic arm after stroke,” Journal of Rehabilitation Research and Development, vol. 43, no. 2, pp. 171–183, 2006.
[30]
P. Langhorne, F. Coupar, and A. Pollock, “Motor recovery after stroke: a systematic review,” The Lancet Neurology, vol. 8, no. 8, pp. 741–754, 2009.
[31]
W. S. Harwin, J. L. Patton, and V. R. Edgerton, “Challenges and opportunities for robot-mediated neurorehabilitation,” Proceedings of the IEEE, vol. 94, no. 9, pp. 1717–1726, 2006.
[32]
S. Masiero, M. Armani, and G. Rosati, “Upper-limb robot-assisted therapy in rehabilitation of acute stroke patients: focused review and results of new randomized controlled trial,” Journal of Rehabilitation Research and Development, vol. 48, no. 4, pp. 355–366, 2011.
[33]
L. De Wit, K. Putman, E. Dejaeger et al., “Use of time by stroke patients: a comparison of four European rehabilitation centers,” Stroke, vol. 36, no. 9, pp. 1977–1983, 2005.
[34]
I. Peretz and R. J. Zatorre, “Brain organization for music processing,” Annual Review of Psychology, vol. 56, pp. 89–114, 2005.
[35]
S. Koelsch, E. Kasper, D. Sammler, K. Schulze, T. Gunter, and A. D. Friederici, “Music, language and meaning: brain signatures of semantic processing,” Nature Neuroscience, vol. 7, no. 3, pp. 302–307, 2004.
[36]
M. Popescu, A. Otsuka, and A. A. Ioannides, “Dynamics of brain activity in motor and frontal cortical areas during music listening: a magnetoencephalographic study,” NeuroImage, vol. 21, no. 4, pp. 1622–1638, 2004.
[37]
S. Koelsch, T. Fritz, D. Y. V. Cramon, K. Müller, and A. D. Friederici, “Investigating emotion with music: an fMRI study,” Human Brain Mapping, vol. 27, no. 3, pp. 239–250, 2006.
[38]
S. B. M. J. Martinez and L. M. Parsons, “Passive music listening spontaneously engages limbic and paralimbic systems,” NeuroReport, vol. 15, no. 13, pp. 2033–2037, 2004.
[39]
V. Menon and D. J. Levitin, “The rewards of music listening: response and physiological connectivity of the mesolimbic system,” NeuroImage, vol. 28, no. 1, pp. 175–184, 2005.
[40]
T. S？rk？m？, M. Tervaniemi, S. Laitinen et al., “Music listening enhances cognitive recovery and mood after middle cerebral artery stroke,” Brain, vol. 131, no. 3, pp. 866–876, 2008.
[41]
W. F. Thompson, E. G. Schellenberg, and G. Husain, “Arousal, mood, and the Mozart effect,” Psychological Science, vol. 12, no. 3, pp. 248–251, 2001.
[42]
E. G. Schellenberg, T. Nakata, P. G. Hunter, and S. Tamoto, “Exposure to music and cognitive performance: tests of children and adults,” Psychology of Music, vol. 35, no. 1, pp. 5–19, 2007.
[43]
R. G. Thompson, C. J. A. Moulin, S. Hayre, and R. W. Jones, “Music enhances category fluency in healthy older adults and Alzheimer's disease patients,” Experimental Aging Research, vol. 31, no. 1, pp. 91–99, 2005.
[44]
H. Kim, M.-H. Lee, H.-K. Chang et al., “Influence of prenatal noise and music on the spatial memory and neurogenesis in the hippocampus of developing rats,” Brain and Development, vol. 28, no. 2, pp. 109–114, 2006.
[45]
F. Xu, R. Cai, J. Xu, J. Zhang, and X. Sun, “Early music exposure modifies GluR2 protein expression in rat auditory cortex and anterior cingulate cortex,” Neuroscience Letters, vol. 420, no. 2, pp. 179–183, 2007.
[46]
F. Angelucci, M. Fiore, E. Ricci, L. Padua, A. Sabino, and P. A. Tonali, “Investigating the neurobiology of music: brain-derived neurotrophic factor modulation in the hippocampus of young adult mice,” Behavioural Pharmacology, vol. 18, no. 5-6, pp. 491–496, 2007.
[47]
F. Angelucci, E. Ricci, L. Padua, A. Sabino, and P. A. Tonali, “Music exposure differentially alters the levels of brain-derived neurotrophic factor and nerve growth factor in the mouse hypothalamus,” Neuroscience Letters, vol. 429, no. 2-3, pp. 152–155, 2007.
[48]
S. Chikahisa, H. Sei, M. Morishima et al., “Exposure to music in the perinatal period enhances learning performance and alters BDNF/TrkB signaling in mice as adults,” Behavioural Brain Research, vol. 169, no. 2, pp. 312–319, 2006.
[49]
D. Centonze, S. Rossi, A. Tortiglione et al., “Synaptic plasticity during recovery from permanent occlusion of the middle cerebral artery,” Neurobiology of Disease, vol. 27, no. 1, pp. 44–53, 2007.
[50]
W.-R. Sch？bitz, T. Steigleder, C. M. Cooper-Kuhn et al., “Intravenous brain-derived neurotrophic factor enhances poststroke sensorimotor recovery and stimulates neurogenesis,” Stroke, vol. 38, no. 7, pp. 2165–2172, 2007.
[51]
M. Bangert and E. O. Altenmüller, “Mapping perception to action in piano practice: a longitudinal DC-EEG study,” BMC Neuroscience, vol. 4, article 26, 2003.
[52]
M. Bangert, T. Peschel, G. Schlaug et al., “Shared networks for auditory and motor processing in professional pianists: evidence from fMRI conjunction,” NeuroImage, vol. 30, no. 3, pp. 917–926, 2006.
[53]
K. E. Watkins, A. P. Strafella, and T. Paus, “Seeing and hearing speech excites the motor system involved in speech production,” Neuropsychologia, vol. 41, no. 8, pp. 989–994, 2003.
[54]
C. Pantev, A. Engelien, V. Candia, and T. Elbert, “Representational cortex in musicians plastic alterations in response to musical practice,” Annals of the New York Academy of Sciences, vol. 930, pp. 300–314, 2001.
[55]
J.-P. Azulay, S. Mesure, B. Amblard, O. Blin, I. Sangla, and J. Pouget, “Visual control of locomotion in Parkinson's disease,” Brain, vol. 122, no. 1, pp. 111–120, 1999.
[56]
Y. Baram and A. Miller, “Auditory feedback control for improvement of gait in patients with Multiple Sclerosis,” Journal of the Neurological Sciences, vol. 254, no. 1-2, pp. 90–94, 2007.
[57]
M. H. Thaut, G. C. McIntosh, and R. R. Rice, “Rhythmic facilitation of gait training in hemiparetic stroke rehabilitation,” Journal of the Neurological Sciences, vol. 151, no. 2, pp. 207–212, 1997.
[58]
M. Schauer and K.-H. Mauritz, “Musical motor feedback (MMF) in walking hemiparetic stroke patients: randomized trials of gait improvement,” Clinical Rehabilitation, vol. 17, no. 7, pp. 713–722, 2003.
[59]
F. DelOlmo and J. Cuderio, “A simple procedure using auditory stimuli to improve movement in Parkinson’s disease: a pilot study,” Neurology & Clinical Neurophysiology, vol. 2, pp. 1–15, 2003.
[60]
T. Mergner, C. Maurer, and R. J. Peterka, “A multisensory posture control model of human upright stance,” Progress in Brain Research, vol. 142, pp. 189–201, 2003.
[61]
E. Altenmüller, “Neurology of musical performance,” Clinical Medicine, vol. 8, pp. 410–413, 2008.
[62]
T. F. Münte, E. Altenmüller, and L. J？ncke, “The musician’s brain as a model of neuroplasticity,” Nature Reviews Neuroscience, vol. 3, pp. 473–8,, 2002.
[63]
K. Amunts, G. Schlaug, L. J？ncke, et al., “Motor cortex and hand motor skills: structural compliance in the human brain,” Human Brain Mapping, vol. 5, pp. 206–215, 1997.
[64]
C. Gaser and G. Schlaug, “Brain structures differ between musicians and non-musicians,” Journal of Neuroscience, vol. 23, no. 27, pp. 9240–9245, 2003.
[65]
R. A. Magill, Motor Learning and Control: Concepts and Applications, McGraw-Hill, New York, NY, USA, 7th edition, 2003.
[66]
P. van Vliet and G. Wulf, “Extrinsic feedback for motor learning after stroke: what is the evidence?” Disability and Rehabilitation, vol. 28, no. 13-14, pp. 831–840, 2006.
[67]
S. Mountford and W. Gaver, The Art of Human-Computer Interface Design, Talking and Listening to Computers, Addison-Wesley, Reading, Mass, USA, 1990.
[68]
D. K. McGookin and S. A. Brewster, “Understanding concurrent earcons: applying auditor y scene analysis principles to concurrent earcon recognition,” ACM Transactions on Applied Perception, vol. 1, no. 2, pp. 130–155, 2004.
[69]
W. W. Gaver, Handbook of Human-Computer Interaction, vol. 1 of Auditory Interfaces, Elsevier, Amsterdam, The Netherlands, 1997.
[70]
M. M. Blattner, D. A. Sumikawa, and R. M. Greenberg, “Earcons and icons: their structure and common design principles,” Human-Computer Interaction, vol. 4, no. 1, pp. 11–44, 1989.
[71]
H. Palom？ki, “Meanings conveyed by simple auditory rhythms,” in Proceedings of the International Conference Auditory Display (ICAD ’06), pp. 99–104, London, UK, June 2006.
[72]
C. Scalett, “Auditory display: sonification, audification, and auditory interfaces,” in Sound Synthesis Algorithms for Auditory Data Representations, vol. 1, pp. 223–251, Adison-Wesley, Reading, Mass, USA, 1994.
[73]
F. Dombois, “Using audification in planetary seismology,” in Proceedings of the International Conference Auditory Display (ICAD ’01), pp. 227–230, Espoo, July 2001.
[74]
G. Baier, T. Hermann, and U. Stephani, “Event-based sonification of eeg rhythms in real time,” Clinical Neurophysiology, vol. 118, no. 6, pp. 1377–1386, 2007.
[75]
T. M. Rutkowski, F. Vialatte, A. Cichocki, D. Mandic, and A. Barros, “Auditory feedback for brain computer interface management -an eeg data sonification approach,” in Knowledge-Based Intelligent Information and Engineering Systems, B. Gabrys, R. J. Howlett, and L. C. Jain, Eds., vol. 4253 of Lecture Notes in Computer Science, pp. 1232–1239, Springer, Berlin, Germany, 2006.
[76]
S. A. Brewster, “Using nonspeech sounds to provide navigation cues,” ACM Transactions on Computer-Human Interaction, vol. 5, no. 3, pp. 224–259, 1998.
[77]
T. Igarashi and J. F. Hughes, “Voice as sound: Using non-verbal voice input for interactive control,” in Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology (UIST '01), pp. 155–156, November 2001.
[78]
G. Kramer, B. Walker, T. Bonebright, et al., “Sonification report: status of the field and research agenda,” International Community for Auditory Display, National Science Foundation, 1999.
[79]
R. Ramloll, W. Yu, B. Riedel, and S. Brewster, “Using non-speech sounds to improve access to 2D tabular numerical information for visually impaired users,” in Proceedings of the British HCI Group, pp. 515–530, Springer, Lille, France, September 2001.
[80]
L. Brown, S. Brewster, S. Ramloll, R. Burton, and B. Riedel, “Design guidelines for audio presentation of graphs and tables,” in Proceedings of the International Conference Auditory Display (ICAD ’03), pp. 284–287, Boston, Mass, USA, July 2003.
[81]
S. Smith, R. D. Bergeron, and G. G. Grinstein, “Stereophonic and surface sound generation for exploratory data analysis,” in Proceedings of the Human Factors in Computing Systems (CHI ’90), pp. 125–132, Seattle, Wash, USA, 1990.
[82]
A. Hunt, T. Hermann, and S. Pauletto, “Interacting with sonification systems: closing the loop,” in Proceedings of the 8th International Conference on Information Visualisation (IV '04), pp. 879–884, July 2004.
[83]
O. Ben-Tal, J. Berger, B. Cook, M. Daniels, G. Scavone, and P. Cook, “Sonart: the sonification application research toolbox,” in Proceedings of the International Conference Auditory Display (ICAD ’02), Kyoto, Japan, 2002.
[84]
B. Walker and J. Cothran, “Sonification sandbox: a graphical toolkit for auditory graphs,” in Proceedings of the International Conference Auditory Display (ICAD ’03), Boston, Mass, USA, 2003.
[85]
S. Pauletto and A. Hunt, “A toolkit for interactive sonification,” in Proceedings of the International Conference Auditory Display (ICAD ’04), Sydney, Australia, 2004.
[86]
T. Lokki and M. Gr？hn, “Navigation with auditory cues in a virtual environment,” IEEE Multimedia, vol. 12, no. 2, pp. 80–86, 2005.
[87]
H.-J. Song, K. Beilharz, and D. Cabrera, “Evaluation of spatial presentation in sonification for identifying concurrent audio streams,” in Proceedings of the Proceedings of the Auditory Display (ICAD ’07), pp. 285–292, Montreal, Canada, 2007.
[88]
J. Daniel, R. Nicol, and S. Moreau, “Further investigations of high order ambisonics and wavefield synthesis for holophonic sound imaging,” in Proceedings of the 114th AES Convention, Amsterdam, The Netherlands, March 2003.
[89]
C. I. Cheng and G. H. Wakefield, “Introduction to head-related transfer functions (HRTFs): representations of HRTFs in time, frequency, and space,” Journal of the Audio Engineering Society, vol. 49, no. 4, pp. 231–249, 2001.
[90]
C. P. Brown and R. O. Duda, “A structural model for binaural sound synthesis,” IEEE Transactions on Speech and Audio Processing, vol. 6, no. 5, pp. 476–488, 1998.
[91]
S. Spagnol, M. Geronazzo, and F. Avanzini, “Fitting pinna-related transfer functions to anthropometry for binaural sound rendering,” in Proceedings of the IEEE International Workshop on Multimedia Signal Processing (MMSP '10), pp. 194–199, Saint-Malo, France, October 2010.
[92]
L. Mion, F. Avanzini, B. Mantel, B. Bardy, and T. A. Stoffregen, “Real-time auditory-visual distance rendering for a virtual reaching task,” in Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST '07), pp. 179–182, Newport Beach, Calif, USA, November 2007.
[93]
M. S. Cameir？o, S. Bermúdez i Badia, L. Zimmerli, E. D. Oller, and P. F. M. J. Verschure, “The rehabilitation gaming system: a virtual reality based system for the evaluation and rehabilitation of motor deficits,” in Proceedings of the IEEE International Conference Virtual Rehabil, pp. 29–33, September 2007.
[94]
R. Loureiro, F. Amirabdollahian, M. Topping, B. Driessen, and W. Harwin, “Upper limb robot mediated stroke therapy—GENTLE/s approach,” Autonomous Robots, vol. 15, no. 1, pp. 35–51, 2003.
[95]
A. G. D. Corrêa, G. A. De Assis, M. Do Nascimento, I. Ficheman, and R. De Deus Lopes, “GenVirtual: An augmented reality musical game for cognitive and motor rehabilitationx,” in Proceedings of the Virtual Rehabilitation (IWVR '07), pp. 1–6, September 2007.
[96]
J. A. Gil, M. Alca？iz, J. Montesa et al., “Low-cost virtual motor rehabilitation system for standing exercises,” in Proceedings of the Virtual Rehabilitation (IWVR '07), pp. 34–38, September 2007.
[97]
H. Igo Krebs, N. Hogan, M. L. Aisen, and B. T. Volpe, “Robot-aided neurorehabilitation,” IEEE Transactions on Rehabilitation Engineering, vol. 6, no. 1, pp. 75–87, 1998.
[98]
H. I. Krebs and N. Hogan, “Therapeutic robotics: a technology push,” Proceedings of the IEEE, vol. 94, no. 9, pp. 1727–1738, 2006.
[99]
M. J. Johnson, H. F. M. Van der Loos, C. G. Burgar, P. Shor, and L. J. Leifer, “Design and evaluation of Driver's SEAT: a car steering simulation environment for upper limb stroke therapy,” Robotica, vol. 21, no. 1, pp. 13–23, 2003.
[100]
R. F. Boian, J. E. Deutsch, C. S. Lee, G. C. Burdea, and J. Lewis, “Haptic effects for virtual reality-based post-stroke rehabilitation,” in Proceedings of the Symposium Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 247–253, Los Alamitos, Calif, USA, 2003.
[101]
D. Hilton, S. Cobb, T. Pridmore, and J. Gladman, “Virtual reality and stroke rehabilitation: a tangible interface to an every day task,” in Proceedings of the International Conference on Disability, Virtual Reality and Associated Technologies, pp. 63–70, 2002.
[102]
S. Masiero, A. Celia, G. Rosati, and M. Armani, “Robotic-assisted rehabilitation of the upper limb after acute stroke,” Archives of Physical Medicine and Rehabilitation, vol. 88, no. 2, pp. 142–149, 2007.
[103]
R. Colombo, F. Pisano, S. Micera et al., “Robotic techniques for upper limb evaluation and rehabilitation of stroke patients,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 13, no. 3, pp. 311–324, 2005.
[104]
A. Schaufelberger, J. Zitzewitz, and R. Riener, “Evaluation of visual and auditory feedback in virtual obstacle walking,” Presence: Teleoperators and Virtual Environments, vol. 17, no. 5, pp. 512–524, 2008.
[105]
T. Koritnik, A. Koenig, T. Bajd, R. Riener, and M. Munih, “Comparison of visual and haptic feedback during training of lower extremities,” Gait and Posture, vol. 32, no. 4, pp. 540–546, 2010.
[106]
P. H. Wilson, J. Duckworth, N. Mumford et al., “A virtual tabletop workspace for the assessment of upper limb function in Traumatic Brain Injury (TBI),” in Proceedings of the Virtual Rehabilitation (IWVR '07), pp. 14–19, September 2007.
[107]
T. Erni and V. Dietz, “Obstacle avoidance during human walking: learning rate and cross-modal transfer,” Journal of Physiology, vol. 534, no. 1, pp. 303–312, 2001.
[108]
H. Sveistrup, “Motor rehabilitation using virtual reality,” Journal of NeuroEngineering and Rehabilitation, vol. 1, article 10, 2004.
[109]
M. K. Holden, “Virtual environments for motor rehabilitation: review,” Cyberpsychology and Behavior, vol. 8, no. 3, pp. 187–211, 2005.
[110]
M. S. Cameir？o, S. B. I. Badia, and P. F. M. J. Verschure, “Virtual reality based upper extremity rehabilitation following stroke: a review,” Journal of Cyber Therapy and Rehabilitation, vol. 1, no. 1, pp. 63–74, 2008.
[111]
C. Senanayake and S. M. N. A. Senanayake, “Emerging robotics devices for therapeutic rehabilitation of the lower extremity,” in Proceedings of the IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM '09), pp. 1142–1147, July 2009.
[112]
J. Stein, R. Hughes, S. E. Facoli, H. I. Krebs, and N. Hogan, Stroke Recovery and Rehabilitation, Technological Aids for Motor Recovery, Demos Medical Publishing, 2009.
[113]
G. Rosati, F. Oscari, S. Spagnol, F. Avanzini, and S. Masiero, “Effect of task-related continuous auditory feedback during learning of tracking motion exercises,” Journal of NeuroEngineering and Rehabilitation, vol. 9, no. 79, 2012.
[114]
D. Zanotto, G. Rosati, S. Spagnol, P. Stegall, and S. K. Agrawal, “Effects of complementary auditory feedback in robot-assisted lower extremity motor adaptation,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 21, no. 5, pp. 775–786, 2013.
[115]
P. Sanderson, A. Wee, E. Seah, and P. Lacherez, “Auditory alarms, medical standards, and urgency,” in Proceedings of the International Conference Auditory Display (ICAD ’06), London, UK, June 2006.
[116]
J. Edworthy, S. Loxley, and I. Dennis, “Improving auditory warning design: relationship between warning sound parameters and perceived urgency,” Human Factors, vol. 33, no. 2, pp. 205–231, 1991.
[117]
M. Watson, “Scalable earcons: bridging the gap between intermittent and continuous auditory displays,” in Proceedings of the International Conference Auditory Display (ICAD ’06), pp. 59–62, London, UK, June 2006.
[118]
S. Pauletto and A. Hunt, “The sonifiction of EMG data,” in Proceedings of the International Conference Auditory Display (ICAD ’06), London, UK, June 2006.
[119]
T. Ifukube, T. Sasaki, and C. Peng, “A blind mobility aid modeled after echolocation of bats,” IEEE Transactions on Biomedical Engineering, vol. 38, no. 5, pp. 461–465, 1991.
[120]
P. B. L. Meijer, “An experimental system for auditory image representations,” IEEE Transactions on Biomedical Engineering, vol. 39, no. 2, pp. 112–121, 1992.
[121]
M. Talbot and B. Cowan, “Trajectory capture in frontal plane geometry for visually impaired,” in Proceedings of the International Conference Auditory Display (ICAD ’06), London, UK, June 2006.
[122]
C. C. Pratt, “The spatial character of high and low tones,” Journal of Experimental Psychology, vol. 13, no. 3, pp. 278–285, 1930.
[123]
S. Kousidou, N. G. Tsagarakis, C. Smith, and D. G. Caldwell, “Task-orientated biofeedback system for the rehabilitation of the upper limb,” in Proceedings of the IEEE 10th International Conference on Rehabilitation Robotics (ICORR '07), pp. 376–384, June 2007.
[124]
C. Karageorghis and P. Terry, “The psycho-physical effects of music in sport and exercise: a review,” The Journal of Sport Behavior, vol. 20, pp. 54–68, 1997.
[125]
C. Hendrix and W. Barfield, “Presence in virtual environments as a function of visual and auditory cues,” in Proceedings of the IEEE Annual Virtual Reality International Symposium, pp. 74–82, March 1995.
[126]
M. Rauterberg and E. Styger, “Positive effects of sound feedback during the operation of a plant simulator,” in Human-Computer Interaction, vol. 876 of Lecture Notes In Computer Science, pp. 35–44, 1994.
[127]
S. Brewster, P. Wright, A. Dix, and A. Edwards, “The sonic enhancement of graphical buttons,” in Proceedings of the IFIP International Conference on Human Computer Interaction (Interact ’95), pp. 43–48, Lillehammer, Norway, 1995.
[128]
E. Zwicker and H. Fastl, Psychoacoustics: Facts and Models, Springer, Berlin, Germany, 1999.
[129]
S. Ferguson, D. Cabrera, K. Beilharz, and H.-J. Song, “Using psychoacoustical models for information sonification,” in Proceedings of the international Conference on Auditory Display (ICAD ’06), pp. 113–120, London, UK, June 2006.
[130]
H. Zhao, C. Plaisant, B. Shneiderman, and J. Lazar, “Data sonification for users with visual impairment: a case study with georeferenced data,” ACM Transactions on Computer-Human Interaction, vol. 15, no. 1, article 4, 2008.
[131]
S. Handel, Listening: An Introduction to the Perception of Auditory Events, MIT Press, Cambridge, Mass, USA, 1989.
[132]
R. Secoli, M.-H. Milot, G. Rosati, and D. J. Reinkensmeyer, “Effect of visual distraction and auditory feedback on patient effort during robot-assisted movement training after stroke,” Journal of NeuroEngineering and Rehabilitation, vol. 8, no. 1, article 21, 2011.
[133]
K. A. Kaczmarek, J. G. Webster, P. Bach-y-Rita, and W. J. Tompkins, “Electrotactile and vibrotactile displays for sensory substitution systems,” IEEE Transactions on Biomedical Engineering, vol. 38, no. 1, pp. 1–16, 1991.
[134]
M. Ohuchi, Y. Iwaya, Y. Suzuki, and T. Munekata, “Cognitive-map formation of blind persons in a virtual sound environment,” in Proceedings of the International Conference on Auditory Display (ICAD ’06), pp. 1–7, London, UK, June 2006.
[135]
F. Oscari, R. Secoli, F. Avanzini, G. Rosati, and D. J. Reinkensmeyer, “Substituting auditory for visual feedback to adapt to altered dynamic and kinematic environments during reaching,” Experimental Brain Research, vol. 221, no. 1, pp. 33–41, 2012.
[136]
H. J. Ezekiel, N. K. Lehto, T. L. Marley, L. R. Wishart, and T. D. Lee, “Application of motor learning principles: the physiotherapy client as a problem-solver. III Augmented feedback,” Physiotherapy Canada, vol. 53, no. 1, pp. 33–39, 2001.
[137]
S. Zanolla, S. Canazza, A. Rodà, A. Camurri, and G. Volpe, “Entertaining listening by means of the stanza logo-motoria: an interactive multimodal environment,” Entertainment Computing, vol. 4, no. 3, pp. 213–220, 2013.
[138]
R. B. Welch and D. H. Warren, “Intersensory interactions,” in Handbook of Perception and Human Performance, K. R. Boff, L. Kaufman, and J. P. Thomas, Eds., vol. 1 of Sensory Processes and Perception, pp. 1–36, John Wiley & Sons, New York, NY, USA, 1986.
[139]
A. Lecuyer, S. Coquillart, A. Kheddar, P. Richard, and P. Coiffet, “Pseudo-haptic feedback: can isometric input devices simulate force feedback?” in IEEE Virtual Reality, pp. 83–90, March 2000.
[140]
I. M. K. van Mensvoort, “What you see is what you feel: exploiting the dominance of the visual over the haptic domain to simulate force-feedback with cursor displacements,” in Proceedings of the 4th Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (DIS '02), pp. 345–348, June 2002.
[141]
F. Avanzini, D. Rocchesso, and S. Serafin, “Friction sounds for sensory substitution,” in Proceedings of the International Conference on Auditory Display (ICAD ’04), Sydney, Australia, 2004.
[142]
J. Emken and D. Reinkensmeyer, “Robot-enhanced motor learning: accelerating internal model formation during locomotion by transient dynamic amplification,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 99, pp. 1–7, 2005.
[143]
M. Rath and D. Rocchesso, “Continuous sonic feedback from a rolling ball,” IEEE Multimedia, vol. 12, no. 2, pp. 60–69, 2005.
[144]
W. W. Gaver, “SonicFinder: an interface that uses auditory icons,” Human-Computer Interaction, vol. 4, no. 1, pp. 67–94, 1989.
[145]
P. Rougier, “Influence of visual feedback on successive control mechanisms in upright quiet stance in humans assessed by fractional Brownian motion modelling,” Neuroscience Letters, vol. 266, no. 3, pp. 157–160, 1999.
[146]
G. Rosati, J. E. Bobrow, and D. J. Reinkensmeyer, “Compliant control of post-stroke rehabilitation robots: using movement-specific models to improve controller performance,” in Proceedings of the ASME International Mechanical Engineering Congress and Exposition (IMECE '08), pp. 167–174, Boston, Mass, USA, November 2008.
[147]
G. De Poli and D. Rocchesso, “Physically based sound modelling,” Organized Sound, vol. 3, no. 1, pp. 61–76, 1998.
[148]
M. Kleiman-Weiner and J. Berger, “The sound of one arm swinging: a model for multidimensional auditory display of physical motion,” in Proceedings of the International Conference on Auditory Display (ICAD ’06), pp. 278–280, London, UK, June 2006.
[149]
G. Peterson and H. Barney, “Control methods used in a study of the vowels,” Journal of the Acoustical Society of America, vol. 24, pp. 175–184, 1952.
[150]
J. E. Deutsch, R. F. Boian, J. A. Lewis, G. C. Burdea, and A. Minsky, “Haptic effects modulate kinetics of gait but not experience of realism in a virtual reality walking simulator,” in Proceedings of the Virtual Rehabilitation (IWVR '08), pp. 36–40, August 2008.
[151]
B. B. Connor, A. M. Wing, G. W. Humphreys, R. M. Bracewell, and D. A. Harvey, “Errorless learning using haptic guidance: research in cognitive rehabilitation following stroke,” in Proceedings of the International Conference Disability, Virtual Reality and Associated Technologies, pp. 77–84, 2002.
[152]
A. Frisoli, M. Bergamasco, M. C. Carboncini, and B. Rossi, “Robotic assisted rehabilitation in virtual reality with the L-EXOS,” Studies in Health Technology and Informatics, vol. 145, pp. 40–54, 2009.
[153]
K. J. Wisneski and M. J. Johnson, “Quantifying kinematics of purposeful movements to real, imagined, or absent functional objects: implications for modelling trajectories for robot-assisted ADL tasks,” Journal of NeuroEngineering and Rehabilitation, vol. 4, article 7, 2007.
[154]
S.-C. Yeh, A. Rizzo, M. McLaughlin, and T. Parsons, “VR enhanced upper extremity motor training for post-stroke rehabilitation: task design, clinical experiment and visualization on performance and progress,” Studies in Health Technology and Informatics, vol. 125, pp. 506–511, 2007.
[155]
J. C. Stewart, S.-C. Yeh, Y. Jung et al., “Intervention to enhance skilled arm and hand movements after stroke: a feasibility study using a new virtual reality system,” Journal of NeuroEngineering and Rehabilitation, vol. 4, article 21, 2007.
[156]
P. Staubli, T. Nef, V. Klamroth-Marganska, and R. Riener, “Effects of intensive arm training with the rehabilitation robot ARMin II in chronic stroke patients: four single-cases,” Journal of NeuroEngineering and Rehabilitation, vol. 6, no. 1, article 46, 2009.
[157]
E. B. Brokaw, T. Nef, T. M. Murray, and P. S. Lum, “Time independent functional training of inter-joint arm coordination using the ARMin III robot,” in Proceedings of the 26th Southern Biomedical Engineering Conference (SBEC '10), pp. 113–117, College Park, Md, USA, May 2010.
[158]
G. Rosati, P. Gallina, and S. Masiero, “Design, implementation and clinical tests of a wire-based robot for neurorehabilitation,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 15, no. 4, pp. 560–569, 2007.
[159]
C.-Y. Shing, C.-P. Fung, T.-Y. Chuang, I.-W. Penn, and J.-L. Doong, “The study of auditory and haptic signals in a virtual reality-based hand rehabilitation system,” Robotica, vol. 21, no. 2, pp. 211–218, 2003.
[160]
M. Wellner, A. Schaufelberger, J. V. Zitzewitz, and R. Riener, “Evaluation of visual and auditory feedback in virtual obstacle walking,” Presence: Teleoperators and Virtual Environments, vol. 17, no. 5, pp. 512–524, 2008.
[161]
A. Koenig, K. Brütsch, L. Zimmerli et al., “Virtual environments increase participation of children with cerebral palsy in robot-aided treadmill training,” in Proceedings of the Virtual Rehabilitation (IWVR '08), pp. 121–126, August 2008.
