This paper presents a user localization system based on the fusion of visual information and sound source localization, implemented on a social robot called Maggie. One of the main requisites to obtain a natural interaction between human-human and human-robot is an adequate spatial situation between the interlocutors, that is, to be orientated and situated at the right distance during the conversation in order to have a satisfactory communicative process. Our social robot uses a complete multimodal dialog system which manages the user-robot interaction during the communicative process. One of its main components is the presented user localization system. To determine the most suitable allocation of the robot in relation to the user, a proxemic study of the human-robot interaction is required, which is described in this paper. The study has been made with two groups of users: children, aged between 8 and 17, and adults. Finally, at the end of the paper, experimental results with the proposed multimodal dialog system are presented.
References
[1]
Yager, D.; Hoy, R. The cyclopean ear: A new sense for the praying mantis. Science 1986, 231, 727–729, doi:10.1126/science.3945806. 3945806
[2]
Hudspeth, A.J. The hair cells of the inner ear. Sci. Am. 1983, 248, 54–64. 6337395
[3]
Brown, A.C. The sense of rotation and the anatomy and physiology of the semicircular canals of the internal ear. J. Anat. Physiol. 1874, 8, 327–331. 17231027
[4]
Dooling, R.; Popper, A. Hearing in birds and reptiles: An overview. In Comparative Hearing: Reptiles and Birds; Springer-Verlag: New York, NY, USA, 2000; pp. 1–12.
[5]
Ross, R.J.; Smith, J.J.B. Detection of substrate vibrations by salamanders: Inner ear sense organ activity. Can. J. Zool. 1978, 56, 1156–1162, doi:10.1139/z78-159.
[6]
Nakadai, K.; Matsuura, D.; Okuno, H.; Kitano, H. Applying Scattering Theory to Robot Audition System: Robust Sound Source Localization and Extraction. Proceedings of the IEEE /RSJ International Conference on Intelligent Robots and Systems, (IROS 2003), Las Vegas, NV, USA, 27– 31 October 2003. Volume 2; pp. 1147–1152.
[7]
Nakadai, K.; Okuno, H.; Kitano, H. Real-Time Sound Source Localization and Separation for Robot Audition. Proceedings of the Seventh International Conference on Spoken Language Processing, Denver, CO, USA, 16–20 September 2002.
[8]
Valin, J.; Michaud, F.; Rouat, J.; Létourneau, D. Robust Sound Source Localization Using a Microphone Array on A Mobile Robot. Proceedings of the IEEE /RSJ International Conference on Intelligent Robots and Systems, (IROS 2003), Las Vegas, NV, USA, 27– 31 October 2003. Volume 2; pp. 1228–1233.
[9]
Murray, J.; Erwin, H.; Wermter, S. Robotics Sound-Source Localization and Tracking Using Interaural Time Difference and Crosscorrelation. Proceedings of NeuroBotics Workshop, Ulm, Germany, 20 September 2004; pp. 89–97.
[10]
Andersson, S.; Handzel, A.; Shah, V.; Krishnaprasad, P. Robot Phonotaxis with Dynamic Sound-Source Localization. In Proceedings of the 2004 IEEE International Conference on Robotics and Automation (ICRA'04), New Orleans, LA, USA, 26 April– 1 May 2004. Volume 5; pp. 4833–4838.
[11]
Alonso-Martin, F.; Salichs, M. Integration of a voice recognition system in a social robot. Cybern. Syst. 2011, 42, 215–245, doi:10.1080/01969722.2011.583593.
[12]
Wright, B.A.; Fitzgerald, M.B. Different patterns of human discrimination learning for two interaural cues to sound-source location. Proc. Natl. Acad. Sci. USA 2001, 98, 12307–12312, doi:10.1073/pnas.211220498. 11593048
[13]
Handzel, A.; Krishnaprasad, P. Biomimetic sound-source localization. IEEE Sens. J. 2002, 2, 607–616, doi:10.1109/JSEN.2002.807772.
[14]
Brandstein, M.; Ward, D. Microphone Arrays: Signal Processing Techniques And Applications; Springer Verlag: Berlin/Heidelberg, Germany, 2001.
[15]
Benesty, J.; Chen, J.; Huang, Y. Microphone Array Signal Processing; Springer Verlag: Berlin/Heidelberg, Germany, 2008. Volume 1.
[16]
Saxena, A.; Ng, A.Y. Learning Sound Location from a Single Microphone. Proceedings of the IEEE International Conference on Robotics and Automation, Kobe, Japan, 12– 17 May 2009; pp. 1737–1742.
[17]
Morrongiello, B.A. Infants' monaural localization of sounds: Effects of unilateral ear infection. J. Acoust. Soc. Am. 1989, 86, 597–602, doi:10.1121/1.398749. 2768674
[18]
Hu, J.-S.; Chan, C.-Y.; Wang, C.-K.; Wang, C.-C. Simultaneous Localization of Mobile Robot and Multiple Sound Sources Using Microphone Array. Proceedings of the IEEE International Conference on Robotics and Automation, Kobe, Japan, 12– 17 May 2009; pp. 29–34.
[19]
Briere, S.; Valin, J.M.; Michaud, F.; Letourneau, D. Embedded Auditory System for Small Mobile Robots. Proceedings of the IEEE International Conference on Robotics and Automation, Pasadena, CA, USA, 19– 23 May 2008; pp. 3463–3468.
[20]
Valin, J.M.; Michaud, F.; Rouat, J. Robust 3D Localization and Tracking of Sound Sources Using Beamforming and Particle Filtering. Proceedings of the IEEE International Conference on Acoustics Speed and Signal Processing, Toulouse, France, 14– 19 May 2006. Volume 4; pp. IV:841–IV:844.
[21]
Briere, S.; Létourneau, D.; Fréchette, M. Embedded and Integration Audition for a Mobile Robot. Proceedings of the AAAI Fall Symposium, Washington, DC, USA, 13– 15 October 2006.
[22]
Nakadai, K.; Okuno, H.G.; Nakajima, H.; Hasegawa, Y.; Tsujino, H. An Open Source Software System for Robot Audition HARK and Its Evaluation. Proceedings of the Humanoids 2008: 8th IEEE -RAS International Conference on Humanoid Robots, Daejeon, South Korea, 1– 3 December 2008; pp. 561–566.
[23]
Takahashi, T.; Nakadai, K.; Komatani, K.; Ogata, T.; Okuno, H. Improvement in Listening Capability for Humanoid Robot HRP-2. Proceedings of 2010 IEEE International Conference on the Robotics and Automation (ICRA), Anchorage, AK, USA, 3– 7 May 2010; pp. 470–475.
[24]
Valin, J.; Michaud, F.; Rouat, J. Robust localization and tracking of simultaneous moving sound sources using beamforming and particle filtering. Robot. Auton. Syst. 2007, 55, 216–228, doi:10.1016/j.robot.2006.08.004.
[25]
Hall, E. The Hidden Dimension; Anchor: Garden, NY, USA, 1966.
[26]
Argyle, M. Bodily Communication; Methuen: New York, NY, USA, 1988; p. 363.
[27]
Argyle, M.; Dean, J. Eye-contact, distance and affiliation. Sociometry 1965, 28, 289–304, doi:10.2307/2786027. 14341239
[28]
Lambert, D. The Diagram Group. In Body Language; Harper Collins: Glasgow, UK, 2004.
[29]
Hayduk, L. Personal space: An evaluative and orienting overview. Psychol. Bull. 1978, 85, 117, doi:10.1037/0033-2909.85.1.117.
[30]
Burgoon, J.K.; Jones, S.B. Toward a Theory of Personal Space Expectations and Their Violations; Human Communication Research: Wiley, London, UK, 1976.
[31]
Nass, C.; Steuer, J.; Tauber, E.R. Computers are Social Actors. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Celebrating Interdependence (CHI '94), Boston, MA, USA, 24–28 April 1994; pp. 72–78.
[32]
Takayama, L.; Pantofaru, C. Influences on Proxemic Behaviors in Human-Robot Interaction. Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, St. Louis, MO, USA, 11– 15 October 2009; pp. 5495–5502.
[33]
Breazeal, C.; Scassellati, B. Infant-like social interactions between a robot and a human caregiver. Adapt. Behav. 2000, 8, 49–74, doi:10.1177/105971230000800104.
[34]
Hüttenrauch, H. Investigating Spatial Relationships in Human-Robot Interaction. Proceedings of 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China, 9– 15 October 2006; pp. 5052–5059.
[35]
Walters, M. An Empirical Framework for Human-Robot Proxemics. Proceedings of Symposium at the AISB09 Convention, Edinburgh, UK, 6– 9 April 2009. Volume 8; pp. 5052–5059.
[36]
Kheng Lee, K.; Syrdal, D.; Walters, M.; Dautenhahn, K. Living with Robots: Investigating the Habituation Effect in Participants' Preferences During a Longitudinal Human-Robot Interaction Study. Proceedings of the 16th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2007), 26– 29 August 2007; pp. 564–569.
[37]
Mumm, J.; Mutlu, B. Human-Robot Proxemics. Proceedings of the 6th International Conference on Human-Robot Interaction (HRI '11), Lausanne, Switzerland, 6–9 March 2011; p. 331.
[38]
Henkel, Z. Towards A Computational Method of Scaling A Robot's Behavior via Proxemics. Proceedings of the AAAI Fall Symposium Series, Arlington, VA, USA, 11–13 November 2010.
[39]
Salichs, M.; Barber, R.; Khamis, A.; Malfaz, M.; Gorostiza, J.; Pacheco, R.; Rivas, R.; Corrales, A.; Delgado, E.; Garcia, D. Maggie: A Robotic Platform for Human-Robot Social Interaction, Proceedings of the 2006 IEEE Conference on Robotics, Automation and Mechatronics, Bangkok, Thailand, 1– 3 June 2006; pp. 1–7.
[40]
Barber, R.; Salichs, M. A New Human Based Architecture for Intelligent Autonomous Robots. Proceedings of the Intelligent Autonomous Vehicles 2001 (IAV 2001): A Proceedings Volume from the 4th IFAC Symposium, Sapporo, Japan, 5–7 September 2001; Pergamon, 2002; p. 81.
[41]
Gorostiza, J.; Barber, R.; Khamis, A.; Malfaz, M.; Pacheco, R.; Rivas, R.; Corrales, A.; Delgado, E.; Salichs, M. Multimodal Human-Robot Interaction Framework for a Personal Robot. Proceedings of the 15th IEEE International Symposium on Robot and Human Interactive Communication, (ROMAN 2006), Hatfield, UK, 6– 8 September 2006; pp. 39–44.
[42]
Gonzalez-Pacheco, V.; Ramey, A.; Alonso-Martin, F.; Castro-Gonzalez, A.; Salichs, M.A. Maggie: A social robot as a gaming platform. Int. J. Soc. Robot. 2011, 3, 371–381, doi:10.1007/s12369-011-0109-8.
[43]
Williams, J. Personal space and its relation to extraversion-introversion. Can. J. Behav. Sci. /Revue Canadienne des sciences du comportement 1971, 3, 156–160, doi:10.1037/h0082257.
[44]
Fisher, J.; Byrne, D. Too close for comfort: Sex differences in response to invasions of personal space. J. Personal. Soc. Psychol. 1975, 32, 15–21, doi:10.1037/h0076837.
[45]
Rae, A.; Khamis, A.; Basir, O.; Kamel, M. Particle filtering for bearing-only audio-visual speaker detection and tracking. Proceedings of the 2009 3rd International Conference on Signals, Circuits and Systems (SCS), Djerba, Tunisia, 6–8 November 2009; pp. 1–6.
[46]
Falb, J.; Rock, T.; Arnautovic, E. Using Communicative Acts in Interaction Design Specifications for Automated Synthesis of User Interfaces. Proceedings of the 21st IEEE /ACM International Conference on Automated Software Engineering (ASE'06), Tokyo, Japan, 18– 22 September 2006; pp. 261–264.
[47]
Falb, J.; Popp, R.; Rock, T.; Jelinek, H.; Arnautovic, E.; Kaindl, H. Fully-Automatic Generation of User Interfaces for Multiple Devices from a High-Level Model Based on Communicative Acts. Proceedings of the 2007 40th Annual Hawaii International Conference on System Sciences (HICSS'07), Big Island, HI, USA, 3–6 January 2007; pp. 26–26.
[48]
Zaslavsky, A. Communicative Acts of Elvin-Enhanced Mobile Agents. Proceedings of the IEEE /WIC International Conference on Intelligent Agent Technology (IAT2003), Halifax, Canada, 13– 17 October 2003; pp. 446–449.
