This paper describes the evaluation of an educational game designed to give learners of foreign languages the opportunity to practice their spoken language skills. Within the speech interactive Computer-Assisted Language Learning (CALL) program, scenarios are presented in which learners interact with virtual characters in the target language using speech recognition technology. Two types of interactive scenarios with virtual characters are presented as part of the game: the one-to-one scenarios which take the form of practice question and answer scenarios where the learner interacts with one virtual character and the interactive scenario which is an immersive contextualised scenario where the learner interacts with two or more virtual characters within the scene to complete a (task-based) communicative goal. The study presented here compares learners’ subjective attitudes towards the different scenarios. In addition, the study investigates the performance of the speech recognition component in this game. Forty-eight students of English as a Foreign Language (EFL) took part in the evaluation. Results indicate that learners’ subjective ratings for the contextualised interactive scenario are higher than for the one-to-one, practice scenarios. In addition, recognition performance was better for these interactive scenarios. 1. Introduction When learning a foreign language, opportunities for interaction in the target language can be limited. Unlike most other school subjects, language learning requires oral practice. A student studying a language in high school may receive only a few hours of language class per week and may have very limited one-on-one time with the teacher of the class. In the classroom situation, it may not be possible, due to time restrictions and resources, for the teacher to engage in a spoken dialogue with every student. However, it is necessary for language learning that the learner has an interlocutor with whom to interact. It has been found that classroom exercises which are detached from real-life issues or activities fail to help the learner use the target language [1, 2]. Learning activities therefore focus less on the (correct) use of forms in a context-free learning environment in favour of using the language for a communicative purpose. Games are used in language learning to stimulate motivation and to create communicative opportunities for learners [3]. Interactivity and individual action are fundamental properties of games and can be related to communicative approaches to language learning [4], where the focus is on
References
[1]
A. Firth and J. Wagner, “On discourse, communication, and (some) fundamental concepts in SLA research,” The Modern Language Journal, vol. 81, no. 3, pp. 285–300, 1997.
[2]
L. van Lier, “From input to affordance: social-interactive learning from an ecological perspective,” in Sociocultural Theory and Second Language Learning, J. P. Lantolf, Ed., pp. 245–259, Oxford University Press, Oxford, UK, 2000.
[3]
M. Warschauer and D. Healey, “Computers and language learning: an overview,” Language Teaching, vol. 31, pp. 57–71, 1998.
[4]
B. H. S？rensen and B. Meyer, “Serious games in language learning and teaching—a theoretical perspective,” in Proceedings of the 3rd International Conference of the Digital Games Research Association, pp. 559–566, Tokyo, Japan, 2007.
[5]
J. C. Richards and T. S. Rogers, Approaches and Methods in Language Teaching, Cambridge University Press, Cambridge, UK, 1986.
[6]
R. Scarcella and D. Crookall, “Simulation/gaming and language acquisition,” in Simulation, Gaming and Language Learning, D. Crookall and R. L. Oxford, Eds., pp. 223–230, Newbury House, New York, NY, USA, 1990.
[7]
L. van Lier, “An ecological-semiotic perspective on language and linguistics,” in Language Acquisition and Language Socialization: Ecological Perspectives, C. Kramsch, Ed., pp. 140–164, Continuum, London, UK, 2002.
[8]
K. Facer, “Computer games and learning,” NESTA Futurlab Discussion Paper, pp. 1–11, 2005, http://www.nestafuturelab.org/research/discuss/02discuss01.htm.
[9]
M. H. Long, “The role of the linguistic environment in second language acquisition,” in Handbook of Second Language Acquisition, W. C. Ritchie and T. K. Bhatia, Eds., pp. 413–468, Academic Press, New York, NY, USA, 1996.
[10]
M. Swain, “Three functions of output in second language learning,” in Principle and Practice in Applied Linguistics: Studies in Honour of H. G. Widdowson, G. Cook and B. Seidlhofer, Eds., pp. 125–144, Oxford University Press, Oxford, UK, 1995.
[11]
C. A. Chapelle, “Multimedia CALL: lessons to be learned from research on instructed SLA,” Language Learning & Technology, vol. 2, no. 1, pp. 22–34, 1998.
[12]
H. Morton, N. Davidson, and M. A. Jack, “Evaluation of a speech interactive CALL system,” in Handbook of Research on Computer-Enhanced Language Acquisition and Learning, F. Zhang and B. Barber, Eds., 2008.
[13]
H. Morton and M. Jack, “Speech interactive computer-assisted language learning: a cross-cultural evaluation,” Computer Assisted Language Learning, vol. 23, no. 4, pp. 295–319, 2010.
[14]
H. Morton, N. Gunson, and M. Jack, “Attitudes to subtitle duration and the effect on user responses in speech interactive foreign language learning,” Journal of Multimedia, vol. 6, no. 5, pp. 436–446, 2011.
[15]
J. Cassell, “Nudge, nudge, wink, wink: elements of face-to-face conversation for embodied conversational agents,” in Embodied Conversational Agents, J. Cassell, J. Sullivan, S. Prevost, and E. Churchill, Eds., pp. 1–27, MIT Press, Cambridge, Mass, USA, 2000.
[16]
W. L. Johnson, S. Choi, S. Marsella, N. Mote, S. Narayanan, and H. Vilhjálmsson, “Tactical language training system: supporting the rapid acquisition of foreign language and cultural skills,” in Proceedings of the InSTIL/ICALL Symposium on Computer Assisted Language Learning, pp. 21–24, Venice, Italy, 2004.
[17]
W. L. Johnson and E. Shaw, “Using agents to overcome difficulties in web-based courseware,” in Workshop on Intelligent Educational Systems on the World Wide Web (AI-ED '97), pp. 1–8, Kobe, Japan, 1997.
[18]
W. L. Johnson, J. W. Rickel, and J. C. Lester, “Animated pedagogical agents: face-to-face interaction in interactive learning environments,” International Journal of Artificial Intelligence in Education, vol. 11, pp. 47–78, 2000.
[19]
W. L. Johnson and A. Valente, “Tactical language and culture training systems: Using artificial intelligence to teach foreign languages and cultures,” in Proceedings of the 23rd AAAI Conference on Artificial Intelligence and the 20th Innovative Applications of Artificial Intelligence Conference (AAAI '08/IAAI '08), pp. 1632–1639, Chicago, Ill, USA, July 2008.
[20]
J. C. Lester, S. A. Converse, S. E. Kahler, S. T. Barlow, B. A. Stone, and R. S. Bhogal, “Animated pedagogical agents and problem-solving effectiveness: a large-scale empirical evaluation,” in Proceedings of the 8th World Conference on Artificial Intelligence in Education, pp. 23–30, 1997.
[21]
R. Moreno, R. E. Mayer, and J. C. Lester, “Life-like pedagogical agents in constructivist multimedia environments: Cognitive consequences of their interaction,” in Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications, pp. 741–746, AACE Press, Charlottesville, VA, USA, 2000.
[22]
W. L. Johnson, J. W. Rickel, and J. C. Lester, “Animated pedagogical agents: face-to-face interaction in interactive learning environments,” International Journal of Artificial Intelligence in Education, vol. 11, pp. 47–78, 2000.
[23]
M. Heim, The Metaphysics of Virtual Reality, Oxford University Press, Oxford, UK, 1994.
[24]
M. Slater, M. Usoh, and A. Steed, “Depth of presence in virtual environments,” Presence, Teleoperators and Virtual Environments, vol. 3, pp. 130–144, 1994.
[25]
C. D. Morris, J. D. Bransford, and J. J. Franks, “Levels of processing versus transfer appropriate processing,” Journal of Verbal Learning and Verbal Behavior, vol. 16, no. 5, pp. 519–533, 1977.
[26]
W. Menzel, D. Herron, R. Morton, D. Pezzotta, P. Bonaventura, and P. Howarth, “Interactive pronunciation training,” ReCALL, vol. 13, no. 1, pp. 67–78, 2001.
[27]
J. Dalby and D. Kewley-Port, “Explicit pronunciation training using automatic speech recognition technology,” CALICO Journal, vol. 16, no. 3, pp. 425–445, 1999.
[28]
M. Eskenazi, “Using a computer in foreign language pronunciation training: what advantages?” CALICO Journal, vol. 16, no. 3, pp. 447–469, 1999.
[29]
J. Murray, “Lessons learned from the Athena language learning project,” in Intelligent Language Tutors: Theory Shaping Technology, V. Holland M, J. D. Kaplan, and M. R. Sams, Eds., pp. 243–256, Lawrence Erlbaum Associates, Mahwah, NJ, USA, 1995.
[30]
J. Bernstein, A. Najmi, and F. Ehsani, “Subarashii: encounters in Japanese spoken language education,” CALICO Journal, vol. 16, no. 3, pp. 361–384, 1999.
[31]
W. G. Harless, M. A. Zier, and R. C. Duncan, “Virtual dialogues with native speakers: the evaluation of an interactive multimedia method,” CALICO Journal, vol. 16, no. 3, pp. 313–337, 1999.
[32]
M. Holland, J. D. Kaplan, and M. Sabol, “Preliminary tests of language learning in a speech-interactive graphics microworld,” CALICO Journal, vol. 16, no. 3, pp. 339–359, 1999.
[33]
H. Strik, F. Cornillie, J. Colp？rt, J. van Doremalen, and C. Cucchiarini, “Developing a CALL system for practicing oral proficiency: how to design for speech technology, pedagogy and learners,” in Proceedings of the SlaTE-2009 Workshop, Warwickshire, UK, 2009.
[34]
T. Pica, “Research on negotiation: what does it reveal about second-language learning conditions, processes, and outcomes?” Language Learning, vol. 44, pp. 493–527, 1994.
[35]
S. Gass, Input, Interaction, and the Second Language Learner, Lawrence Erlbaum Associates, Mahwah, NJ, USA, 1997.
[36]
M. Swain, “Communicative competence: some roles of comprehensible input and comprehensible output in its development,” in Input in Second Language Acquisition, S. Gass and C. Madden, Eds., pp. 235–253, Newbury House Press, Rowley, Mass, USA, 1985.
[37]
K. Wachowicz and B. Scott, “Software that listens: it’s not a question of whether, it’s a question of how,” CALICO Journal, vol. 16, no. 3, pp. 253–276, 1999.
[38]
C. Doughty and J. Williams, “Pedagogical choices in focus on form,” in Focus on Form in Classroom Second Language Acquisition, C. Doughty and J. Williams, Eds., pp. 197–261, Cambridge University Press, Cambridge, UK, 1998.
[39]
S. M. Gass, “Integrating research areas: a framework for second language studies,” Applied Linguistics, vol. 9, no. 2, pp. 198–217, 1988.
[40]
H. Strik, A. Neri, and C. Cucchiarini, “Speech technology for language tutoring,” in Proceedings of the Language and Speech Technology Conference (LangTech '08), pp. 73–76, Rome, Italy, 2008.
[41]
J. Morgan and S. LaRocca, “Making a speech recognizer tolerate non-native speech through Gaussian mixture merging,” in Proceedings of InSTIL/ICALL Symposium on Computer Assisted Language Learning,, pp. 213–216, Venice, Italy, 2004.
[42]
N. Cylwik, A. Wagner, and G. Demenko, “The EURONOUNCE corpus of non-native polish for ASR-based pronunciation tutoring system,” in Proceedings of the SLATE Workshop on Speech and Language Technology in Education, Warwickshire, UK, 2009.
[43]
L. Neumeyer, H. Franco, M. Weintraub, and P. Price, “Automatic text-independent pronunciation scoring of foreign language student speech,” in Proceedings of the International Conference on Spoken Language Processing (ICSLP '96), pp. 1457–1460, October 1996.
[44]
G. Kawai and K. Hirose, “A method for measuring the intelligibility and non-nativeness of phone quality in foreign language pronunciation training,” in Proceedings of the International Conference on Spoken Language Processing (ICSLP '98), pp. 1823–1826, Sydney, Australia, 1998.
[45]
R. Likert, A Technique for the Measurement of Attitudes, Columbia University Press, New York, NY, USA, 1932.
[46]
P. Hubbard, “Interactive participatory dramas for language learning,” Simulation and Gaming, vol. 33, no. 2, pp. 210–216, 2002.
[47]
A. Mitchell and C. Savill-Smith, “The use of computer and video games for learning. A review of the literature, Ultralab,” 2004, http://gmedia.glos.ac.uk/docs/books/computergames4learning.pdf.
[48]
Becta, “Computer games in education,” Project Report, 2001.
[49]
C. Bisson and J. Luckner, “Fun in learning: the pedagogical role of fun in adventure education,” Journal of Experimental Education, vol. 19, no. 2, pp. 108–112, 1996.
[50]
J. P. Gee, “Pleasure, learning, video, games and life: the projective stance,” E-Learning and Digital Media, vol. 2, no. 3, pp. 21–223, 2005.
[51]
M. Prensky, Digital Game-Based Learning, McGraw Hill, New York, NY, USA, 2001.
[52]
K. Salen and E. Zimmerman, Rules of Play: Game Design Fundamentals, MIT Press, Cambridge, Mass, USA, 2004.
[53]
S. Franciosi, “A comparison of computer game and language learning task design using flow theory,” CALL-EJ, vol. 12, no. 1, pp. 1–25, 2011.
[54]
R. Sandford and B. Williamson, Games and Learning: A Handbook From Futurlab, 2005, http://www2.futurelab.org.uk/resources/documents/handbooks/games_and_learning2.pdf.
