This study compares the translation outputs of an English into Arabic text using the three machine translators of Google Translate, Microsoft Bing, and Ginger. To carry this evaluation of the machine translation (MT) outputs, an English text and its Arabic counterpart were selected from the UN records. The English source text was segmented into 84 semantic chunks. Depending on the Arabic counterpart model text, each chunk was rated as “correct or incorrect” at the two levels of the translation attributes: fidelity and intelligibility. To perform the quantitative description of the evaluation process, the numbers of fidelity and intelligibility errors and their percentages were calculated. Results of this evaluation process revealed that none of the three translated versions of the source text was perfectly translated. Although the translation of Microsoft Bing was rated the best, Google’s translation was found the least accurate due to the high percentage of fidelity and intelligibility errors detected in its translation output. However, the quality of Ginger’s translation was found slightly less accurate than that of Microsoft Bing, but remarkably better than Google’s translation. The findings of this study imply that these MT applications can be implemented to perform English into Arabic translation to get the broad gist of a source text, but a deep and thorough post-editing process looks essential for a full and accurate understanding of an English into Arabic MT output. The study recommends that more studies are encouraged to continue to assess the quality of MT that will further highlight its weaknesses and the strategies that should be adopted to overcome them.
References
[1]
Alawneh, M. F., & Sembok, T. M. (2011). Rule-Based and Example-Based Machine Translation from English to Arabic. In 2011 Sixth International Conference on Bio-Inspired Computing: Theories and Applications (pp. 343-347).
[2]
Albat, T. F. (2012). Systems and Methods for Automatically Estimating a Translation Time. US Patent 0185235, 19 July 2012.
[3]
Ali, M. A. (2016). Artificial Intelligence and Natural Language Processing: The Arabic Corpora in Online Translation Software. International Journal of Advanced and Applied Science, 3, 59-66. https://doi.org/10.21833/ijaas.2016.09.010 http://science-gate.com/IJAAS/Articles/2016-3-9/10%202016-3-9-pp.59-66.pdf
[4]
Ali, M. A. (2018). The Human Intelligence vs. Artificial Intelligence: Issues and Challenges in Computer Assisted Language Learning. International Journal of English Linguistics, 8, 259-271. http://www.ccsenet.org/journal/index.php/ijel/article/view/75235 https://doi.org/10.5539/ijel.v8n5p259
[5]
Al-Khresheh, M. (2016). A Review Study of Contrastive Analysis Theory. Journal of Advances in Humanities and Social Sciences, 2, 330-338. https://doi.org/10.20474/jahss-2.6.5
[6]
Al-Khresheh, M. H., & Almaaytah, S. A. (2018). English Proverbs into Arabic through Machine Translation. International Journal of Applied Linguistics & English Literature, 7, 158-166. https://doi.org/10.7575/aiac.ijalel.v.7n.5p.158
[7]
Anderson, D. D. (1995). Machine Translation as a Tool in Second Language Learning. CALICO Journal, 13, 68-96.
[8]
Belam, J. (2003). Teaching Machine Translation Evaluation by Assessed Project Work. In 6th EAMT Workshop Teaching Machine Translation (pp. 131-136). Manchester.
[9]
Bowker, L. (2014). Can Machine Translation Facilitate Outreach to Newcomers? A Pilot Study Investigating the Needs of Spanish-Speaking Users of the Ottawa Public Library. 2013 OCLC/ALISE Research Grant Report Published Electronically by OCLC Research. http://www.oclc.org/research/grants/reports/2013/bowker2013.pdf
[10]
Cieslak, M. (2011). The Scope and Limits of Machine Translation. eLingUp [Centro de Linguística da Universidade do Porto], 3, 156-174. http://193.137.34.194/index.php/elingUP/article/download/2528/2316
[11]
Daniele, F. (2019). Performance of an Automatic Translator in Translating Medical Abstracts. Heliyon, 5, e02687. https://doi.org/10.1016/j.heliyon.2019.e02687 https://www.researchgate.net/publication/336706218_Performance_of_an_automatic _translator_in_translating_medical_abstracts
[12]
Doherty, S. (2016). The Impact of Translation Technologies on the Process and Product of Translation. International Journal of Communication, 10, 947-969. https://www.researchgate.net/publication/284725157_The_impact_of_translation_ technologies_on_the_process_and_product_of_translation
[13]
Fiederer, R., & O’Brien, S. (2009). Quality and Machine Translation: A Realistic Objective? The Journal of Specialised Translation, 11, 52-74.
[14]
Forner, M., & White, J. S. (2001). Predicting MT Fidelity from Noun-Compound Handling. In Workshop on MT Evaluation, “Who Did What to Whom?”. Mt Summit VIII, Santiago de Compostela, Spain.
[15]
Hacioglu, K., & Wayne, W. (2003). Target Word Detection: Semantic Role Chunking Using Support Vector Machines. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology: Companion Volume of the Proceedings of HLT-NAACL 2003—Short Papers (Vol. 2, pp. 25-27). HIL-NAACL. http://dl.acm.org/citation.cfm?id=1073483.1073492 https://doi.org/10.3115/1073483.1073492
[16]
Han, A. L. F., Wong, D. F., & Chao, L. S. (2012). LEPOR: A Robust Evaluation Metric for Machine Translation with Augmented Factors. In Proceedings of the 24th International Conference on Computational Linguistics (COLING 2012) (pp. 441-450). Posters, Mumbai, India. https://github.com/aaronlifenghan/aaron-project-lepor
[17]
Hirschman, L., Reeder, F., Burger, J., & Miller, K. (2000). Name Translation as a Machine Translation Evaluation Task. In Proceedings of the Workshop on Machine Translation Evaluation (pp. 21-28). New Jersey: LREC-2000. https://www.jostrans.org/issue11/art_fiederer_obrien.php
[18]
Hutchins, J., & Somers, H. (1992). An Introduction to Machine Translation. London: Academic Press Limited.
[19]
Kliffer, M. D. (2005). An Experiment in MT Post-Editing by a Class of Intermediate/Advanced French Majors. In Proceedings EAMT 10th Annual Conference (pp. 160-165). Budapest.
[20]
La Torre, M. D. (1999). A Web-Based Resource to Improve Translation Skills. ReCALL, 11, 41-49.
[21]
Lee, S.-M. (2019). The Impact of Using Machine Translation on EFL Students’ Writing. Computer Assisted Language Learning, 33, 157-175. https://doi.org/10.1080/09588221.2018.1553186
[22]
Niño, A. (2004). Recycling MT: A Course on FL Writing via MT Post-Editing (pp. 179-187). Paper Presented at CLUK (Computational Linguistics United Kingdom 7th Annual Research Colloquium), 6th and 7th January 2004 in the University of Birmingham, UK. https://scholar.google.com/scholar?q=Ni%C3%B1o+A+.+(2004)+Recycling+MT:+A+cour se+on+FL+writing+via+MT+post-editing.+Paper+presented+at+CLUK+(Computational+ Linguistics+United+Kingdom+7th+Annual+Research+Colloquium)+6th+and+7th+January+2004 +in+the+University+of+Birmingham+UK
[23]
Niño, A. (2009). Machine Translation in Foreign Language Learning: Language Learners’ and Tutors’ Perceptions of Its Advantages and Disadvantages. ReCALL, 21, 105-122. https://doi.org/10.1017/S0958344009000172
[24]
Papineni, K., Roukos, S., Ward, T., & Zhu, W. J. (2002). BLEU: A Method for Automatic Evaluation of Machine Translation. In ACL-2002: 40th Annual Meeting of the Association for Computational Linguistics (pp. 311-318). Stroudsburg, https://doi.org/10.3115/1073083.1073135
[25]
Taleghani, M., & Pazouki, E. (2018). Free Online Translators: A Comparative Assessment in Terms of Idioms and Phrasal Verbs. International Journal of English Language & Translation Studies, 6, 15-19.
[26]
Turian, J., Shen, L., & Melamed, I. D. (2003). Evaluation of Machine Translation and Its Evaluation. In Proceedings of the MT Summit IX (pp. 386-393).
[27]
White, J. (1995). Approaches to Black-Box Machine Translation Evaluation. In Proceedings of MT Summit 1995 (pp. 386-393). New Orleans, USA.
[28]
White, J. (2001). Predicting Intelligibility from Fidelity. In Proceedings of the Workshop on Evaluation. MT Summit VI, Santiago, Spain. http://www.eamt.org/events/summitVIII/papers/white-1.pdf
[29]
White, J. S. (2003). How to Evaluate Machine Translation. In H. Somers (Ed.), Computers and Translation (pp. 211-244). Amsterdam: John Benjamins. https://doi.org/10.1075/btl.35.16whi
[30]
Wilss, W. (1982). The Science of Translation. Problems and Methods. Vol. 80, Tubingen: Narr.
[31]
Zong, Z. (2018). Research on the Relations between Machine Translation and Human Translation. Journal of Physics: Conference Series, 1087, Article ID: 062046. https://doi.org/10.1088/1742-6596/1087/6/062046 https://iopscience.iop.org/article/10.1088/1742-6596/1087/6/062046/pdf
