|
|
后ChatGPT时代计算语言学和自然语言处理研究方法
|
Abstract:
本研究全方位地探索了ChatGPT在自然语言处理各个子领域的应用及其产生的影响，涵盖了问答系统、对话系统、文本生成、机器翻译，以及语音识别和生成等方面。研究结果表明，ChatGPT不仅在上述任务中提升了性能，开拓了新的研究和应用途径，同时也为ChatGPT后的计算语言学和自然语言处理领域带来了新的挑战和机遇。本研究详尽地讨论了如何优化理解和生成自然语言的方法，如何更有效地处理低资源语言和特定领域的问题。此外，本文还对未来的研究提出了一些建议和期待，强调了模型解释性和可靠性的重要性，以及关注模型对社会产生的影响的必要性。期望本研究的发现和建议能为ChatGPT后的计算语言学和自然语言处理研究提供有益的参考和启示。
This study explores the application and impact of ChatGPT in various subfields of natural language processing, encompassing question answering systems, dialogue systems, text generation, machine translation, as well as speech recognition and generation. The findings suggest that ChatGPT not only enhances performance in these tasks and paves the way for new research and application pathways, but also presents new challenges and opportunities for computational linguistics and natural language processing in the post-ChatGPT era. The study discusses in detail how to optimize the understanding and generation of natural language, and how to more effectively handle low-resource languages and specific domains. In addition, this paper proposes suggestions and ex-pectations for future research, emphasizing the importance of model interpretability and reliability, as well as the necessity to pay attention to the social impact of language models. It is hoped that the findings and suggestions of this study can provide valuable references and insights for research in computational linguistics and natural language processing in the post-ChatGPT era.
| [1] | Ponte, J.M. and Croft, W.B. (2017) A Language Modeling Approach to Information Retrieval. ACM SIGIR Forum, 51, 202-208. https://doi.org/10.1145/3130348.3130368 |
| [2] | Ding, N., Qin, Y.J., Yang, G., et al. (2003) Parame-ter-Efficient Fine-Tuning of Large-Scale Pre-Trained Language Models. Nature Machine Intelligence, 5, 220-235. https://doi.org/10.1038/s42256-023-00626-4 |
| [3] | Haleem, A., Javaid, M. and Singh, R.P. (2022) An Era of ChatGPT as a Significant Futuristic Support Tool: A Study on Features, Abilities, and Challenges. Bench Council Transactions on Benchmarks, Standards and Evaluations, 2, Article ID: 100089. https://doi.org/10.1016/j.tbench.2023.100089 |
| [4] | Kuhn, R. and De Mori, R. (1990) A Cache-Based Natural Lan-guage Model for Speech Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12, 570-583. https://doi.org/10.1109/34.56193 |
| [5] | Brown, P., Cocke, J., Pietra, S.A., et al. (1990) A Statistical Approach to Machine Translation. Computational Linguistics, 16, 79-85. |
| [6] | Brown, P.F., Della Pietra, V.J., Desouza, P.V., et al. (1992) Class-Based n-gram Models of Natural Language. Computational Linguistics, 18, 467-480. |
| [7] | Vaswani, A., Shazeer, N., Parmar, N., et al. (2017) Attention Is All You Need. In: Proceedings of the 30th Annual Conference on Neural Information Processing Systems, Curran Associates, New York, 5990-6008. |
| [8] | Devlin, J., Chang, M.W., Lee, K., et al. (2018) Bert: Pre-Training of Deep Bidirectional Transformers for Language Understanding. |
| [9] | Ding, N., Qin, Y.J., Yang, G., et al. (2023) Parameter-Efficient Fine-Tuning of Large-Scale Pre-Trained Language Models. Nature Machine Intelligence, 5, 220-235. https://doi.org/10.1038/s42256-023-00626-4 |
| [10] | Chung, H.W., Hou, L., Longpre, S., et al. (2002) Scaling Instruction-Fine-Tuned Language Models. |
| [11] | Wang, Z.Z., Xie, Q.M., Ding, Z.X., et al. (2023) Is ChatGPT a Good Sentiment Analyzer? A Preliminary Study.
https://arxiv.org/pdf/2304.04339.pdf |
| [12] | Peters, M., Neumann, M. and Iyyer, M. (2018) Deep Contextualized Word Representations. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computa-tional Linguistics: Human Language Technologies, Volume 1, 2227-2237. https://doi.org/10.18653/v1/N18-1202 |
| [13] | Mauldin, M.L. (1984) Semantic Rule Based Text Generation. Proceed-ings of the 10th International Conference on Computational Linguistics and the 22nd Annual Meeting of the Association for Computational Linguistics, Stanford, 2-6 July 1984, 376-380. https://doi.org/10.3115/980491.980568 |
| [14] | Zhang, T.Y., Ladhak, F., Durmus, E., et al. (2023) Benchmarking Large Language Models for News Summarization. |
| [15] | 宗成庆. 统计自然语言处理[M]. 第2版. 北京: 清华大学出版社, 2013. |
| [16] | Ouyang, L., Wu, J., Jiang, X., et al. (2022) Training Language Models to Follow Instructions with Human Feedback. 36th Conference on Neural Information Processing Systems (NeurIPS 2022), New Orleans, 28 No-vember-9 December 2022, 27730-27744. |
| [17] | Cohen, J.P., Luck, M. and Honari, S. (2018) Distribution Matching Losses Can Hallucinate Features in Medical Image Translation. Medical Image Computing and Computer Assisted In-tervention—MICCAI 2018: 21st International Conference, Granada, 16-20 September 2018, 529-536. https://doi.org/10.1007/978-3-030-00928-1_60 |
| [18] | Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y. and Fung, P. (2023) Survey of Hallucination in Natural Language Generation. ACM Computing Surveys, 55, Article No. 248. https://doi.org/10.1145/3571730 |
