%0 Journal Article
%T 基于Transformer的自然语言处理模型综述<br>A Survey of Transformer-Based Natural Language Processing Models
%A 赖鸣姝
%J Artificial Intelligence and Robotics Research
%P 219-225
%@ 2326-3423
%D 2023
%I Hans Publishing
%R 10.12677/AIRR.2023.123025
%X 自然语言处理是计算机科学中深度学习领域的一个分支，旨在使计算机能够理解、解析或生成人类语言(包括文字、音频等)。本文主要介绍了自然语言处理(Natural Language Processing, NLP)中基于Transformer结构所衍生出的多种类型的模型。近年，随着深度学习技术的快速发展，自然语言处理模型的性能也得到了极大的提升，更多的自然语言处理任务得到了更好的解决。这些进展主要得益于神经网络模型的不断发展。本文讲解了当前最为流行的基于Transformer的几类自然语言处理模型，包括BERT (Bidirectional Encoder Representations from Transformers)系列、GPT (Generative Pre-trained Transformer)系列和T5系列等。主要介绍了上述系列的模型各自的发展变化以及其在模型结构，设计思路等方面的区别与联系。同时，对于自然语言处理领域未来的发展方向进行了展望。<br />
Natural language processing is a subfield of deep learning in computer science that aims to enable computers to understand, parse, or generate human language (text, audio, etc.). This paper mainly introduces various types of models derived from the Transformer structure in Natural Language Processing (NLP). In recent years, with the rapid development of deep learning technology, the performance of natural language processing models has also been greatly improved, and more natural language processing tasks have been better solved. These advances are mainly due to the continuous development of neural network models. This article explains the most popular Transformer-based natural language processing models. These include BERT (Bidirectional Encoder Representations from Transformers) family, GPT (Generative Pre-trained Transformer) family, the T5 family, etc. This paper mainly introduces the development and changes of the above series of models, as well as their differences and connections in model structure, design ideas and other aspects. At the same time, the future development direction of natural language processing is prospected.
%K 人工智能，深度学习，自然语言处理<br>Artificial Intelligence
%K Deep Learning
%K Natural Language Processing
%U http://www.hanspub.org/journal/PaperInformation.aspx?PaperID=70392