全部 标题 作者
关键词 摘要

OALib Journal期刊
ISSN: 2333-9721
费用:99美元

查看量下载量

相关文章

更多...

Word2vec的核心架构及其应用

, PP. 43-48

Keywords: 自然语言处理,Word2vecCBOWSkip-gram,中文语言处理

Full-Text   Cite this paper   Add to My Lib

Abstract:

神经网络概率语言模型是一种新兴的自然语言处理算法,该模型通过学习训练语料获得词向量和概率密度函数,词向量是多维实数向量,向量中包含了自然语言中的语义和语法关系,词向量之间余弦距离的大小代表了词语之间关系的远近,词向量的加减代数运算则是计算机在“遣词造句”.近年来,神经网络概率语言模型发展迅速,Word2vec是最新技术理论的合集.首先,重点介绍Word2vec的核心架构CBOW及Skip-gram;接着,使用英文语料训练Word2vec模型,对比两种架构的异同;最后,探讨了Word2vec模型在中文语料处理中的应用.

References

[1]  Bengio Y,Ducharme R,Vincent P. A neural probabilistic language model[J]. Journal of Machine Learning Research,2003,3(7):1 137-1 155.
[2]  Michael U G,AapoHy(¨overv)rinen. Noise-contrastive estimation of unnormalized statistical models,with applications to natural image statistics[J]. The Journal of Machine Learning Research,2012,13(2):307-361.
[3]  Tomas M,Chen K,Corrado G. Efficient estimation of word representations in vector space[EB/OL].(2013-08-18)[2013-09-07]http://arxiv.org/abs/1301.3781.
[4]  Bengio Y,LeCun Y. Scaling Learning Algorithms Towards AI[M]//Large-Scale Kernel Machines. Cambridge:MIT Press,2007.
[5]  Mikolov T,Karafi M,Burget L,et al. Recurrent neural network based language model[C]//Proceedings of Interspeech. Chiba,Japan:MIT Press,2010:131-138.
[6]  Mikolov T,Ilya S,Kai C,et al. Distributed representations of words and phrases and their compositionality[EB/OL]. [2013-10-16]http://arxiv.org/abs/1310.4546.
[7]  Elman J. Finding structure in time[J]. Cognitive Science,1990,14(7):179-211.
[8]  Rumelhart D E,Hinton G E,Williams R J. Learning internal representations by back-propagating errors[J]. Nature,1986,323(9):533-536.
[9]  李雷. 基于人工智能机器学习的文字识别方法研究[D]. 成都:电子科技大学机械电子工程学院,2013.
[10]  Li Lei. Character recognition research based on artificial intelligence and machine learning[D]. Chengdu:School of Mechatronics Engineering of University of Electronics Science and Technology of China,2013.(in Chinese)
[11]  Andriy M,Yee W T. A fast and simple algorithm for training neural probabilistic language models[EB/OL].(2009-10-12)[2012-06-10]http://arxiv.org/ftp/arxiv/papers/12061.
[12]  Frederic M,Yoshua B. Hierarchical probabilistic neural network language model[C]//Proceedings of the International Workshop on Artificial Intelligence and Statistics. Barbados:MIT Press,2005:246-252.
[13]  Mikolov T,Kopeck J,Burget L,et al. Neural network based language models for highly inflective languages[C]//Proc. ICASSP. Taipei:ICA,2009:126-129.
[14]  Hinton G E,McClelland J L,Rumelhart D E. Distributed Representations[M]//Parallel Dis-Tributed Processing:Explorations in the Microstructure of Cognition. Cambridge:MIT Press,1986.
[15]  许炎,金芝,李戈,等. 基于多Web信息源的主体概念网络获取[J]. 计算机研究与发展,2013,50(9):1 843-1 854.
[16]  Xu Yan,Jin Zhi,Li Ge,et al. Acquiring topical concept network from multiple Web information sources[J]. Journal of Computer Research and Development,2013,50(9):1 843-1 854.(in Chinese)

Full-Text

Contact Us

service@oalib.com

QQ:3279437679

WhatsApp +8615387084133