基于改进BERT的知识图谱问答研究
Research on Knowledge-Based Question Answering Based on Improved Bert

DOI: 10.12677/CSA.2020.1012250, PP. 2361-2370

Keywords: 知识图谱问答，命名实体识别，BERT，属性抽取
Knowledge-Based Question Answering, Named Entity Recognition, BERT, Attribute Extraction

Full-Text   Cite this paper   Add to My Lib

Abstract:

基于知识图谱的自动问答是自然语言处理领域的研究热点之一。针对现有的中文开放领域知识库问答，本文将知识图谱问答过程分为实体识别、属性抽取和答案检索三个步骤。首先采用改进BERT结合BiLSTM-CRF的命名实体识别模型来提取问句中的相关实体，然后采用改进BERT结合softmax的分类模型进行属性抽取，最后利用前两步的结果进行答案检索。实验结果显示，该方法在NLPCC-ICCPOL的KBQA数据集上取得了97.54%的F1分数。
Automatic Question Answering Based on knowledge map is one of the research hotspots in the field of natural language processing. In this paper, the process of knowledge extraction is divided into three steps: entity recognition, attribute extraction and answer retrieval. Firstly, the named entity recognition model based on improved Bert combined with BiLSTM-CRF is used to extract the related entities in the question, and then the classification model of improved Bert combined with soft-max is used for attribute extraction. Finally, the results of the first two steps are used for answer retrieval. The experimental results show that the method achieves 97.54% F1 score on the KBQA dataset of NLPCC-ICCPOL.

References

[1]	Park, S., Kwon, S., Kim, B., et al. (2015) Question Answering System using Multiple Information Source and Open Type Answer Merge. Proceedings of the 2015 Conference of the North American Chapter of the Association for Com-putational Linguistics: Demonstrations, Denver, June 2015, 111-115. https://doi.org/10.3115/v1/N15-3023
[2]	Hristovski, D., Dinevski, D., Kastrin, A. and Rindflesch, T.C. (2015) Biomedical Question Answering Using Semantic Relations. BMC Bioinformatics, 16, Article No. 6. https://doi.org/10.1186/s12859-014-0365-3
[3]	Zhao, S., Zheng, Y., Zhu, C., et al. (2016) Semantic Computation in Geography Question Answering. 2016 12th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), Changsha, 13-15 August 2016, 1572-1576. https://doi.org/10.1109/FSKD.2016.7603410
[4]	Liang, P., Jordan, M.I. and Klein, D. (2013) Lambda Depend-ency-Based Compositional Semantics. Computational Linguistics, 39, 389-446. https://doi.org/10.1162/COLI_a_00127
[5]	Yao, X. and Van Durme, B. (2014) Information Extraction over Structured Data: Question Answering with Freebase. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, June 2014, 956-966. https://doi.org/10.3115/v1/P14-1090
[6]	Dong, L., Wei, F., Zhou, M. and Xu, K. (2015) Question Answering over Freebase with Multi-Column Convolutional Neural Networks. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics & the 7th Interna-tional Joint Conference on Natural Language Processing, Beijing, July 2015, 260-269. https://doi.org/10.3115/v1/P15-1026
[7]	Liu, L. and Wang, D.B. (2018) A Review on Named Entity Recognition. Journal of the China Society for Scientific and Technical Information, 37, 329-340.
[8]	Lample, G., Ballesteros, M., Subramanian, S., et al. (2016) Neural Architectures for Named Entity Recognition. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, June 2016, 260-270. https://doi.org/10.18653/v1/N16-1030
[9]	Strubell, E., Verga, P., Belanger, D., et al. (2017) Fast and Accurate Entity Recognition with Iterated Dilated Convolutions. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, September 2017, 2670-2680. https://doi.org/10.18653/v1/D17-1283
[10]	Ma, X. and Hovy, E. (2016) End-to-End Sequence Labeling via Bi-directional LSTM-CNNs-CRF. Proceedings of the 54th Annual Meeting of the Association for Computational Lin-guistics (Volume 1: Long Papers), Berlin, August 2016, 1064-1074. https://doi.org/10.18653/v1/P16-1101
[11]	Peters, M.E., Neumann, M., Iyyer, M., et al. (2018) Deep Contextual-ized Word Representations. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), New Orleans, June 2018, 2227-2237. https://doi.org/10.18653/v1/N18-1202
[12]	Devlin, J., Chang, M.W., Lee, K., et al. (2018) BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding.
[13]	Vaswani, A., Shazeer, N., Parmar, N., et al. (2017) Attention Is All You Need. 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, 5998-6008.
[14]	Jawahar, G., Sagot, B. and Seddah, D. (2019) What Does BERT Learn about the Struc-ture of Language? Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, July 2019, 3651-3657. https://doi.org/10.18653/v1/P19-1356
[15]	Loshchilov, I. and Hutter, F. (2019) Decoupled Weight Decay Regular-ization. International Conference on Learning Representations, 1-8.
[16]	王玥, 张日崇. 基于动态规划的知识库问答方法[J]. 郑州大学学报(理学版), 2019, 51(4): 37-42.
[17]	周博通, 孙承杰, 林磊, 刘秉权. 基于LSTM的大规模知识库自动问答[J]. 北京大学学报(自然科学版), 2018, 54(2): 286-292.
[18]	Lai, Y., Jia, Y., Lin, Y., Feng, Y. and Zhao, D. (2018) A Chinese Question Answering System for Single-Relation Factoid Questions. In: Huang, X., Jiang, J., Zhao, D., Feng, Y. and Hong, Y., Eds., Natural Language Processing and Chinese Computing. Lecture Notes in Com-puter Science, Vol. 10619, Springer, Cham, 124-135. https://doi.org/10.1007/978-3-319-73618-1_11
[19]	张芳容, 杨青. 知识库问答系统中实体关系抽取方法研究[J]. 计算机工程与应用, 2020, 56(11): 219-224.
[20]	吴天波, 刘露平, 罗晓东, 卿粼波, 何小海. 基于弱依赖信息的知识库问答[J]. 计算机工程, 2020, 1-8. https://doi.org/10.19678/j.issn.1000-3428.0058312

Full-Text