|
|
融合关系类型的图卷积网络关系抽取模型
|
Abstract:
关系抽取是信息抽取和检索中的一项重要任务，它旨在从运行的文本中提取给定实体之间的关系。以前的研究表明，要想在这个任务中取得良好的表现，需要对上下文信息进行良好的建模，其中输入句子的依存树可以成为不同类型上下文信息中的一个有益来源。然而，这些研究大多集中在词与词之间的依赖关系上，对依赖类型的利用关注有限。现有的研究大多存在依存树中存在噪声的问题，尤其是在自动生成依存树时，大量利用依赖信息可能会给关系分类带来混乱，因此对依存树进行必要的修剪非常重要。此外，他们在建模时经常平等地对待不同的依赖连接，因此在自动生成的依存树中会受到干扰(不准确的依赖解析)。该论文提出了一种用于关系提取的注意力机制的图卷积神经网络方法，该方法将基于图卷积网络的注意机制应用于依赖解析器获得的依存树中不同上下文单词，以区分不同单词依赖的重要性。又加入一个新定义的模块，将其命名为Key-Value Slot (简称KV Slot)。对于实体中的每个单词，KV Slot模块将所有关联的单词及其之间的依赖性进行映射，然后根据对关系提取的贡献为其分配一个权重。该方法不仅利用了单词之间的依赖连接和类型，而且还将可靠的依赖信息与嘈杂的信息区分开来，在此基础上并对它们进行适当的建模。在SemEval2010-Task8和KBP37数据集上的实验证明了我们的方法的有效性，模型在性能上有了较大提升。
Relation extraction is an important task in information extraction and retrieval, which aims to ex-tract relations between given entities from running text. Previous studies have shown that good performance in this task requires good modeling of contextual information, where the dependency tree of the input sentence can be a beneficial source among different types of contextual infor-mation. However, most of these studies focus on word-to-word dependencies and pay limited atten-tion to the exploitation of dependency types. Most of the existing studies have the problem of noise in the dependency tree, especially when the dependency tree is generated automatically, a large amount of dependency information may bring confusion to the relation classification, so it is very important to do necessary pruning of the dependency tree. Moreover, they often treat different de-pendency connections equally when modeling, and thus suffer from interference (inaccurate de-pendency parsing) in the automatically generated dependency tree. This paper proposes a graph convolutional neural network method for the attention mechanism of relation extraction, which ap-plies the attention mechanism based on graph convolutional network to different context words in the dependency tree obtained by the dependency parser to distinguish the importance of different word dependencies. This paper adds a new module called the Key-Value Slot (KV Slot for short). For each word in an entity, the KV Slot module maps all associated words and dependencies between them, and then assigns it a weight based on its contribution to relation extraction. The proposed method not only exploits the dependency connections and types between words, but also distin-guishes reliable dependency information from noisy information, builds on it and models them ap-propriately. Experiments on Semeval2010-task8 and KBP37 datasets prove the effectiveness of our method, and the performance of the model has been greatly improved.
| [1] | Distiawan, B., Weikum, G., Qi, J.Z. and Zhang, R. (2019) Neural Relation Extraction for Knowledge Base Enrichment. Pro-ceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, July 2019, 229-240. |
| [2] | Sun, K., Zhang, R., Mensah, S., Mao, Y.Y. and Liu, X.D. (2019) Aspect-Level Sentiment Analysis via Convolution over Depend-ency Tree. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Interna-tional Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, November 2019, 5683-5692. https://doi.org/10.18653/v1/D19-1569 |
| [3] | Xu, K., Reddy, S., Feng, Y.S., Huang, S.F. and Zhao, D.Y. (2016) Question Answering on Freebase via Relation Extraction and Textual Evidence. Proceedings of the 54th Annual Meeting of the Associa-tion for Computational Linguistics, Berlin, August 2016, 2326-2336. https://doi.org/10.18653/v1/P16-1220 |
| [4] | Wang, L. and Cardie, C. (2012) Focused Meeting Summarization via Unsupervised Relation Extraction. Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Seoul, July 2012, 304-313. |
| [5] | Zeng, D.J., Liu, K., Lai, S.W., Zhou, G.Y. and Zhao, J. (2014) Relation Classifification via Convolutional Deep Neural Network. Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, Dublin, August 2014, 2335-2344. |
| [6] | Xu, Y., Mou, L.L., Li, G., Chen, Y.C., Peng, H. and Jin, Z. (2015) Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Paths. Proceedings of the 2015 Conference on Empirical Methods in Nat-ural Language Processing, Lisbon, September 2015, 1785-1794. https://doi.org/10.18653/v1/D15-1206 |
| [7] | Zhang, D.X. and Wang, D. (2015) Relation Classifification via Recurrent Neural Network. arXiv: 1508.01006. |
| [8] | Wang, L.L., Cao, Z., De Melo, G. and Liu, Z.Y. (2016) Relation Classifification via Multi-Level Attention CNNs. Proceedings of the 54th Annual Meet-ing of the Association for Computational Linguistics (Volume 1: Long Papers), Berlin, August 2016, 1298-1307. https://doi.org/10.18653/v1/P16-1123 |
| [9] | Zhang, Y.H., Zhong, V., Chen, D.Q., Angeli, G. and Manning, C.D. (2017) Positionaware Attention and Supervised Data Improve Slot Filling. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, September 2017, 35-45. https://doi.org/10.18653/v1/D17-1004 |
| [10] | Miwa, M. and Bansal, M. (2016) End-to-End Relation Extraction Using LSTMs on Sequences and Tree Structures. Proceedings of the 54th Annual Meeting of the Association for Computational Lin-guistics (Volume 1: Long Papers), Berlin, August 2016, 1105-1116. https://doi.org/10.18653/v1/P16-1105 |
| [11] | Zhang, Y.H., Qi, P. and Manning, C.D. (2018) Graph Convolution over Pruned Dependency Trees Improves Relation Extraction. Pro-ceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, October-November 2018, 2205-2215. https://doi.org/10.18653/v1/D18-1244 |
| [12] | Sun, K., Zhang, R.C., Mao, Y.Y., Mensah, S. and Liu, X.D. (2020) Relation Extraction with Convolutional Network over Learnable SyntaxTransport Graph. Proceedings of the AAAI Con-ference on Artificial Intelligence, 34, 8928-8935.
https://doi.org/10.1609/aaai.v34i05.6423 |
| [13] | Chen, G.M., Tian, Y.H., Song, Y. and Wan, X. (2021) Relation Extraction with Type-aware Map Memories of Word Dependencies. In: Zong, C.Q., Xia, F., Li, W.J. and Navigli, R., Eds., Findings of the Association for Computational Linguistics: ACLIJCNLP 2021, Association for Computational Linguistics, Ohio, 2501-2512.
https://doi.org/10.18653/v1/2021.findings-acl.221 |
| [14] | Yu, B.W., Xue, M.G., Zhang, Z.Y., Liu, T.W., Wang, Y.B. and Wang, B. (2020) Learning to Prune Dependency Trees with Rethinking for Neural Relation Extraction. Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, 8-13 December 2020, 3842-3852. |
| [15] | Nasukawa, T. and Yi, J. (2003) ‘Sentiment Analysis: Capturing Favorability Using Natural Language Processing. Proceedings of the 2nd Interna-tional Conference on Knowledge Capture, Sanibel Island, 23-25 October 2003, 70-77.
https://doi.org/10.1145/945645.945658 |
| [16] | Poria, S., Cambria, E., Winterstein, G. and Huang, G.B. (2014) ‘Sentic Pat-terns: Dependency-Based Rules for Concept-Level Sentiment Analysis. Knowledge-Based Systems, 69, 45-63. https://doi.org/10.1016/j.knosys.2014.05.005 |
| [17] | Yang, B. and Cardie, C. (2013) Joint Inference for Fifine-Grained Opinion Extraction. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, Sofia, August 2013, 1640-1649. |
| [18] | Niu, F., Zhang, C., Ré, C. and Shavlik, J.W. (2012) DeepDive: Web-Scale Knowledge-Base Construc-tion Using Statistical Learning and Inference. VLDS’12, Istanbul, 31 August 2012, 25-28. |
| [19] | Toutanova, K. and Chen, D. (2015) Observed versus Latent Features for Knowledge Base and Text Inference. Proceedings of the 3rd Workshop on Contin-uous Vector Space Models and their Compositionality, Beijing, July 2015, 57-66. https://doi.org/10.18653/v1/W15-4007 |
| [20] | Xu, K., Feng, Y., Huang, S. and Zhao, D. (2015) Semantic Relation Classifi-cation via Convolutional Neural Networks with Simple Negative Sampling. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, September 2015, 536-540. https://doi.org/10.18653/v1/D15-1062 |
| [21] | Cai, R., Zhang, X. and Wang, H. (2016) Bidirectional Recurrent Convolutional Neural Network for Relation Classification. Proceedings of the 54th Annual Meeting of the Association for Computational Lin-guistics, Berlin, August 2016, 756-765. https://doi.org/10.18653/v1/P16-1072 |
| [22] | Ren, F., Zhou, D., Liu, Z., Li, Y., Zhao, R., Liu, Y. and Liang, X. (2018) Neural Relation Classification with Text Descriptions. Proceedings of the 27th Interna-tional Conference on Computational Linguistics, Santa Fe, August 2018, 1167-1177. |
| [23] | Zhou, L., Wang, T., Qu, H., Huang, L. and Liu, Y. (2020) A Weighted GCN with Logical Adjacency Matrix for Relation Extraction. Proceedings of the 24th Euro-pean Conference on Artificial Intelligence, Santiago de Compostela, August/September 2020, 2314-2321. |
| [24] | Guo, Z., Zhang, Y. and Lu, W. (2019) Attention Guided Graph Convolutional Networks for Relation Extraction. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, July 2019, 241-251.
https://doi.org/10.18653/v1/P19-1024 |
| [25] | Chen, H., Liu, L., Zhou, X., Qing, L. and Wang, M. (2020) A Robust Graph Convolutional Network for Relation Extraction by Combining Edge Information. 2020 IEEE 5th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA), Chengdu, 10-13 April 2020, 150-156. https://doi.org/10.1109/ICCCBDA49378.2020.9095628 |
| [26] | Sun, C., Gong, Y., Wu, Y., Gong, M., Jiang, D., Lan, M., Sun, S. and Duan, N. (2019) Joint Type Inference on Entities and Relations via Graph Convolutional Networks. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, July 2019, 1361-1370. https://doi.org/10.18653/v1/P19-1131 |
| [27] | Hong, Y., Liu, Y., Yang, S., Zhang, K., Wen, A. and Hu, J. (2020) Improving Graph Convolutional Networks Based on Relation-Aware Attention for End-to-End Relation Extraction. IEEE Access, 8, 51315-51323.
https://doi.org/10.1109/ACCESS.2020.2980859 |
| [28] | Sahu, S.K., Christopoulou, F., Miwa, M. and Ananiadou, S. (2019) Inter-Sentence Relation Extraction with Document-Level Graph Convolutional Neural Network. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, July 2019, 4309-4316. https://doi.org/10.18653/v1/P19-1423 |
| [29] | Song, Y. and Xia, F. (2013) A Common Case of Jekyll and Hyde: The Syner-gistic Effect of Using Divided Source Training Data for Feature Augmentation. Proceedings of the Sixth International Joint Conference on Natural Language Processing, Nagoya, 14-19 October 2013, 623-631. |
| [30] | Mandya, A., Bollegala, D. and Coenen, F. (2020) Graph Convolution over Multiple Dependency Sub-Graphs for Relation Extraction. Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, December 2020, 6424-6435. https://doi.org/10.18653/v1/2020.coling-main.565 |
| [31] | Nie, Y.Y., Tian, Y.H., Song, Y., Ao, X. and Wan, X. (2020) Im-proving Named Entity Recognition with Attentive Ensemble of Syntactic Information. In: Cohn, T., He, Y.L. and Liu, Y., Eds., Findings of the Association for Computational Linguistics: EMNLP 2020, Association for Computational Linguistics, Dublin, 4231-4245. |
| [32] | Guo, Z.J., Zhang, Y. and Lu, W. (2019) Attention Guided Graph Convolutional Networks for Relation Ex-traction. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, July 2019, 241- 251. |
| [33] | Chen, G.M., Tian, Y.H. and Song, Y. (2020) Joint Aspect Extraction and Sentiment Analysis with Directional Graph Convolutional Networks. Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, December 2020, 272-279. https://doi.org/10.18653/v1/2020.coling-main.24 |
| [34] | Tian, Y.H., Song, Y. and Xia, F. (2020) Supertagging Combinatory Categorial Grammar with Attentive Graph Convolutional Networks. Proceedings of the 2020 Con-ference on Empirical Methods in Natural Language Processing (EMNLP), November 2020, 6037-6044. https://doi.org/10.18653/v1/2020.emnlp-main.487 |
| [35] | Tian, Y.H., Chen, G.M. and Song, Y. (2021) Aspect-Based Sen-timent Analysis with Type-aware Graph Convolutional Networks and Layer Ensemble. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2021, 2910-2922.
https://doi.org/10.18653/v1/2021.naacl-main.231 |
| [36] | Zeng, D., Liu, K., Lai, S., et al. (2014) Relation Classifica-tion via Convolutional Deep Neural Network. Proceedings of the 25th International Conference on Computational Linguistics: Technical Papers, Dublin, August 2014, 2335-2344. |
| [37] | Zhang, D. and Wang, D. (2015) Relation Classification via Recur-rent Neural Network.
https://arxiv.org/pdf/1508.01006.pdf |
| [38] | Zhou, P., Shi, W., Tian, J., et al. (2016) Attention-Based Bidirectional Long Short-Term Memory Network for Relation Classification. Proceeding of 54th Annual Meeting of the Association for Computa-tional Linguistics, Berlin, August 2016, 207-212. https://doi.org/10.18653/v1/P16-2034 |
| [39] | Soares, L.B., FitzGerald, N., Ling, J. and Kwiatkowski, T. (2019) Matching the Blanks: Distributional Similarity for Relation Learning. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, July 2019, 2895-2905. https://doi.org/10.18653/v1/P19-1279 |
| [40] | Tian, Y.H., Chen, G.M., Song, Y., et al. (2021) Dependency-Driven Relation Extraction with Attentive Graph Convolutional Networks. Proceedings of the 59th Annual Meeting of the Association for Com-putational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 4458-4471. |
