一种融合语法信息的句子压缩方法
A Method Incorporated Syntax Attention for Sentence Compression

DOI: 10.12677/CSA.2020.103058, PP. 564-574

Keywords: 句子压缩，语法注意力机制，长短期以及网络，鲁棒性
Sentence Compression, Syntax Attention Mechanism, Long Short-Term Memory, Robustness

Full-Text   Cite this paper   Add to My Lib

Abstract:

英文句子压缩任务由于词典容量等限制，使用深度学习方法容易造成压缩后的句意与原句不同并一定程度影响语法逻辑。针对这一问题，文中提出一种融合语法信息的句子压缩方法。首先通过两组编解码器来对单词和词性分别进行编解码，在解码阶段通过带有语法注意力机制的长短期记忆网络(Syntax-LSTM)融合单词和词性信息产生语法注意力机制进而引导输出结果。与现有方法相比，实验结果表明该算法的F1值在领域数据集上达到了0.7742，在跨领域数据集上达到了0.4186，证明了其输出具有更好的可读性和鲁棒性。
The size of dictionary in English Sentence Compression is limited, so using deep learning methods to compress sentences are prone to delete the keywords by mistake, then affect the meaning of the sentences after compression. To address this problem, this paper proposes a method incorporated syntax attention for sentence compression. Firstly, using two sets of encoder-decoder to encode and decode words and syntax, in the decoder stage, the Syntax-LSTM using syntax gates generates a syntax attention mechanism to lead a more grammatical output. The experimental results show that the F1 value reaches 0.7742 on the same domain dataset and 0.4186 on the cross-domain data, which proves that its results are more readable and more robustness compared with the existing methods.

References

[1]	Jing, H. (2000) Sentence Reduction for Automatic Text Summarization. Conference on Applied Natural Language Processing, Seattle, 29 April-4 May 2000, 310-315. https://doi.org/10.3115/974147.974190
[2]	Knight, K. and Marcu, D. (2000) Statistics-Based Summarization—Step One: Sentence Compression. National Conference on Artifi-cial Intelligence, Austin, 703-710.
[3]	Filippova, K., Alfonseca, E., Colmenares, C.A., et al. (2015) Sentence Com-pression by Deletion with LSTMs. Empirical Methods in Natural Language Processing, Lisbon, September 2015, 360-368. https://doi.org/10.18653/v1/D15-1042
[4]	Hochreiter, S. and Schmidhuber, J. (1996) LSTM Can Solve Hard Long Time Lag Problems. Neural Information Processing Systems, Denver, 473-479.
[5]	Gers, F.A., Schmidhuber, J., Cummins, F., et al. (2000) Learning to Forget: Continual Prediction with LSTM. Neural Computation, 12, 2451-2471. https://doi.org/10.1162/089976600300015015
[6]	Cohn, T. and Lapata, M. (2009) Sentence Compression as Tree Transduction. Journal of Artificial Intelligence Research, 34, 637-674. https://doi.org/10.1613/jair.2655
[7]	Filippova, K. and Strube, M. (2008) Dependency Tree Based Sentence Compression. International Conference on Natural Language Generation, Salt Fork, 12-14 June 2008, 25-32. https://doi.org/10.3115/1708322.1708329
[8]	Lai, D.V., Truong, N. and Minh, N.L. (2017) Deletion-Based Sentence Compression Using Bi-enc-dec LSTM, 2017. In: Conference of the Pacific Association for Computational Linguistics, Springer, Singapore, 249-260. https://doi.org/10.1007/978-981-10-8438-6_20
[9]	Tran, N., Luong, V., Nguyen, N.L., et al. (2016) Effective Attention-Based Neural Architectures for Sentence Compression with Bidirectional Long Short-Term Memory. Sym-posium on Information and Communication Technology, Ho Chi Minh City, 8-9 December 2016, 123-130. https://doi.org/10.1145/3011077.3011111
[10]	鹿忠磊, 刘文芬, 周艳芳, 等. 基于预读及简单注意力机制的句子压缩方法[J]. 计算机应用研究, 2019, 36(2): 57-61+80.
[11]	Wilks, Y. and Stevenson, M. (1998) The Grammar of Sense: Using Part-of-Speech Tags as a First Step in Semantic Disambiguation. Natural Language Engineering, 4, 135-143. https://doi.org/10.1017/S1351324998001946
[12]	刘杰, 孙娜, 袁克柔, 等. 中文作文句间逻辑合理性智能判别方法研究[J]. 计算机应用与软件, 2019, 36(1): 77-83.
[13]	Wang, L., Jiang, J., Chieu, H.L., et al. (2017) Can Syntax Help? Improving an LSTM-Based Sentence Compression Model for New Domains. Meeting of the Asso-ciation for Computational Linguistics, Vancouver, July 2017, 1385-1393. https://doi.org/10.18653/v1/P17-1127
[14]	He, K., Zhang, X., Ren, S., et al. (2016) Deep Residual Learning for Image Recognition. Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 770-778. https://doi.org/10.1109/CVPR.2016.90
[15]	Mikolov, T., Sutskever, I., Chen, K., et al. (2013) Distributed Representations of Words and Phrases and Their Compositionality. Advances in Neural Information Processing Systems, Lake Tahoe, December 2013, 3111-3119.
[16]	Mikolov, T., Chen, K., Corrado, G., et al. (2013) Efficient Estimation of Word Representations in Vector Space. International Conference on Learning Representations. arXiv preprint arXiv:1301.3781
[17]	Filippova, K. and Altun, Y. (2013) Overcoming the Lack of Parallel Data in Sentence Com-pression. Empirical Methods in Natural Language Processing, Seattle, 1481-1491.
[18]	Prechelt, L. (1998) Early Stopping—But When? In: Orr, G.B. and Müller, K.-R., Eds., Neural Networks: Tricks of the Trade, Vol. 1524, Springer, Berlin, Heidelberg, 55-69. https://doi.org/10.1007/3-540-49430-8_3

Full-Text