全部 标题 作者
关键词 摘要

OALib Journal期刊
ISSN: 2333-9721
费用:99美元

查看量下载量

相关文章

更多...

基于实体知识推理的端到端任务型对话
End-to-End Task-Oriented Dialogue with Entity-Based Knowledge Inference

DOI: 10.12677/mos.2024.133293, PP. 3212-3221

Keywords: 端到端任务型对话,实体预测,响应生成,联合学习
End-to-End Task-Oriented Dialogue
, Entity Prediction, Response Generation, Joint Learning

Full-Text   Cite this paper   Add to My Lib

Abstract:

端到端任务型对话系统通过人机对话的方式来完成特定的任务以满足人类的需求,为了促进响应生成和实体预测,提出了一种基于实体知识推理的端到端任务型对话系统。该系统利用标记着对话历史实体信息的实体掩码,从对话历史编码中提取实体信息,并根据对话上下文推理出相关的实体信息以丰富当前上下文的实体知识;同时利用候选实体的正负样本与响应生成器的交叉注意力结合进行对比训练,将响应生成模型中的实体预测信号直接反馈给编码器以增强监督学习的能力;最终通过联合学习提高了模型的响应生成和实体预测能力。实验结果表明,在SMD、Multi-WOZ 2.1和CamRest数据集上,提出的模型能够生成丰富的响应内容,有着精准的实体预测准确率,并且验证了模型的有效性与实用性。
End-to-end task-oriented dialogue systems facilitate the completion of specific tasks through human-computer dialogue to meet human needs. To enhance response generation and entity prediction, we propose an end-to-end task-oriented dialogue system based on entity knowledge inference. This system leverages entity masks, annotated with dialogue history entity information, to extract entity information from the dialogue history encoding. It then infers relevant entity information from the dialogue context to enrich the current context’s entity knowledge. Simultaneously, the system employs contrastive training by integrating positive and negative samples of candidate entities with the cross-attention of the response generator. This setup directly feeds back entity prediction signals from the response generation model to the encoder, thereby enhancing the capability of supervised learning. Ultimately, the model’s response generation and entity prediction abilities are improved through joint learning. Experimental results indicate that on the SMD, Multi-WOZ 2.1, and CamRest datasets, the proposed model can generate rich response content with precise entity prediction accuracy, validating the model’s effectiveness and practicality.

References

[1]  Wen, T.-H., Vandyke, D., Mrksic, N., et al. (2017) A Network-Based End-to-End Trainable Task-Oriented Dialogue System. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, Vol. 1, Valencia, 3-7 April 2017, 438-449.
https://doi.org/10.18653/v1/E17-1042
[2]  Eric, M., Krishnan, L., Charette, F. and Manning, C.D. (2017) Key-Value Retrieval Networks for Task-Oriented Dialogue. Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, Saarbrücken, 15-17 August 2017, 37-49.
https://doi.org/10.18653/v1/W17-5506
[3]  Eric, M., Goel, R., Paul, S., et al. (2020) MultiWOZ 2.1: A Consolidated Multi-Domain Dialogue Dataset with State Corrections and State Tracking Baselines. Proceedings of the Twelfth Language Resources and Evaluation Conference, Marseille, 11-16 May 2020, 422-428.
[4]  Wei, Z., Liu, Q., Peng, B., et al. (2018) Task-Oriented Dialogue System for Automatic Diagnosis. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Vol. 2, Melbourne, 15-20 July 2018, 201-207.
https://doi.org/10.18653/v1/P18-2033
[5]  Wu, C.-S., Hoi, S.C.H., Socher, R., et al. (2020) TOD-BERT: Pre-Trained Natural Language Understanding for Task-Oriented Dialogue. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online, 16-20 November 2020, 917-929.
https://doi.org/10.18653/v1/2020.emnlp-main.66
[6]  Madotto, A., Wu, C.-S. and Fung, P. (2018) Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Vol. 1, Melbourne, 15-20 July 2018, 1468-1478.
https://doi.org/10.18653/v1/P18-1136
[7]  Wen, H., Liu, Y., Che, W., et al. (2018) Sequence-to-Sequence Learning for Task-Oriented Dialogue with Dialogue State Representation. Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, 20-26 August 2018, 3781-3792.
[8]  Reddy, R.G., Contractor, D., Raghu, D., et al. (2019) Multi-Level Memory for Task Oriented Dialogs. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 1, Minneapolis, 2-7 June 2019, 3744-3754.
[9]  Qin, L., Liu, Y., Che, W., et al. (2019) Entity-Consistent End-to-End Task-Oriented Dialogue System with KB Retriever. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, 3-7 November 2019, 133-142.
https://doi.org/10.18653/v1/D19-1013
[10]  Wu, C.-S., Socher, R. and Xiong, C. (2019) Global-to-Local Memory Pointer Networks for Task-Oriented Dialogue. arXiv Preprint arXiv:1901.04713
[11]  Qin, L., Xu, X., Che, W., et al. (2020) Dynamic Fusion Network for Multi-Domain End-to-End Task-Oriented Dialog. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, 5-10 July 2020, 6344-6354.
https://doi.org/10.18653/v1/2020.acl-main.565
[12]  Liu, Q., Bai, G., He, S., et al. (2021) Heterogeneous Relational Graph Neural Networks with Adaptive Objective for End-to-End Task-Oriented Dialogue. Knowledge-Based Systems, 227, Article 107186.
https://doi.org/10.1016/j.knosys.2021.107186
[13]  Wu, J., Harris, I.G. and Zhao, H. (2022) GraphMemDialog: Optimizing End-to-End Task-Oriented Dialog Systems Using Graph Memory Networks. Proceedings of the Thirty-Sixth AAAI Conference on Artificial Intelligence, Vol. 36, Online, 22 February-1 March 2022, 11504-11512.
https://doi.org/10.1609/aaai.v36i10.21403
[14]  Wan, F., Shen, W., Yang, K., et al. (2023) Multi-Grained Knowledge Retrieval for End-to-End Task-Oriented Dialog. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Vol. 1, Toronto, 9-14 July 2023, 11196-11210.
https://doi.org/10.18653/v1/2023.acl-long.627
[15]  Bordes, A., Boureau, Y.-L. and Weston, J. (2016) Learning End-to-End Goal-Oriented Dialog. arXiv Preprint arXiv:1605.07683
[16]  Chung, J., Gulcehre, C., Cho, K.H., et al. (2014) Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. arXiv Preprint arXiv:1412.3555
[17]  Lei, W., Jin, X., Kan, M.-Y., et al. (2018) Sequicity: Simplifying Task-Oriented Dialogue Systems with Single Sequence-to-Sequence Architectures. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Vol. 1, Melbourne, 15-20 July 2018, 1437-1447.
https://doi.org/10.18653/v1/P18-1133
[18]  Cho, K., van Merri?nboer, B., Gulcehre, C., et al. (2014) Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, 25-29 October 2014, 1724-1734.
https://doi.org/10.3115/v1/D14-1179
[19]  Sukhbaatar, S., Weston, J. and Fergus, R. (2015) End-to-End Memory Networks. Proceedings of the 28th International Conference on Neural Information Processing Systems, Montreal, 7-12 December 2015, 2440-2448.
[20]  Lin, Z., Huang, X., Ji, F., et al. (2019) Task-Oriented Conversation Generation Using Heterogeneous Memory Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, 3-7 November 2019, 4558-4567.
https://doi.org/10.18653/v1/D19-1463
[21]  Mehri, S., Srinivasan, T. and Eskenazi, M. (2019) Structured Fusion Networks for Dialog. Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue, Stockholm, 11-13 September 2019, 165-177.
https://doi.org/10.18653/v1/W19-5921
[22]  Zhang, Y., Ou, Z. and Yu, Z. (2020) Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context. Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, Vol. 34, 7-12 February 2020, New York, 9604-9611.
https://doi.org/10.1609/aaai.v34i05.6507
[23]  Vaswani, A., Shazeer, N., Parmar, N., et al. (2017) Attention Is All You Need. Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, 4-9 December 2017, 5998-6008.
[24]  Xie, T., Wu, C.H., Shi, P., et al. (2022) UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Abu Dhabi, 7-11 December 2022, 602-631.
https://doi.org/10.18653/v1/2022.emnlp-main.39
[25]  Rony, R.A.H., Usbeck, R. and Lehmann, J. (2022) DialoKG: Knowledge-Structure Aware Task-Oriented Dialogue Generation. Findings of the Association for Computational Linguistics: NAACL 2022, Seattle, 10-15 July 2022, 2557-2571.
https://doi.org/10.18653/v1/2022.findings-naacl.195
[26]  Tian, X., Lin, Y., Song, M., et al. (2022) Q-TOD: A Query-Driven Task-Oriented Dialogue System. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Abu Dhabi, 7-11 December 2022, 7260-7271.
https://doi.org/10.18653/v1/2022.emnlp-main.489
[27]  Shen, W., Gao, Y., Huang, C., et al. (2023) Retrieval-Generation Alignment for End-to-End Task-Oriented Dialogue System. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore, 6-10 December 2023, 8261-8275.
https://doi.org/10.18653/v1/2023.emnlp-main.514
[28]  Shi, T., Li, L., Lin, Z., et al. (2023) Dual-Feedback Knowledge Retrieval for Task-Oriented Dialogue Systems. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore, 6-10 December 2023, 6566-6580.
https://doi.org/10.18653/v1/2023.emnlp-main.405
[29]  Johnson, J., Douze, M. and Jégou, H. (2019) Billion-Scale Similarity Search with GPUs. IEEE Transactions on Big Data, 7, 535-547.
https://doi.org/10.1109/TBDATA.2019.2921572
[30]  Izacard, G. and Grave, E. (2020) Distilling Knowledge from Reader to Retriever for Question Answering. arXiv Preprint arXiv:2012.04584
[31]  Raffel, C., Shazeer, N., Roberts, A., et al. (2020) Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. The Journal of Machine Learning Research, 21, 5485-5551.
[32]  Raghu, D., Jain, A. and Joshi, S. (2021) Constraint Based Knowledge Base Distillation in End-to-End Task Oriented Dialogs. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, Online, 1-6 August 2021, 5051-5061.
https://doi.org/10.18653/v1/2021.findings-acl.448
[33]  Huang, G., Quan, X. and Wang, Q. (2022) Autoregressive Entity Generation for End-to-End Task-Oriented Dialog. Proceedings of the 29th International Conference on Computational Linguistics, Gyeongju, 12-17 October 2022, 323-332.
[34]  Dong, X. and Chen, J. (2023) PluDG: Enhancing Task-Oriented Dialogue System with Knowledge Graph Plug-In Module. PeerJ Computer Science, 9, e1707.
https://doi.org/10.7717/peerj-cs.1707
[35]  Papineni, K., Roukos, S., Ward, T., et al. (2002) Bleu: A Method for Automatic Evaluation of Machine Translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, 7-12 July 2002, 311-318.

Full-Text

Contact Us

service@oalib.com

QQ:3279437679

WhatsApp +8615387084133