|
|
基于图同构网络的多模态虚假新闻检测
|
Abstract:
多模态虚假新闻检测是指通过整合来自多个模态的信息，识别和分类虚假新闻的一种技术。本文基于当前虚假新闻通常涵盖文本、图像以及复杂的社会关系等多重信息，且现有模型处理社交上下文信息结构信息捕捉严重不足，本文提出基于图同构网络的多模态虚假新闻检测框架(Graph Isomorphism Networks Fake News Detection, GIN_FND)，通过提取新闻中文本、图像和社交上下文信息等多模态信息，基于图同构网络提取社交信息的方式有效捕捉社交图节点的局部依赖和全局关联，有效提升模型检测准确性和鲁棒性。在公开数据集上对本模型进行的系统评估表明本文方法的有效性。
Multimodal fake news detection refers to a technology that identifies and classifies fake news by integrating information from multiple modalities. Given that current fake news often encompasses text, images, and complex social relationships, and existing models significantly lack in capturing the structural information of social context, this paper proposes a multimodal fake news detection framework based on Graph Isomorphism Networks (Graph Isomorphism Networks Fake News Detection, GIN_FND). By extracting multimodal information such as text, images, and social context from the news, the framework effectively captures the local dependencies and global associations of nodes in the social graph through the use of Graph Isomorphism Networks, thereby enhancing the model’s detection accuracy and robustness. Systematic evaluations on public datasets demonstrate the effectiveness of the proposed method.
| [1] | Allcott, H. and Gentzkow, M. (2017) Social Media and Fake News in the 2016 Election. Journal of Economic Perspectives, 31, 211-236. https://doi.org/10.1257/jep.31.2.211 |
| [2] | Vosoughi, S., Roy, D. and Aral, S. (2018) The Spread of True and False News Online. Science, 359, 1146-1151. https://doi.org/10.1126/science.aap9559 |
| [3] | Chen, Y., Li, D., Zhang, P., Sui, J., Lv, Q., Tun, L., et al. (2022) Cross-Modal Ambiguity Learning for Multimodal Fake News Detection. Proceedings of the ACM Web Conference 2022, Virtual Event, 25-29 April 2022, 2897-2905. https://doi.org/10.1145/3485447.3511968 |
| [4] | Dou, Y., Shu, K., Xia, C., Yu, P.S. and Sun, L. (2021) User Preference-Aware Fake News Detection. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, 11-15 July 2021, 2051-2055. https://doi.org/10.1145/3404835.3462990 |
| [5] | Peng, H., Li, J., He, Y., Liu, Y., Bao, M., Wang, L., et al. (2018) Large-Scale Hierarchical Text Classification with Recursively Regularized Deep Graph-CNN. Proceedings of the 2018 World Wide Web Conference on World Wide Web, Lyon, 23-27 April 2018, 1063-1072. https://doi.org/10.1145/3178876.3186005 |
| [6] | Zhou, X. and Zafarani, R. (2019) Network-Based Fake News Detection. ACM SIGKDD Explorations Newsletter, 21, 48-60. https://doi.org/10.1145/3373464.3373473 |
| [7] | Cheng, M., Nazarian, S. and Bogdan, P. (2020) VRoC: Variational Autoencoder-Aided Multi-Task Rumor Classifier Based on Text. Proceedings of The Web Conference 2020, 20-24 April 2020, 2892-2898. https://doi.org/10.1145/3366423.3380054 |
| [8] | Jin, Z., Cao, J., Zhang, Y., Zhou, J. and Tian, Q. (2017) Novel Visual and Statistical Image Features for Microblogs News Verification. IEEE Transactions on Multimedia, 19, 598-608. https://doi.org/10.1109/tmm.2016.2617078 |
| [9] | Khudeyer, R.S. and Almoosawi, N.M. (2023) Fake Image Detection Using Deep Learning. Informatica, 47, 115-120. https://doi.org/10.31449/inf.v47i7.4741 |
| [10] | Qi, P., Cao, J., Yang, T., Guo, J. and Li, J. (2019) Exploiting Multi-Domain Visual Information for Fake News Detection. 2019 IEEE International Conference on Data Mining (ICDM), Beijing, 8-11 November 2019, 518-527. https://doi.org/10.1109/icdm.2019.00062 |
| [11] | Singhal, S., Shah, R.R., Chakraborty, T., Kumaraguru, P. and Satoh, S. (2019) SpotFake: A Multi-Modal Framework for Fake News Detection. 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM), Singapore, 11-13 September 2019, 39-47. https://doi.org/10.1109/bigmm.2019.00-44 |
| [12] | Singhal, S., Kabra, A., Sharma, M., Shah, R.R., Chakraborty, T. and Kumaraguru, P. (2020) SpotFake+: A Multimodal Framework for Fake News Detection via Transfer Learning (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence, 34, 13915-13916. https://doi.org/10.1609/aaai.v34i10.7230 |
| [13] | Wang, Y., Ma, F., Jin, Z., Yuan, Y., Xun, G., Jha, K., et al. (2018) EANN: Event Adversarial Neural Networks for Multi-Modal Fake News Detection. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, 19-23 August 2018, 849-857. https://doi.org/10.1145/3219819.3219903 |
| [14] | Zhou, X., Wu, J. and Zafarani, R. (2020) SAFE: Similarity-Aware Multi-Modal Fake News Detection. Advances in Knowledge Discovery and Data Mining, Singapore, 11-14 May 2020, 354-367. https://doi.org/10.1007/978-3-030-47436-2_27 |
| [15] | Wang, Y., Qian, S., Hu, J., Fang, Q. and Xu, C. (2020) Fake News Detection via Knowledge-Driven Multimodal Graph Convolutional Networks. Proceedings of the 2020 International Conference on Multimedia Retrieval, Dublin, 8-11 June 2020, 540-547. https://doi.org/10.1145/3372278.3390713 |
| [16] | Hu, L., Yang, T., Zhang, L., Zhong, W., Tang, D., Shi, C., et al. (2021) Compare to the Knowledge: Graph Neural Fake News Detection with External Knowledge. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Online, 1-6 August 2021, 754-763. https://doi.org/10.18653/v1/2021.acl-long.62 |
| [17] | Kipf, T.N. and Welling, M. (2017) Semi-Supervised Classification with Graph Convolutional Networks. Proceedings of the International Conference on Learning Representations (ICLR), 24-26 April 2017, 1-14. |
| [18] | Bian, T., Xiao, X., Xu, T., Zhao, P., Huang, W., Rong, Y., et al. (2020) Rumor Detection on Social Media with Bi-Directional Graph Convolutional Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 34, 549-556. https://doi.org/10.1609/aaai.v34i01.5393 |
| [19] | Kang, Z., Cao, Y., Shang, Y., Liang, T., Tang, H. and Tong, L. (2021) Fake News Detection with Heterogenous Deep Graph Convolutional Network. Advances in Knowledge Discovery and Data Mining, Virtual Event, 11-14 May 2021, 408-420. https://doi.org/10.1007/978-3-030-75762-5_33 |
| [20] | Xu, K., Hu, W., Leskovec, J., et al. (2019) How Powerful Are Graph Neural Networks? Proceedings of the International Conference on Learning Representations (ICLR), 6-9 May 2019, 1-17. |
| [21] | Devlin, J., Chang, M.-W., Lee, K. and Toutanova, K. (2019) BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), Minneapolis, 2-7 June 2019, 4171-4186. |
| [22] | He, K., Zhang, X., Ren, S. and Sun, J. (2016) Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 27-30 June 2016, 770-778. https://doi.org/10.1109/cvpr.2016.90 |
| [23] | Shervashidze, N., Schweitzer, P., Van Leeuwen, E.J., et al. (2011) Weisfeiler-Lehman Graph Kernels. Journal of Machine Learning Research, 12, 2539-2561. |
| [24] | Ma, J., Gao, W., Mitra, P., et al. (2016) Detecting Rumors from Microblogs with Recurrent Neural Networks. Proceedings of the 25th International Joint Conference on Artificial Intelligence, New York, 9-15 July 2016, 3818-3824. |
| [25] | Zubiaga, A., Liakata, M. and Procter, R. (2017) Exploiting Context for Rumour Detection in Social Media. Social Informatics, Oxford, 13-15 September 2017, 109-123. https://doi.org/10.1007/978-3-319-67217-5_8 |
| [26] | Yu, F., Liu, Q., Wu, S., Wang, L. and Tan, T. (2017) A Convolutional Approach for Misinformation Identification. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, Melbourne, 19-25 August 2017, 3901-3907. https://doi.org/10.24963/ijcai.2017/545 |
| [27] | Jin, Z., Cao, J., Guo, H., Zhang, Y. and Luo, J. (2017) Multimodal Fusion with Recurrent Neural Networks for Rumor Detection on Microblogs. Proceedings of the 25th ACM International Conference on Multimedia, Mountain View, 23-27 October 2017, 795-816. https://doi.org/10.1145/3123266.3123454 |
| [28] | Zheng, J., Zhang, X., Guo, S., Wang, Q., Zang, W. and Zhang, Y. (2022) MFAN: Multi-Modal Feature-Enhanced Attention Networks for Rumor Detection. Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, Vienna, 23-29 July 2022, 2413-2419. https://doi.org/10.24963/ijcai.2022/335 |
| [29] | Kingma, D.P. (2014) Adam: A Method for Stochastic Optimization. arXiv: 1412.6980. https://doi.org/10.48550/arXiv.1412.6980 |
