|
|
基于情感信息抽取与双向长短期记忆网络的情感分析研究
|
Abstract:
网络舆情已成为当下不可忽视的研究领域，利用深度学习技术对舆情文本进行情感分析，可以通过分析结果对事件采取合适的决策。针对当前网络文本中因文本序列长度变化以及复杂的文本含义而导致的语义情感特征精准提取的问题，本文提出了一种基于CNN-BiLSTM与多头注意力(CBLMAtt)的舆情分析模型。该模型首先通过对网络文本数据和方面级数据进行训练学习，利用卷积神经网络(CNN)的变体对文本中的高级语义特征进行提取。其次将提取到的特征作为输入传递到双向长短期记忆网络(BiLSTM)层，利用该层的特点对文中的上下文特征进行捕获表示。然后将BiLSTM层直接对输入数据进行计算。最后在这两种方法所得到的变量中，输出隐藏表示被传递到多头注意力层，使用softmax激活函数的输出层进行情感极性分类。我们在公开的数据集上评估了我们的模型，在这些数据集上通过精确率的平均值和F1值的对比表明，与传统的模型相比，我们的方法相比这些模型都提升两个百分点以上。此外，我们也进行了消融实验，以显示不同文档级别权重对学习技术的影响。
Online public opinion has become a research field that cannot be ignored nowadays, and the use of deep learning techniques for sentiment analysis of public opinion texts can be used to take appropriate decisions on events. This paper proposes a public opinion analysis model based on CNN BiLSTM and multi head attention (CBLMAtt) to address the issue of precise semantic and emotional feature extraction in current online texts due to changes in text sequence length and complex text meanings. The model trains and learns from both online text data and aspect level data. Firstly, a variant of Convolutional Neural Network (CNN) is used to extract advanced semantic features from text. Secondly, the extracted features are transferred as input to the bidirectional short-term memory network (BiLSTM) layer, and the context features in the text are captured and represented using the characteristics of this layer. Then, the BiLSTM layer is directly used to calculate the input data. Finally, among the variables obtained by the two methods, the output hiding representation is transferred to the multi head attention layer, and the output layer of the softmax activation function is used to classify the emotional polarity. We evaluated our model on publicly available datasets and compared the average accuracy and F1 values on these datasets, indicating that our method outperforms traditional models by more than two percentage points. In addition, we also conducted ablation studies to demonstrate the impact of different document level weights on learning techniques.
| [1] | 王家辉, 夏志杰, 王诣铭, 等. 基于句法规则和社会网络分析的网络舆情热点主题可视化及演化研究[J]. 情报科学, 2020, 38(7): 132-139. |
| [2] | 邓君, 孙绍丹, 王阮, 等. 基于Word2Vec和SVM的微博舆情情感演化分析[J]. 情报理论与实践, 2020, 43(8): 112-119. |
| [3] | 范涛, 吴鹏, 曹琪. 基于深度学习的多模态融合网民情感识别研究[J]. 信息资源管理学报, 2020, 10(1): 39-48. |
| [4] | 吴杰胜, 陆奎. 基于多部情感词典和规则集的中文微博情感分析研究[J]. 计算机应用与软件, 2019, 36(9): 93-99. |
| [5] | 陈珂, 谢博, 朱兴统. 基于情感词典和Transformer模型的情感分析算法研究[J]. 南京邮电大学学报(自然科学版), 2020, 40(1): 55-62. |
| [6] | Kononenko, I. (1989) Bayesian Neural Networks. Biological Cybernetics, 61, 361-370. https://doi.org/10.1007/bf00200801 |
| [7] | Anguita, D., Ridella, S. and Rovetta, S. (1998) Circuital Implementation of Support Vector Machines. Electronics Letters, 34, 1596-1597. https://doi.org/10.1049/el:19981092 |
| [8] | 戚天梅, 过弋, 王吉祥, 等. 基于机器学习的外汇新闻情感分析[J]. 计算机工程与设计, 2020, 41(6): 1742-1748. |
| [9] | 濮泽堃. 基于机器学习的电商评论情感分析系统[D]: [硕士学位论文]. 南京: 南京邮电大学, 2020. |
| [10] | Lu, Q., Zhu, Z., Zhang, G., Kang, S. and Liu, P. (2021) Aspect-Gated Graph Convolutional Networks for Aspect-Based Sentiment Analysis. Applied Intelligence, 51, 4408-4419. https://doi.org/10.1007/s10489-020-02095-3 |
| [11] | Nemes, L. and Kiss, A. (2020) Social Media Sentiment Analysis Based on Covid-19. Journal of Information and Telecommunication, 5, 1-15. https://doi.org/10.1080/24751839.2020.1790793 |
| [12] | Priyadarshini, I., Mohanty, P., Kumar, R., Son, L.H., Chau, H.T.M., Nhu, V., et al. (2020) Analysis of Outbreak and Global Impacts of the Covid-19. Healthcare, 8, Article No. 148. https://doi.org/10.3390/healthcare8020148 |
| [13] | Dansana, D., Kumar, R., Das Adhikari, J., Mohapatra, M., Sharma, R., Priyadarshini, I., et al. (2020) Global Forecasting Confirmed and Fatal Cases of COVID-19 Outbreak Using Autoregressive Integrated Moving Average Model. Frontiers in Public Health, 8, Article ID: 580327. https://doi.org/10.3389/fpubh.2020.580327 |
| [14] | dos Santos, C.N. and Gatti, M. (2014) Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts. Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, Dublin, August 2014, 69-78. |
| [15] | Wang, J., Yu, L., Lai, K.R. and Zhang, X. (2016) Dimensional Sentiment Analysis Using a Regional CNN-LSTM Model. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Volume 2, 225-230. https://doi.org/10.18653/v1/p16-2037 |
| [16] | Yoon, J. and Kim, H. (2017) Multi-Channel Lexicon Integrated CNN-BiLSTM Models for Sentiment Analysis. Proceedings of the 29th Conference on Computational Linguistics and Speech Processing (ROCLING 2017), Taipei, 27-28 November 2017, 244-253. |
| [17] | Wang, X.Y., Jiang, W.J. and Luo, Z.Y. (2016) Combination of Convolutional and Recurrent Neural Network for Sentiment Analysis of Short Texts. Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, Osaka, December 2016, 2428-2437. |
| [18] | Al Sallab, A., Hajj, H., Badaro, G., Baly, R., El Hajj, W. and Bashir Shaban, K. (2015) Deep Learning Models for Sentiment Analysis in Arabic. Proceedings of the 2nd Workshop on Arabic Natural Language Processing, Beijing, 30 July 2015, 9-17. https://doi.org/10.18653/v1/w15-3202 |
| [19] | Van, V.D., Thai, T. and Nghiem, M. (2017) Combining Convolution and Recursive Neural Networks for Sentiment Analysis. Proceedings of the 8th International Symposium on Information and Communication Technology, Nha Trang, 7-8 December 2017, 151-158. https://doi.org/10.1145/3155133.3155158 |
| [20] | Chen, X., Wang, Y. and Liu, Q. (2017) Visual and Textual Sentiment Analysis Using Deep Fusion Convolutional Neural Networks. 2017 IEEE International Conference on Image Processing (ICIP), Beijing, 17-20 September 2017, 1557-1561. https://doi.org/10.1109/icip.2017.8296543 |
| [21] | Lin, H. and Bu, N. (2022) A CNN-Based Framework for Predicting Public Emotion and Multi-Level Behaviors Based on Network Public Opinion. Frontiers in Psychology, 13, Article ID: 909439. https://doi.org/10.3389/fpsyg.2022.909439 |
| [22] | Sagnika, S., Mishra, B.S.P. and Meher, S.K. (2021) An Attention-Based CNN-LSTM Model for Subjectivity Detection in Opinion-Mining. Neural Computing and Applications, 33, 17425-17438. https://doi.org/10.1007/s00521-021-06328-5 |
| [23] | Priyadarshini, I. and Cotton, C. (2021) A Novel LSTM-CNN-Grid Search-Based Deep Neural Network for Sentiment Analysis. The Journal of Supercomputing, 77, 13911-13932. https://doi.org/10.1007/s11227-021-03838-w |
| [24] | Zhang, W., Li, L., Zhu, Y., Yu, P. and Wen, J. (2022) CNN-LSTM Neural Network Model for Fine-Grained Negative Emotion Computing in Emergencies. Alexandria Engineering Journal, 61, 6755-6767. https://doi.org/10.1016/j.aej.2021.12.022 |
| [25] | Wang, Y., Huang, M., Zhu, X. and Zhao, L. (2016) Attention-Based LSTM for Aspect-Level Sentiment Classification. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, 1-5 November 2016, 606-615. https://doi.org/10.18653/v1/d16-1058 |
| [26] | Liu, J. and Zhang, Y. (2017) Attention Modeling for Targeted Sentiment. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, Volume 2, 572-577. https://doi.org/10.18653/v1/e17-2091 |
| [27] | Ma, D., Li, S., Zhang, X. and Wang, H. (2017). Interactive Attention Networks for Aspect-Level Sentiment Classification. Proceedings of the 26th International Joint Conference on Artificial Intelligence, Melbourne, 19-25 August 2017, 4068-4074. https://doi.org/10.24963/ijcai.2017/568 |
| [28] | Chen, P., Sun, Z., Bing, L. and Yang, W. (2017) Recurrent Attention Network on Memory for Aspect Sentiment Analysis. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, 9-11 September 2017, 452-461. https://doi.org/10.18653/v1/d17-1047 |
| [29] | Fan, F., Feng, Y. and Zhao, D. (2018) Multi-Grained Attention Network for Aspect-Level Sentiment Classification. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, 31 October-4 November 2018, 3433-3442. https://doi.org/10.18653/v1/d18-1380 |
| [30] | Huang, B., Ou, Y. and Carley, K.M. (2018) Aspect Level Sentiment Classification with Attention-over-Attention Neural Networks. In: Thomson, R., et al., Eds., Social, Cultural, and Behavioral Modeling, Springer International Publishing, 197-206. https://doi.org/10.1007/978-3-319-93372-6_22 |
| [31] | Yang, C., Zhang, H., Jiang, B. and Li, K. (2019) Aspect-Based Sentiment Analysis with Alternating Coattention Networks. Information Processing & Management, 56, 463-478. https://doi.org/10.1016/j.ipm.2018.12.004 |
| [32] | Zhang, S., Xu, X., Pang, Y. and Han, J. (2019) Multi-Layer Attention Based CNN for Target-Dependent Sentiment Classification. Neural Processing Letters, 51, 2089-2103. https://doi.org/10.1007/s11063-019-10017-9 |
| [33] | Gan, C., Wang, L., Zhang, Z. and Wang, Z. (2020) Sparse Attention Based Separable Dilated Convolutional Neural Network for Targeted Sentiment Analysis. Knowledge-Based Systems, 188, Article ID: 104827. https://doi.org/10.1016/j.knosys.2019.06.035 |
| [34] | Lei, Z., Yang, Y. and Yang, M. (2018) Sentiment Lexicon Enhanced Attention-Based LSTM for Sentiment Classification. Proceedings of the AAAI Conference on Artificial Intelligence, 32, 8105-8106. https://doi.org/10.1609/aaai.v32i1.12142 |
| [35] | Pascanu, R., Mikolov, T. and Bengio, Y. (2013) On the Difficulty of Training Recurrent Neural Networks. Proceedings of the 30th International Conference on Machine Learning, Atlanta, 17-19 June 2013, 1310-1318. |
| [36] | Hochreiter, S. and Schmidhuber, J. (1997) Long Short-Term Memory. Neural Computation, 9, 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735 |
| [37] | Pontiki, M., Galanis, D., Pavlopoulos, J., Papageorgiou, H., Androutsopoulos, I. and Manandhar, S. (2014) Semeval-2014 Task 4: Aspect Based Sentiment Analysis. Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), Dublin, 23-24 August 2014, 27-35. https://doi.org/10.3115/v1/s14-2004 |
| [38] | Hercig, T., Brychcín, T., Svoboda, L. and Konkol, M. (2016) UWB at Semeval-2016 Task 5: Aspect Based Sentiment Analysis. Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), San Diego, 16-17 June 2016, 486-495. https://doi.org/10.18653/v1/s16-1055 |
| [39] | Pontiki, M., Galanis, D., Papageorgiou, H., Androutsopoulos, I., Manandhar, S., AL-Smadi, M., et al. (2016) Semeval-2016 Task 5: Aspect Based Sentiment Analysis. Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), San Diego, 16-17 June 2016, 19-30. https://doi.org/10.18653/v1/s16-1002 |
