|
|
时频融合和特征交叉融合的序列推荐算法
|
Abstract:
为了有效融合项目ID嵌入和文本嵌入，提出一种时频融合和特征交叉融合的序列推荐算法(Time-frequency fusion and feature cross-fusion sequential recommendation algorithm, TFFCRec)。使用对项目文本进行编码，多样化混合专家调制方法获得的是易于区分的文本表示，将项目ID嵌入和文本嵌入通过结合快速傅里叶变换(FFT)和短时傅里叶变换(STFT)，提取用户的全局频域特征和局部时频特征。这样的方法使得算法既能考虑用户的长期兴趣偏好，又能捕捉用户的短期兴趣变化。此外，我们引入了特征交叉融合，并通过优化的Mamba-like的线性注意力(OMLLA)来捕获特征之间更深层次的非线性关系，提取更深层次的特征表示。我们设计了一个融合网络，自适应地学习不同嵌入表示的权重，将FFT、STFT和OMLLA得到的特征向量进行加权融合，通过SASRec来进行序列推荐。在Instant video、Beauty、Digital Music、Tools and Home improvement数据集上进行实验，本文方法较基准方法在Recall@10指标上分别提升了6.3%、13.2%、3.7%、6.5%。
To effectively integrate project ID embeddings and text embeddings, we propose a sequence recommendation algorithm called Time-frequency Fusion and Feature Cross-fusion Sequential Recommendation Algorithm (TFFCRec). RoBERTa is used to encode the project text, and a diversity mixture expert modulation method is applied to obtain distinguishable text representations. Project ID embeddings and text embeddings are combined using Fast Fourier Transform (FFT) and Short-Time Fourier Transform (STFT), extracting the user’s global frequency-domain features and local time-frequency features. This approach enables the algorithm to capture both the user’s long-term interest preferences and short-term interest variations. In addition, we introduce feature cross-fusion and use the optimized Mamba-like Linear Attention (OMLLA) to capture deeper non-linear relationships between features and extract more profound feature representations. We design a fusion network that adaptively learns the weights of different embedding representations and performs weighted fusion of the feature vectors obtained from FFT, STFT, and OMLLA. These fused features are then passed into SASRec for sequence recommendation. Experiments are carried out on the Instant Video, Beauty, Digital Music, and Tools and Home Improvement datasets. Compared with the benchmark methods, the proposed method in this paper has improved the Recall@10 metric by 6.3%, 13.2%, 3.7%, and 6.5% respectively.
| [1] | Kang, W. and McAuley, J. (2018) Self-Attentive Sequential Recommendation. 2018 IEEE International Conference on Data Mining (ICDM), Singapore, 17-20 November 2018, 197-206. https://doi.org/10.1109/icdm.2018.00035 |
| [2] | Liu, Y. (2019) Roberta: A Robustly Optimized Bert Pretraining Approach. arXiv: 1907.11692. |
| [3] | Xu, L., Tian, Z., Li, B., Zhang, J., Wang, D., Wang, H., et al. (2024) Sequence-Level Semantic Representation Fusion for Recommender Systems. Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, Boise, 21-25 October 2024, 5015-5022. https://doi.org/10.1145/3627673.3680037 |
| [4] | Church, K.W. (2016) Word2Vec. Natural Language Engineering, 23, 155-162. https://doi.org/10.1017/s1351324916000334 |
| [5] | Pennington, J., Socher, R. and Manning, C. (2014) Glove: Global Vectors for Word Representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, 25-29 October 2014, 1532-1543. https://doi.org/10.3115/v1/d14-1162 |
| [6] | Alaparthi, S. and Mishra, M. (2020) Bidirectional Encoder Representations from Transformers (BERT): A Sentiment analysis Odyssey. arXiv: 2007.01127. |
| [7] | Huang, J., Tang, D., Zhong, W., Lu, S., Shou, L., Gong, M., et al. (2021) WhiteningBERT: An Easy Unsupervised Sentence Embedding Approach. Findings of the Association for Computational Linguistics: EMNLP 2021, Punta Cana, 16-20 November 2021, 238-244. https://doi.org/10.18653/v1/2021.findings-emnlp.23 |
| [8] | Rendle, S., Freudenthaler, C. and Schmidt-Thieme, L. (2010) Factorizing Personalized Markov Chains for Next-Basket Recommendation. Proceedings of the 19th International Conference on World Wide Web, Raleigh, 26-30 April 2010, 811-820. https://doi.org/10.1145/1772690.1772773 |
| [9] | Jannach, D. and Ludewig, M. (2017) When Recurrent Neural Networks Meet the Neighborhood for Session-Based Recommendation. Proceedings of the Eleventh ACM Conference on Recommender Systems, Como, 27-31 August 2017, 306-310. https://doi.org/10.1145/3109859.3109872 |
| [10] | Tang, J. and Wang, K. (2018) Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding. Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, Marina Del Rey, 5-9 February 2018, 565-573. https://doi.org/10.1145/3159652.3159656 |
| [11] | Sun, F., Liu, J., Wu, J., Pei, C., Lin, X., Ou, W., et al. (2019) BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Trans-Former. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, 3-7 November 2019, 1441-1450. https://doi.org/10.1145/3357384.3357895 |
| [12] | Zhou, K., Wang, H., Zhao, W.X., Zhu, Y., Wang, S., Zhang, F., et al. (2020) S3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization. Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 19-23 October 2020, 1893-1902. https://doi.org/10.1145/3340531.3411954 |
| [13] | Xu, C., Zhao, P., Liu, Y., Sheng, V.S., Xu, J., Zhuang, F., et al. (2019) Graph Contextualized Self-Attention Network for Session-Based Recommendation. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, Macao SAR, 10-16 August 2019, 3940-3946. https://doi.org/10.24963/ijcai.2019/547 |
| [14] | Fan, X., Liu, Z., Lian, J., Zhao, W.X., Xie, X. and Wen, J. (2021) Lighter and Better: Low-Rank Decomposed Self-Attention Networks for Next-Item Recommendation. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 11-15 July 2021, 1733-1737. https://doi.org/10.1145/3404835.3462978 |
| [15] | Zhou, K., Yu, H., Zhao, W.X. and Wen, J. (2022) Filter-Enhanced MLP Is All You Need for Sequential Recommendation. Proceedings of the ACM Web Conference 2022, 25-29 April 2022, 2388-2399. https://doi.org/10.1145/3485447.3512111 |
| [16] | Hou, Y., Mu, S., Zhao, W.X., Li, Y., Ding, B. and Wen, J. (2022) Towards Universal Sequence Representation Learning for Recommender Systems. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington DC, 14-18 August 2022, 585-593. https://doi.org/10.1145/3534678.3539381 |
| [17] | Han, D., Wang, Z., Xia, Z., et al. (2024) Demystify Mamba in Vision: A Linear Attention Perspective. arXiv: 2405.16605. |
| [18] | Shi, D. (2024) TransNeXt: Robust Foveal Visual Perception for Vision Transformers. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 16-22 June 2024, 17773-17783. https://doi.org/10.1109/cvpr52733.2024.01683 |
| [19] | Du, X., Yuan, H., Zhao, P., Qu, J., Zhuang, F., Liu, G., et al. (2023) Frequency Enhanced Hybrid Attention Network for Sequential Recommendation. Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, Taipei City, 23-27 July 2023, 78-88. https://doi.org/10.1145/3539618.3591689 |
| [20] | Zhao, W.X., Mu, S., Hou, Y., Lin, Z., Chen, Y., Pan, X., et al. (2021) RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms. Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 1-5 November 2021, 4653-4664. https://doi.org/10.1145/3459637.3482016 |
| [21] | Paszke, A., Gross, S., Massa, F., et al. (2019) PyTorch: An Imperative Style, High-Performance Deep Learning Library. arXiv: 1912.01703. |
