|
|
基于解耦对比注意力的图像聚类算法研究
|
Abstract:
针对深度聚类中图像特征判别力不足、复杂背景干扰以及传统对比损失函数存在的正负耦合效应导致优化效率低下等问题，提出一种鲁棒且高效的图像聚类方法——解耦对比注意力聚类网络(DCACN)。首先，采用多裁剪数据增强策略，融合全局语义视图与局部细节视图来捕捉图像的多尺度不变性特征。然后，在ResNet-34骨干网络中无缝集成卷积块注意力模块，进而利用通道和空间注意力机制引导网络聚焦前景主体并抑制背景噪声。最后，引入解耦对比学习模块，消除正负样本间的梯度耦合来提升训练效率。在ImageNet-10、CIFAR-10和STL-10数据集上的实验结果表明，DCACN的聚类准确率分别达到了0.935、0.918和0.876，显著优于其他主流算法。
To address issues such as insufficient discriminative power of image features in deep clustering, interference from complex backgrounds, and the inefficiency of optimization caused by positive-negative coupling effects in traditional contrastive loss functions, a robust and efficient image clustering method—Decoupled Contrastive Attention Clustering Network (DCACN)—is proposed. First, a multi-crop data augmentation strategy is employed to integrate global semantic views and local detail views, capturing multi-scale invariant features of images. Then, the Convolutional Block Attention Module is seamlessly integrated into the ResNet-34 backbone network, utilizing channel and spatial attention mechanisms to guide the network in focusing on foreground subjects while suppressing background noise. Finally, a decoupled contrastive learning module is introduced to eliminate gradient coupling between positive and negative samples, thereby improving training efficiency. Experimental results on ImageNet-10, CIFAR-10, and STL-10 datasets demonstrate that DCACN achieves clustering accuracies of 0.935, 0.918, and 0.876, significantly outperforming other mainstream algorithms.
| [1] | Vincent, P., Larochelle, H., Bengio, Y., et al. (2023) Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion. Journal of Machine Learning Research, 11, 3371-3408. |
| [2] | Vaswani, A., Shazeer, N., Parmar, N., et al. (2017) Attention Is All You Need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, 4-9 December 2017, 6000-6010. |
| [3] | Hu, J., Shen, L. and Sun, G. (2018) Squeeze-and-Excitation Networks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-22 June 2018, 7132-7141. https://doi.org/10.1109/cvpr.2018.00745 |
| [4] | Xie, J., Girshick, R. and Farhadi, A. (2024) Unsupervised Deep Embedding for Clustering Analysis. International Conference on Machine Learning. PMLR, Vienna, 21-27 July 2024, 478-487. |
| [5] | Hinton, G.E. and Salakhutdinov, R.R. (2006) Reducing the Dimensionality of Data with Neural Networks. Science, 313, 504-507. https://doi.org/10.1126/science.1127647 |
| [6] | Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J. and Wojna, Z. (2016) Rethinking the Inception Architecture for Computer Vision. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 27-30 June 2016, 2818-2826. https://doi.org/10.1109/cvpr.2016.308 |
| [7] | Li, Y., Hu, P., Liu, Z., Peng, D., Zhou, J.T. and Peng, X. (2021) Contrastive Clustering. Proceedings of the AAAI Conference on Artificial Intelligence, 35, 8547-8555. https://doi.org/10.1609/aaai.v35i10.17037 |
| [8] | Woo, S., Park, J., Lee, J. and Kweon, I.S. (2018) CBAM: Convolutional Block Attention Module. In: Ferrari, V., et al., Eds., Proceedings of the European Conference on Computer Vision, Springer International Publishing, 3-19. https://doi.org/10.1007/978-3-030-01234-2_1 |
| [9] | He, K., Zhang, X., Ren, S. and Sun, J. (2016) Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 27-30 June 2016, 770-778. https://doi.org/10.1109/cvpr.2016.90 |
| [10] | MacQueen, J. (1967) Some Methods for Classification and Analysis of Multivariate Observations. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, 1, 281-297. |
| [11] | Reynolds, D.A. (2009) Gaussian Mixture Models. In: Li, S.Z. and Jain, A., Eds., Encyclopedia of Biometrics, Springer US, 659-663. https://doi.org/10.1007/978-0-387-73003-5_196 |
| [12] | Caron, M., Misra, I., Mairal, J., et al. (2020) Unsupervised Learning of Visual Features by Contrasting Cluster Assignments. Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, 6-12 December 2020, 9912-9924. |
| [13] | Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al. (2021) An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale. Proceedings of the 9th International Conference on Learning Representations, 3-7 May 2021, 1-15. |
| [14] | Guo, X., Gao, L., Liu, X. and Yin, J. (2017) Improved Deep Embedded Clustering with Local Structure Preservation. Proceedings of the 26th International Joint Conference on Artificial Intelligence, Melbourne, 19-25 August 2017, 1753-1759. https://doi.org/10.24963/ijcai.2017/243 |
| [15] | Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W. and Hu, Q. (2020) ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 13-19 June 2020, 11534-11542. https://doi.org/10.1109/cvpr42600.2020.01155 |
| [16] | Dizaji, K.G., Herandi, A., Deng, C., Cai, W. and Huang, H. (2017) Deep Clustering via Joint Convolutional Autoencoder Embedding and Relative Entropy Minimization. 2017 IEEE International Conference on Computer Vision (ICCV), Venice, 22-29 October 2017, 5736-5745. https://doi.org/10.1109/iccv.2017.612 |
| [17] | Kingma, D.P. and Welling, M. (2014) Auto-Encoding Variational Bayes. Proceedings of the International Conference on Learning Representations, Banff, 14-16 April 2014, 1-14. |
| [18] | Guo, X., Liu, X., Zhu, E. and Yin, J. (2017) Deep Clustering with Convolutional Autoencoders. In: Liu, D.R., et al., Eds., Neural Information Processing, Springer International Publishing, 373-382. https://doi.org/10.1007/978-3-319-70096-0_39 |
| [19] | Blum, A. and Mitchell, T. (1998) Combining Labeled and Unlabeled Data with Co-Training. Proceedings of the 11th Annual Conference on Computational Learning Theory, Madisson, 24-26 July 1998, 92-100. https://doi.org/10.1145/279943.279962 |
| [20] | Jiang, Z., Zheng, Y., Tan, H., Tang, B. and Zhou, H. (2017) Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering. Proceedings of the 26th International Joint Conference on Artificial Intelligence, Melbourne, 19-25 August 2017, 1965-1972. https://doi.org/10.24963/ijcai.2017/273 |
| [21] | Almahairi, A., Ballas, N., Cooijmans, T., et al. (2016) Dynamic Capacity Networks. Proceedings of the 33rd International Conference on Machine Learning, New York, 20-22 June 2016, 1228-1237. |
| [22] | Zagoruyko, S. and Komodakis, N. (2016) Wide Residual Networks. In: Proceedings of the British Machine Vision Conference 2016, BMVA Press, 87.1-87.12. https://doi.org/10.5244/c.30.87 |
| [23] | Ji, X., Vedaldi, A. and Henriques, J. (2019) Invariant Information Clustering for Unsupervised Image Classification and Segmentation. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, 27-28 October 2019, 9865-9874. https://doi.org/10.1109/iccv.2019.00996 |
| [24] | Huang, J., Gong, S. and Zhu, X. (2020) Deep Semantic Clustering by Partition Confidence Maximisation. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 13-19 June 2020, 8849-8858. https://doi.org/10.1109/cvpr42600.2020.00887 |
