Industrial appearance anomaly detection (AD) focuses on accurately identifying and locating abnormal regions in images. However, due to issues such as scarce abnormal samples, complex abnormal manifestations, and difficult abnormal annotation, the detection accuracy is limited. To solve these problems, based on the knowledge distillation framework, this paper proposes an unsupervised anomaly detection algorithm—Bidirectional knowledge distillation AD (BKD). This algorithm combines the advantages of forward and reverse distillation, enabling efficient anomaly detection. Experimental results have shown that the proposed method outperforms the state-of-the-art AD methods by 3% - 8% in AUC on the MVTec benchmarks.
References
[1]
Tao, X., Gong, X., Zhang, X., Yan, S. and Adak, C. (2022) Deep Learning for Unsupervised Anomaly Localization in Industrial Images: A Survey. IEEE Transactions on Instrumentation and Measurement, 71, 1-21. https://doi.org/10.1109/tim.2022.3196436
[2]
Chalapathy, R., Khoa, N.L.D. and Chawla, S. (2020). Robust Deep Learning Methods for Anomaly Detection. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, California, 6-10 July 2020, 3507-3508. https://doi.org/10.1145/3394486.3406704
[3]
Erfani, S.M., Rajasegarar, S., Karunasekera, S. and Leckie, C. (2016) High-Dimensional and Large-Scale Anomaly Detection Using a Linear One-Class SVM with Deep Learning. Pattern Recognition, 58, 121-134. https://doi.org/10.1016/j.patcog.2016.03.028
[4]
Ruff, L., Vandermeulen, R., Goernitz, N., Deecke, L., Siddiqui, S.A., Binder, A. and Kloft, M. (2018) Deep One-Class Classification. 2018 International Conference on Machine Learning, Stockholm, 10-15 July 2018, 4393-4402.
[5]
Zhou, Y., Liang, X., Zhang, W., Zhang, L. and Song, X. (2021) Vae-Based Deep SVDD for Anomaly Detection. Neurocomputing, 453, 131-140. https://doi.org/10.1016/j.neucom.2021.04.089
[6]
Yi, J. and Yoon, S. (2021) Patch SVDD: Patch-Level SVDD for Anomaly Detection and Segmentation. In: Lecture Notes in Computer Science, Springer, 375-390. https://doi.org/10.1007/978-3-030-69544-6_23
[7]
Defard, T., Setkov, A., Loesch, A. and Audigier, R. (2021) Padim: A Patch Distribution Modeling Framework for Anomaly Detection and Localization. In: Lecture Notes in Computer Science, Springer, 475-489. https://doi.org/10.1007/978-3-030-68799-1_35
[8]
Cohen, N. and Hoshen, Y. (2020) Sub-Image Anomaly Detection with Deep Pyramid Correspondences. https://arxiv.org/pdf/2005.02357
[9]
Chebotar, Y. and Waters, A. (2016) Distilling Knowledge from Ensembles of Neural Networks for Speech Recognition. InterSpeech 2016, San Francisco, 8-12 September 2016, 3439-3443. https://doi.org/10.21437/interspeech.2016-1190
[10]
Chen, P., Liu, S., Zhao, H. and Jia, J. (2021) Distilling Knowledge via Knowledge Review. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, 20-25 June 2021, 5006-5015. https://doi.org/10.1109/cvpr46437.2021.00497
[11]
Bergmann, P., Fauser, M., Sattlegger, D. and Steger, C. (2020) Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddings. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 13-19 June 2020, 4182-4191. https://doi.org/10.1109/cvpr42600.2020.00424
[12]
Salehi, M., Sadjadi, N., Baselizadeh, S., Rohban, M.H. and Rabiee, H.R. (2021) Multiresolution Knowledge Distillation for Anomaly Detection. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, 20-25 June 2021, 14897-14907. https://doi.org/10.1109/cvpr46437.2021.01466
[13]
Yamada, S. and Hotta, K. (2021) Reconstruction Student with Attention for Student-Teacher Pyramid Matching. https://arxiv.org/pdf/2111.15376
[14]
Deng, H. and Li, X. (2022) Anomaly Detection via Reverse Distillation from One-Class Embedding. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, 18-24 June 2022, 9727-9736. https://doi.org/10.1109/cvpr52688.2022.00951
[15]
Gu, Z., Liu, L., Chen, X., Yi, R., Zhang, J., Wang, Y., et al. (2023) Remembering Normality: Memory-Guided Knowledge Distillation for Unsupervised Anomaly Detection. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, 1-6 October 2023, 16355-16363. https://doi.org/10.1109/iccv51070.2023.01503
[16]
Liu, X., Wang, J., Leng, B. and Zhang, S. (2024) Dual-Modeling Decouple Distillation for Unsupervised Anomaly Detection. Proceedings of the 32nd ACM International Conference on Multimedia, Melbourne, 28 October-1 November 2024, 5035-5044. https://doi.org/10.1145/3664647.3681669
[17]
Cao, W. and Chen, X. (2019) Deformable Convolutional Networks Tracker. DEStech Transactions on Computer Science and Engineering, 2475-8841. https://doi.org/10.12783/dtcse/iteee2019/28747
[18]
Zhu, X., Hu, H., Lin, S. and Dai, J. (2019) Deformable ConvNets V2: More Deformable, Better Results. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 9300-9308. https://doi.org/10.1109/cvpr.2019.00953
[19]
Batagelj, V., Doreian, P., et al. (2014) Understanding Large Temporal Networks and Spatial Networks. Wiley.
[20]
Bergmann, P., Fauser, M., Sattlegger, D. and Steger, C. (2019) Mvtec AD—A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 9584-9592. https://doi.org/10.1109/cvpr.2019.00982
[21]
Jezek, S., Jonak, M., Burget, R., Dvorak, P. and Skotak, M. (2021) Deep Learning-Based Defect Detection of Metal Parts: Evaluating Current Methods in Complex Conditions. 2021 13th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT), Brno, 25-27 October 2021, 66-71. https://doi.org/10.1109/icumt54235.2021.9631567
[22]
Rudolph, M., Wandt, B. and Rosenhahn, B. (2021) Same but Different: Semi-Supervised Defect Detection with Normalizing Flows. 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa, 3-8 January 2021, 1906-1915. https://doi.org/10.1109/wacv48630.2021.00195
