|
海洋环境下具有原型的集成实例分割网络
|
Abstract:
随着人类对海洋世界的探索活动日益增多,自动检测与识别海洋物体愈发重要。相对于仅仅获得目标大小与位置信息,海洋生物的实例分割更具价值,因为其可以进一步提供目标的形状信息。基于上述背景,本文提出了一种新的方法,通过整合原型模块到实例分割模型中,以获得更好的性能。原型模块由原型训练与原型区域两部分组成来保证原型的类代表性与边界稳定性。其中,我们通过随机挑选部分异类相似样本来训练原型区域模块,来更好地模拟类原型边界。最终,我们将原型模块融合到传统实例分割模型中,以实现更准确的海洋生物分割。实验结果表明,我们提出的原型整合方法在海洋数据集上取得了显著的精度提升,并能更好地区分异类相似样本,从而有效改善了实例分割模型的性能。
The importance of automatically detecting and identifying marine objects has increased in tandem with the rise in human exploration activities in the maritime environment. Compared to obtaining only target size and position information, instance segmentation of marine organisms is more valuable as it can further provide shape information about the targets. This paper proposes a new method by integrating a prototype module into the instance segmentation model to achieve better performance. The prototype module consists of prototype training and prototype region to ensure the class representativeness and boundary stability of the prototypes. Specifically, we train the prototype region module by randomly selecting some dissimilar samples to better simulate the boundary of class prototypes. Ultimately, we integrate the prototype module into the traditional instance segmentation model to achieve more accurate segmentation of marine organisms. Experimental results demonstrate that our proposed prototype integration method achieves significant accuracy improvement on the marine dataset and better distinguishes dissimilar samples, effectively improving the performance of the instance segmentation model.
[1] | He, K., Zhang, X., Ren, S., et al. (2016) Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 27-30 June 2016, 770-778. https://doi.org/10.1109/CVPR.2016.90 |
[2] | Liu, Z., Lin, Y., Cao, Y., et al. (2021) Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, 10-17 October 2021, 9992-10002. https://doi.org/10.1109/ICCV48922.2021.00986 |
[3] | Ge, Z., Liu, S., Wang, F., et al. (2021) YOLOX: Exceeding YOLO Series in 2021. arXiv: 2107.08430. |
[4] | Girshick, R. (2015) Fast R-CNN. 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, 7-13 December 2015, 1440-1448. https://doi.org/10.1109/ICCV.2015.169 |
[5] | Weng, W. and Zhu, X. (2021) INet: Convolutional Networks for Biomedical Image Segmentation. IEEE Access, 9, 16591-16603. https://doi.org/10.1109/ACCESS.2021.3053408 |
[6] | Ma, Y., Meng, J., Sun, L., et al. (2023) Oceanic Internal Wave Signature Extraction in the Sulu Sea by a Pixel Attention U-Net: PAU-Net. IEEE Geoscience and Remote Sensing Letters, 20, 1-5. https://doi.org/10.1109/LGRS.2022.3230086 |
[7] | He, K., Gkioxari, G., Dollar, P., et al. (2017) Mask R-CNN. International Conference on Computer Vision, Venice, 22-29 October 2017, 2980-2988. https://doi.org/10.1109/ICCV.2017.322 |
[8] | Xie, E., Sun, P., Song, X., et al. (2020) PolarMask: Single Shot Instance Segmentation with Polar Representation. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 13-19 June 2020, 12190-12199. https://doi.org/10.1109/CVPR42600.2020.01221 |
[9] | Wang, X., Kong, T., Shen, C., et al. (2020) SOLO: Segmenting Objects by Locations. In: Vedaldi, A., Bischof, H., Brox, T. and Frahm, JM., Eds., Computer Vision ECCV 2020, Springer, Cham. https://doi.org/10.1007/978-3-030-58523-5_38 |
[10] | Gu, Z., Chen, H., Xu, Z., et al. (2022) DiffusionInst: Diffusion Model for Instance Segmentation. arXiv: 2212.02773. |
[11] | Siddiqui, S.A., Salman, A., Malik, M.I., Shafait, F., Mian, A., Shortis, M.R. and Harvey, E.S. (2018) Automatic Fish Species Classification in Underwater Videos: Exploiting Pre-Trained Deep Neural Network Models to Compensate for Limited Labelled Data. ICES Journal of Marine Science, 75, 374-389. https://doi.org/10.1093/icesjms/fsx109 |
[12] | Tseng, C.-H., Hsieh, C.-L. and Kuo, Y.-F. (2020) Automatic Measurement of the Body Length of Harvested Fish Using Convolutional Neural Networks. Biosystems Engineering, 189, 36-47. https://doi.org/10.1016/j.biosystemseng.2019.11.002 |
[13] | Yang, H.M., Zhang, X.Y., Yin, F., et al. (2018) Robust Classification with Convolutional Prototype Learning. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-23 June 2018, 3474-3482. https://doi.org/10.1109/CVPR.2018.00366 |
[14] | Ma, C., Chen, L., Yang, C., et al. (2019) A Deep Learning Based Personnel Positioning System for Key Cabin of Ship. 2019 International Conference on Intelligent Computing, Automation and Systems (ICICAS), Chongqing, 6-8 December 2019, 492-496. https://doi.org/10.1109/ICICAS48597.2019.00108 |
[15] | Hong, J., Fulton, M. and Sattar, J. (2020) TrashCan: A Semantically-Segmented Dataset towards Visual Detection of Marine Debris. |
[16] | Kirillov, A., Wu, Y., He, K., et al. (2020) PointRend: Image Segmentation as Rendering. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 13-19 June 2020, 9796-9805. https://doi.org/10.1109/CVPR42600.2020.00982 |
[17] | Brabandere, B.D., Neven, D. and Gool, L.V. (2017) Semantic Instance Segmentation with a Discriminative Loss Function. arXiv: 1708.02551. |
[18] | Bolya, D., Zhou, C., Xiao, F., et al. (2019) YOLACT : Real-Time Instance Segmentation. arXiv: 1912.06218. |
[19] | Reus, G., Moller, T., Jager, J., et al. (2018) Looking for Seagrass: Deep Learning for Visual Coverage Estimation. 2018 OCEANS-MTS/IEEE Kobe Techno-Ocean (OTO), Kobe, 28-31 May 2018, 1-6. https://doi.org/10.1109/OCEANSKOBE.2018.8559302 |
[20] | Lu, J., Xu, Y., Li, H., et al. (2022) PMAL: Open Set Recognition via Robust Prototype Mining. Proceedings of the AAAI Conference on Artificial Intelligence, 36, 1872-1880. https://doi.org/10.1609/aaai.v36i2.20081 |
[21] | Li, J., Zhou, P., Xiong, C., et al. (2020) Prototypical Contrastive Learning of Unsupervised Representations. arXiv: 2005.04966. |