|
|
基于时空特征与动态融合的多序列MRI肝脏肿瘤分类
|
Abstract:
多序列磁共振成像在肝脏肿瘤诊断中提供了丰富的时空信息，但现有分析方法存在跨序列信息交互不足和时空特征整合不充分的问题。针对该问题，本文提出了一种基于时空特征与动态融合的Transformer模型。该模型根据多序列MRI的成像特性对强相关序列进行分组处理，通过动态瓶颈桥接模块在不同序列间建立信息交互通道，结合动态权重机制调节序列重要性；同时采用时空特征协同融合模块利用双向注意力机制实现空间与时间特征的语义对齐，并通过融合策略实现时空特征的有效整合。在LLD-MRI 2023数据集上的实验表明，该方法取得了84.62%的准确率、96.58%的曲线下面积、84.04%的F1分数和81.13%的Kappa系数，在多数指标上优于现有方法。
Multi-sequence magnetic resonance imaging provides rich spatio-temporal information for liver tumor diagnosis, but existing analysis methods suffer from insufficient cross-sequence information interaction and inadequate spatio-temporal feature integration. To address these problems, a Transformer model based on spatio-temporal features and dynamic fusion is proposed in this paper. The model groups strongly correlated sequences according to the imaging characteristics of multi-sequence MRI, establishes information interaction channels between different sequences through the Dynamic Bottleneck Bridge module, and adjusts sequence importance by combining a dynamic weighting mechanism. Meanwhile, the Spatio-Temporal Fusion module is employed to achieve semantic alignment of spatial and temporal features using a bidirectional attention mechanism and realizes effective integration of spatio-temporal features through a fusion strategy. Experiments on the LLD-MRI 2023 dataset demonstrate that the method achieves 84.62% accuracy, 96.58% area under the curve, 84.04% F1-score, and 81.13% Kappa coefficient, outperforming existing methods on most metrics.
| [1] | Ferlay, J., Colombet, M., Soerjomataram, I., Parkin, D.M., Piñeros, M., Znaor, A., et al. (2021) Cancer Statistics for the Year 2020: An Overview. International Journal of Cancer, 149, 778-789. https://doi.org/10.1002/ijc.33588 |
| [2] | Lee, S., Kim, Y., Shin, J., Hwang, S.H., Roh, Y.H., Chung, Y.E., et al. (2020) CT and MRI Liver Imaging Reporting and Data System Version 2018 for Hepatocellular Carcinoma: A Systematic Review with Meta-Analysis. Journal of the American College of Radiology, 17, 1199-1206. https://doi.org/10.1016/j.jacr.2020.06.005 |
| [3] | 赵燕风, 欧阳汉. 肝脏肿瘤的MR检查与诊断思路[J]. 磁共振成像, 2012, 3(6): 456-464. |
| [4] | Wojciechowska, M., Irving, B., Dennis, A., Wilman, H.R., Banerjee, R., Brady, S.M., et al. (2018) Automated Detection of Cystic Lesions in Quantitative T1 Liver Images. In: Nixon, M., Mahmoodi, S. and Zwiggelaar, R., Eds., Medical Image Understanding and Analysis, Springer, 51-56. https://doi.org/10.1007/978-3-319-95921-4_6 |
| [5] | Chlebus, G., Meine, H., Abolmaali, N., et al. (2018) Automatic Liver and Tumor Segmentation in Late-Phase MRI Using Fully Convolutional Neural Networks. Proceedings of the Annual Meeting of the German Society of Computer- and Robot-Assisted Surgery, Leipzig, September 2018, 195-200. |
| [6] | Trivizakis, E., Manikis, G.C., Nikiforaki, K., Drevelegas, K., Constantinides, M., Drevelegas, A., et al. (2019) Extending 2-D Convolutional Neural Networks to 3-D for Advancing Deep Learning Cancer Classification with Application to MRI Liver Tumor Differentiation. IEEE Journal of Biomedical and Health Informatics, 23, 923-930. https://doi.org/10.1109/jbhi.2018.2886276 |
| [7] | Mo, S., Cai, M., Lin, L., Tong, R., Chen, Q., Wang, F., et al. (2020) Multimodal Priors Guided Segmentation of Liver Lesions in MRI Using Mutual Information Based Graph Co-Attention Networks. In: Martel, A.L., et al., Eds., Medical Image Computing and Computer Assisted Intervention—MICCAI 2020, Springer, 429-438. https://doi.org/10.1007/978-3-030-59719-1_42 |
| [8] | Xiao, X., Hu, Q.V. and Wang, G. (2023) Edge-Aware Multi-Task Network for Integrating Quantification Segmentation and Uncertainty Prediction of Liver Tumor on Multi-Modality Non-Contrast MRI. In: Greenspan, H., et al., Eds., Medical Image Computing and Computer Assisted Intervention—MICCAI 2023, Springer, 652-661. https://doi.org/10.1007/978-3-031-43901-8_62 |
| [9] | 胡瑶, 王远军. 一种双输入稀疏连接的肝细胞癌和肝内胆管细胞癌分类网络[J]. 小型微型计算机系统, 2024, 45(5): 1129-1135. |
| [10] | Qiu, Z., Pan, Y., Wei, J., Wu, D., Xia, Y. and Shen, D. (2021) Predicting Symptoms from Multiphasic MRI via Multi-Instance Attention Learning for Hepatocellular Carcinoma Grading. In: de Bruijne, M., et al., Eds., Medical Image Computing and Computer Assisted Intervention—MICCAI 2021, Springer, 439-448. https://doi.org/10.1007/978-3-030-87240-3_42 |
| [11] | Kumar, V., Li, Y. and Zhang, S. (2025) Uncertainty-Guided and Cross-Modality Attention Network for Liver Tumor Segmentation and Classification. Expert Systems with Applications, 212, Article ID: 120069. |
| [12] | Phan, A., Cao, H., Le, T., Trieu, T. and Phan, T. (2023) Improving Liver Lesions Classification on CT/MRI Images Based on Hounsfield Units Attenuation and Deep Learning. Gene Expression Patterns, 47, Article ID: 119289. https://doi.org/10.1016/j.gep.2022.119289 |
| [13] | Wang, S., Han, X., Du, J., Wang, Z., Yuan, C., Chen, Y., et al. (2021) Saliency-Based 3D Convolutional Neural Network for Categorising Common Focal Liver Lesions on Multisequence MRI. Insights into Imaging, 12, Article No. 173. https://doi.org/10.1186/s13244-021-01117-z |
| [14] | Lou, M., Ying, H., Liu, X., Zhou, H., Zhang, Y. and Yu, Y. (2025) SDR-Former: A Siamese Dual-Resolution Transformer for Liver Lesion Classification Using 3D Multi-Phase Imaging. Neural Networks, 185, Article ID: 107228. https://doi.org/10.1016/j.neunet.2025.107228 |
| [15] | Wang, R., Shi, X., Pang, S., Chen, Y., Zhu, X., Wang, W., et al. (2025) Cross-Attention Guided Loss-Based Deep Dual-Branch Fusion Network for Liver Tumor Classification. Information Fusion, 114, Article ID: 102713. https://doi.org/10.1016/j.inffus.2024.102713 |
| [16] | Huang, S., Hong, Z., Wu, B., Liang, J. and Huang, Q. (2025) Spatio-Temporal Collaborative Multiple-Stream Transformer Network for Liver Lesion Classification on Multiple-Sequence Magnetic Resonance Imaging. Engineering Applications of Artificial Intelligence, 142, Article ID: 109933. https://doi.org/10.1016/j.engappai.2024.109933 |
| [17] | Hu, M., Wang, S., Wu, M., et al. (2024) Automatic Classification of Focal Liver Lesions Based on Multi-Sequence MRI. Journal of Imaging Informatics in Medicine, 38, 1986-1998. |
| [18] | He, K., Zhang, X., Ren, S. and Sun, J. (2016) Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 27-30 June 2016, 770-778. https://doi.org/10.1109/cvpr.2016.90 |
| [19] | Huang, G., Liu, Z., Van Der Maaten, L. and Weinberger, K.Q. (2017) Densely Connected Convolutional Networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 21-26 July 2017, 2261-2269. https://doi.org/10.1109/cvpr.2017.243 |
| [20] | Srinivas, A., Lin, T., Parmar, N., Shlens, J., Abbeel, P. and Vaswani, A. (2021) Bottleneck Transformers for Visual Recognition. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, 20-25 June 2021, 16514-16524. https://doi.org/10.1109/cvpr46437.2021.01625 |
| [21] | Li, K., Wang, Y., Gao, P., et al. (2022) UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learning. arXiv: 2201.04676. |
| [22] | He, A., Wang, K., Li, T., Du, C., Xia, S. and Fu, H. (2023) H2Former: An Efficient Hierarchical Hybrid Transformer for Medical Image Segmentation. IEEE Transactions on Medical Imaging, 42, 2763-2775. https://doi.org/10.1109/tmi.2023.3264513 |
| [23] | Xu, X., Wang, C., Guo, J., Gan, Y., Wang, J., Bai, H., et al. (2020) MSCS-DeepLN: Evaluating Lung Nodule Malignancy Using Multi-Scale Cost-Sensitive Neural Networks. Medical Image Analysis, 65, Article ID: 101772. https://doi.org/10.1016/j.media.2020.101772 |
| [24] | Xing, Z., Ye, T., Yang, Y., Liu, G. and Zhu, L. (2024) SegMamba: Long-Range Sequential Modeling Mamba for 3D Medical Image Segmentation. In: Linguraru, M.G., et al., Eds., Medical Image Computing and Computer Assisted Intervention—MICCAI 2024, Springer, 578-588. https://doi.org/10.1007/978-3-031-72111-3_54 |
