|
基于迁移学习的糖尿病视网膜病变眼底图像合成研究
|
Abstract:
文章研究了利用生成对抗网络(GAN)和迁移学习在小数据集上生成糖尿病视网膜病变(DR)眼底图像的有效方法。传统的图像生成技术在高维数据下表现不佳,难以准确学习数据分布。自2014年GAN被提出以来,众多研究致力于其优化与应用,然而在小样本情况下仍面临训练难度和过拟合问题。文章通过迁移学习方法,利用在大数据集上预训练的GAN模型,结合条件生成对抗网络(CGAN),在小数据集上重新训练部分参数,从而生成高清晰度的DR图像,并提高生成图像的类别控制能力。研究表明,采用迁移学习的方法显著提升了图像生成质量,实验结果显示,基于迁移学习的GAN和CGAN在小数据集上的FID分数明显优于无迁移学习的对照组,表明生成的图像在质量和多样性上有显著改善。这项研究不仅为医学图像生成提供了新的思路,也为解决小数据集和不平衡问题提供了有效的解决方案。
This paper investigates an effective method for generating fundus images of diabetic retinopathy (DR) on small datasets using Generative Adversarial Networks (GANs) and transfer learning. Traditional image generation techniques struggle to accurately learn data distributions in high-dimensional data, leading to poor performance. Since the introduction of GANs in 2014, numerous studies have focused on their optimization and applications; however, they still face challenges such as training difficulties and overfitting in small sample cases. This study employs transfer learning by utilizing a GAN model pre-trained on a large dataset, combined with Conditional Generative Adversarial Networks (CGAN), to retrain some parameters on a small dataset. This approach generates high-quality DR images and enhances the class control capability of the generated images. Research shows that using transfer learning significantly improves the quality of image generation. Experimental results indicate that the FID scores of GANs and CGANs based on transfer learning are notably superior to those of the control group without transfer learning, demonstrating significant improvements in the quality and diversity of the generated images. This research not only provides new insights for medical image generation but also offers effective solutions for addressing small datasets and imbalance issues.
[1] | 陈佛计, 朱枫, 吴清潇, 等. 生成对抗网络及其在图像生成中的应用研究综述[J]. 计算机学报, 2021(44): 347-369. |
[2] | Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., et al. (2014) Generative Adversarial Networks. 27th International Conference on Neural Information Processing Systems, Bangkok, 8-13 December 2014, 2672-2680. |
[3] | Singh, N.K. and Raza, K. (2021) Medical Image Generation Using Generative Adversarial Networks: A Review. In: Studies in Computational Intelligence, Springer, 77-96. https://doi.org/10.1007/978-981-15-9735-0_5 |
[4] | Chuquicusma, M.J.M., Hussein, S., Burt, J. and Bagci, U. (2018) How to Fool Radiologists with Generative Adversarial Networks? A Visual Turing Test for Lung Cancer Diagnosis. 2018 IEEE 15th International Symposium on Biomedical Imaging, Washington, 4-7 April 2018, 240-244. https://doi.org/10.1109/isbi.2018.8363564 |
[5] | Frid-Adar, M., Diamant, I., Klang, E., Amitai, M., Goldberger, J. and Greenspan, H. (2018) Gan-Based Synthetic Medical Image Augmentation for Increased CNN Performance in Liver Lesion Classification. Neurocomputing, 321, 321-331. https://doi.org/10.1016/j.neucom.2018.09.013 |
[6] | 林志鹏, 曾立波, 吴琼水. 基于生成对抗网络的宫颈细胞图像数据增强[J]. 科学技术与工程, 2020, 20(28): 11672-11677. |
[7] | Plassard, A.J., Davis, L.T., Newton, A.T., Resnick, S.M., Landman, B.A. and Bermudez, C. (2018) Learning Implicit Brain MRI Manifolds with Deep Learning. Medical Imaging 2018: Image Processing, Houston, 11-13 February 2018, Article 105741. https://doi.org/10.1117/12.2293515 |
[8] | Jin, D., Xu, Z., Tang, Y., et al. (2018) CT-Realistic Lung Nodule Simulation from 3D Conditional Generative Adversarial Networks for Robust Lung Segmentation. https://ui.adsabs.harvard.edu/abs/2018arXiv180604051J |
[9] | Mok, T.C.W. and Chung, A.C.S. (2019) Learning Data Augmentation for Brain Tumor Segmentation with Coarse-to-Fine Generative Adversarial Networks. In: Lecture Notes in Computer Science, Springer, 70-80. https://doi.org/10.1007/978-3-030-11723-8_7 |
[10] | Chartsias, A., Joyce, T., Dharmakumar, R. and Tsaftaris, S.A. (2017) Adversarial Image Synthesis for Unpaired Multi-Modal Cardiac Data. In: Lecture Notes in Computer Science, Springer, 3-13. https://doi.org/10.1007/978-3-319-68127-6_1 |
[11] | Maspero, M., Savenije, M.H.F., Dinkla, A.M., Seevinck, P.R., Intven, M.P.W., Jurgenliemk-Schulz, I.M., et al. (2018) Dose Evaluation of Fast Synthetic-CT Generation Using a Generative Adversarial Network for General Pelvis MR-Only Radiotherapy. Physics in Medicine & Biology, 63, Article 185001. https://doi.org/10.1088/1361-6560/aada6d |
[12] | Yang, H., Sun, J., Carass, A., Zhao, C., Lee, J., Xu, Z., et al. (2018) Unpaired Brain MR-to-CT Synthesis Using a Structure-Constrained Cyclegan. In: Lecture Notes in Computer Science, Springer, 174-182. https://doi.org/10.1007/978-3-030-00889-5_20 |
[13] | Dugas, E., Jared, Jorge and Cukierski, W. (2020) Diabetic Retinopathy Detection. https://www.kaggle.com/c/diabetic-retinopathy-detection |
[14] | Zhou, Y., Wang, B., He, X., et al. (2020) DR-GAN: Conditional Generative Adversarial Network for Fine-Grained Lesion Synthesis on Diabetic Retinopathy Images. IEEE Journal of Biomedical and Health Informatics, 26, 56-66. |
[15] | 马岽奡, 唐娉, 赵理君, 等. 深度学习图像数据增广方法研究综述[J]. 中国图象图形学报, 2021(26): 487-502. |
[16] | Pan, S.J. and Yang, Q. (2010) A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering, 22, 1345-1359. https://doi.org/10.1109/tkde.2009.191 |
[17] | Zhao, M., Cong, Y. and Carin, L. (2020) On Leveraging Pretrained GANs for Generation with Limited Data. https://ui.adsabs.harvard.edu/abs/2020arXiv200211810Z |
[18] | Mo, S., Cho, M. and Shin, J. (2020) Freeze the Discriminator: A Simple Baseline for Fine-Tuning GANs. https://ui.adsabs.harvard.edu/abs/2020arXiv200210964M |
[19] | Giacomello, E., Loiacono, D. and Mainardi, L. (2020) Brain MRI Tumor Segmentation with Adversarial Networks. 2020 International Joint Conference on Neural Networks, Glasgow, 19-24 July 2020, 1-8. https://doi.org/10.1109/ijcnn48605.2020.9207220 |
[20] | Mirza, M. and Osindero, S. (2014) Conditional Generative Adversarial Nets. Computer Science, 2014, 2672-2680. |
[21] | Karras, T., Laine, S. and Aila, T. (2019) A Style-Based Generator Architecture for Generative Adversarial Networks. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, 15-20 June 2019, 4396-4405. https://doi.org/10.1109/cvpr.2019.00453 |
[22] | Brock, A., Donahue, J. and Simonyan, K. (2018) Large Scale GAN Training for High Fidelity Natural Image Synthesis. https://ui.adsabs.harvard.edu/abs/2018arXiv180911096B |
[23] | Karras, T., Aila, T., Laine, S., et al. (2018) Progressive Growing of GANs for Improved Quality, Stability, and Variation. 6th International Conference on Learning Representations, Vancouver, 30 April-3 May 2018, 10-15. |
[24] | Choe, J., Oh, S.J., Lee, S., Chun, S., Akata, Z. and Shim, H. (2020) Evaluating Weakly Supervised Object Localization Methods Right. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, 13-19 June 2020, 3130-3139. https://doi.org/10.1109/cvpr42600.2020.00320 |