一种结合自注意力和门控机制的图像超分辨率重建算法
An Image Super-Resolution Reconstruction Algorithm Combining Self-Attention and Gating Mechanism

DOI: 10.12677/CSA.2020.1012245, PP. 2323-2330

Keywords: 图像超分辨率重建，残差网络，自注意力机制，门控机制，特征提取
Image Super Resolution Reconstruction, Residual Network, Self Attention Mechanism, Gating Mechanism, Feature Extraction

Full-Text   Cite this paper   Add to My Lib

Abstract:

图像超分辨率重建旨在将低分辨率图像重建为更加清晰的高分辨率图像。超分辨率重建算法有助于提高图像质量，可以尽可能精确地恢复出原始图像缺失的纹理、细节信息，在图像处理领域具有重要的科学意义和应用价值。为了进一步提高图像重建质量，本文将稀疏表示以及深度学习算法相结合，利用稀疏表示模型得到的重构高分辨率图像作为深度学习模型的输入，在VDSR网络的基础上减少卷积层并引入自注意力机制以及门控机制，模型可以在训练过程中动态学习到不同特征的重要性，从而进一步丰富图像的特征。我们在Set5、set14、B100、Urban100等公开的超分重建数据集上进行了大量的实验，结果表明，本文提出的基于自注意力机制和门控机制残差网络图像超分辨率重建算法相较于现有的重建方法，可以获得更好的重建细节以及更高的PSNR/SSIM值。
Image super resolution reconstruction aims to reconstruct a low resolution image into a clearer high-resolution image. The super resolution reconstruction algorithm is helpful to improve the image quality and can recover the missing texture and detail information as accurately as possible. It has important scientific significance and application value in the field of image processing. In order to further improve the quality of image reconstruction, this paper combines the sparse representation and deep learning algorithm. The reconstruction of the sparse representation model is used to get the high resolution image as input of deep learning model, and on the basis of introducing the VDSR network since attention mechanism and gating mechanism, models can be dynamically in the process of training to learn the importance of different characteristics. Thus, the pixel size and characteristics of granularity further enrich the characteristics of the image. We carried out a large number of experiments on the public super-fractional reconstruction data sets, such as Set5, SET14, B100 and Urban100. The results show that the multi-granularity feature extraction reconstruction algorithm proposed in this paper can obtain better reconstruction details and higher PSNR/SSIM values compared with the existing reconstruction methods.

References

[1]	Dai, S., Han, M., Xu, W., Wu, Y. and Gong, Y. (2007) Soft Edge Smoothness Prior for Alpha Channel Super Resolu-tion. Proceedings of the IEEE Conference on Computer Vision and Pattern Classification (CVPR), Minneapolis, 17-22 June 2007, 1-8. https://doi.org/10.1109/CVPR.2007.383028
[2]	Sun, J., Xu, Z. and Shum, H. (2008) Image Su-per-Resolution Using Gradient Profile Prior. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Anchorage, 24-26 June 2008, 1-8.
[3]	Yang, J.C., Wright, J., Huang, T., et al. (2010) Image Su-per-Resolution via Sparse Representation. IEEE Transactions on Image Processing, 19, 2861-2873. https://doi.org/10.1109/TIP.2010.2050625
[4]	Yeganli, F., Nazzal, M., Unal, M. and Ozkaramanli, H. (2014) Image Super Resolution Viasparse Representation over Coupled Dictionary Learning Based on Patch Sharpness. Euro-pean Modelling Symposium, Prague, 21-23 October 2014, 203-208. https://doi.org/10.1109/EMS.2014.67
[5]	Zhang, Y., Wu, W., Dai, Y., Yang, X., Yan, B. and Lu, W. (2013) Re-mote Sensing Images Super-Resolution Based on Sparse Dictionaries and Residual Dictionaries. IEEE 11th International Conference on Dependable, Autonomic and Secure Computing, Chengdu, 21-22 December 2013, 318-323. https://doi.org/10.1109/DASC.2013.82
[6]	Fu, C.-H., Chen, H., Zhang, H. and Chan, Y.-L. (2014) Single Image Super Resolution Based on Sparse Representation and Adaptive Dictionary Selection. 19th International Conference on Digital Signal Processing, Hong Kong, 20-23 August 2014, 449-453.
[7]	Timofte, R., De Smet V. and Van Gool, L. (2014) A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution. In: Proceedings of the Asian Con-ference on Computer Vision (ACCV), Springer, Berlin, 111-126. https://doi.org/10.1007/978-3-319-16817-3_8
[8]	Zeiler, M.D. and Fergus, R. (2014) Visualizing and Under-standing Convolutional Networks. In: Proceedings of the European Conference on Computer Vision (ECCV), Springer, Berlin, 818-833. https://doi.org/10.1007/978-3-319-10590-1_53
[9]	Radford, A., Metz, L. and Chintala, S. (2015) Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. ICLR 2016, Computer Sci-ence.
[10]	Dong, C., Loy, C.C., He, K. and Tang, X. (2015) Image Super Resolution Using Deep Convolutional Net-works. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38, 295-307. https://doi.org/10.1109/TPAMI.2015.2439281
[11]	Dong, C., Chen, C.L. and Tang, X. (2016) Accelerating the Super Resolution Convolutional Neural Network. In: European Conference on Computer Vision, Springer, Cham, 391-407. https://doi.org/10.1007/978-3-319-46475-6_25
[12]	Kim, J., Kwon, L.J. and Mu, L.K. (2016) Accurate Image Super Resolution Using Very Deep Convolutional Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 1646-1654. https://doi.org/10.1109/CVPR.2016.182
[13]	He, K., Zhang, X., Ren, S. and Sun, J. (2016) Deep Residual Learn-ing for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 27-30 June 2016, 770-778. https://doi.org/10.1109/CVPR.2016.90
[14]	Zhang, H., Goodfellow, I., Metaxas, D. and Odena, A. (2018) Self-Attention Generative Adversarial Networks.
[15]	Zhang, Y., Li, K., Li, K., Zhong, B. and Fu, Y. (2019) Residual Non-Local Attention Networks for Image Restoration.
[16]	Yi, P., Wang, Z.Y., Jiang, K., Jiang, J.J. and Ma, J.Y. (2019) Progressive Fusion Video Super Resolution Network via Exploiting Non Local Spatiotemporal Correlations. Proceed-ings of the IEEE International Conference on Computer Vision (ICCV), Seoul, 27 October-2 November 2019, 3106-3115. https://doi.org/10.1109/ICCV.2019.00320
[17]	Luong, M.T., Pham, H. and Manning, C.D. (2015) Ef-fective Approaches to Attention Based Neural Machine Translation. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, September 2015, 1412-1421. https://doi.org/10.18653/v1/D15-1166
[18]	Lai, W.-S., Huang, J.-B., Ahuja, N. and Yang, M.-H. (2017) Deep La-placian Pyramid Networks for Fast and Accurate Super Resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, 21-26 July 2017, 5835-5843. https://doi.org/10.1109/CVPR.2017.618

Full-Text