|
|
算子约束的阶段递进超分辨率优化
|
Abstract:
多阶段深度展开超分辨率网络通过交替求解数据一致性子问题与先验子问题,将显式退化模型与学习型先验相结合。然而,现有训练方法通常对不同展开阶段采用同质化监督,并在教师指导训练中默认教师信息在不同样本和阶段上具有相同可信度。针对上述问题,本文提出算子约束的阶段递进优化(OSPO)。首先,在先验更新后的完整阶段输出上施加与确定性退化过程一致的干净退化算子约束,以限制先验更新对可观测成分的破坏;其次,利用无参数阶段进度协调算子域与高频域监督,使训练重点随展开过程由退化一致性逐步转向细节恢复。在教师指导实例中,进一步根据教师残差消息与真实残差消息在双域中的一致程度,自适应调节教师紧凑表征监督,形成算子约束的阶段递进可靠性优化(OSPR)。在Set5(×4)及12种非双三次模糊核构成的受控测试协议下,OSPO相对USRNet基线将RGB-PSNR提高0.0364 dB;在紧凑残差先验模块实例上,完整OSPR相对相应基线提高0.0429 dB。所提方法仅调整训练目标,不引入额外推理模块,也无需在部署阶段调用教师模型。
Multi-stage deep unfolding networks combine explicit degradation models with learned priors by alternately solving data-consistency and prior subproblems. However, existing training schemes typically apply homogeneous supervision across unfolding stages and, in teacher-guided training, implicitly assume equal teacher reliability across samples and stages. To address these limitations, we propose Operator-Constrained Stage-Progressive Optimization (OSPO). First, a clean degradation-operator constraint consistent with the deterministic data-generation process is imposed on the complete stage output after the prior update, limiting changes to observable components. Second, a parameter-free stage schedule coordinates operator-domain and high-frequency supervision, progressively shifting the training emphasis from degradation consistency to detail recovery. For the teacher-guided instance, the teacher compact-representation loss is further modulated according to the dual-domain agreement between teacher and ground-truth residual messages, yielding Operator-Constrained Stage-Progressive Reliability Optimization (OSPR). Under a controlled protocol on Set5 at ×4 with 12 non-bicubic blur kernels, OSPO improves RGB-PSNR by 0.0364 dB over the USRNet baseline, while full OSPR improves it by 0.0429 dB over the corresponding compact residual-prior baseline. The proposed objectives modify only the training loss and require neither additional inference modules nor teacher calls at deployment.
| [1] | Dong, C., Loy, C.C., He, K. and Tang, X. (2016) Image Super-Resolution Using Deep Convolutional Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38, 295-307. https://doi.org/10.1109/tpami.2015.2439281 |
| [2] | Lim, B., Son, S., Kim, H., Nah, S. and Lee, K.M. (2017) Enhanced Deep Residual Networks for Single Image Super-Resolution. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, 21-26 July 2017, 1132-1140. https://doi.org/10.1109/cvprw.2017.151 |
| [3] | Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L. and Timofte, R. (2021) SwinIR: Image Restoration Using Swin Transformer. 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Montreal, 11-17 October 2021, 1833-1844. https://doi.org/10.1109/iccvw54120.2021.00210 |
| [4] | Zhang, K., Zuo, W. and Zhang, L. (2018) Learning a Single Convolutional Super-Resolution Network for Multiple Degradations. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-23 June 2018, 3262-3271. https://doi.org/10.1109/cvpr.2018.00344 |
| [5] | Zhang, K., Zuo, W. and Zhang, L. (2019) Deep Plug-and-Play Super-Resolution for Arbitrary Blur Kernels. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 1671-1681. https://doi.org/10.1109/cvpr.2019.00177 |
| [6] | Gu, J., Lu, H., Zuo, W. and Dong, C. (2019) Blind Super-Resolution with Iterative Kernel Correction. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 1604-1613. https://doi.org/10.1109/cvpr.2019.00170 |
| [7] | Luo, Z., Huang, Y., Li, S., Wang, L. and Tan, T. (2020) Unfolding the Alternating Optimization for Blind Super Resolution. Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, 6-12 December 2020, 5632-5643. |
| [8] | Zhang, K., Van Gool, L. and Timofte, R. (2020) Deep Unfolding Network for Image Super-Resolution. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 13-19 June 2020, 3217-3226. https://doi.org/10.1109/cvpr42600.2020.00328 |
| [9] | Venkatakrishnan, S.V., Bouman, C.A. and Wohlberg, B. (2013) Plug-and-Play Priors for Model Based Reconstruction. 2013 IEEE Global Conference on Signal and Information Processing, Austin, 3-5 December 2013, 945-948. https://doi.org/10.1109/globalsip.2013.6737048 |
| [10] | Romano, Y., Elad, M. and Milanfar, P. (2017) The Little Engine That Could: Regularization by Denoising (Red). SIAM Journal on Imaging Sciences, 10, 1804-1844. https://doi.org/10.1137/16m1102884 |
| [11] | Zhang, K., Zuo, W., Gu, S. and Zhang, L. (2017). Learning Deep CNN Denoiser Prior for Image Restoration. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 21-26 July 2017, 3929-3938. https://doi.org/10.1109/cvpr.2017.300 |
| [12] | Zhang, K., Li, Y., Zuo, W., Zhang, L., Van Gool, L. and Timofte, R. (2022) Plug-and-Play Image Restoration with Deep Denoiser Prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44, 6360-6376. https://doi.org/10.1109/tpami.2021.3088914 |
| [13] | Gregor, K. and LeCun, Y. (2010) Learning Fast Approximations of Sparse Coding. Proceedings of the 27th International Conference on Machine Learning, Haifa, 21-24 June 2010, 399-406. |
| [14] | Monga, V., Li, Y. and Eldar, Y.C. (2021) Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing. IEEE Signal Processing Magazine, 38, 18-44. https://doi.org/10.1109/msp.2020.3016905 |
| [15] | Lee, C.Y., Xie, S., Gallagher, P., Zhang, Z. and Tu, Z. (2015) Deeply-Supervised Nets. Proceedings of the 18th International Conference on Artificial Intelligence and Statistics, PMLR, Vol. 38, 562-570. |
| [16] | Jiang, L., Dai, B., Wu, W. and Loy, C.C. (2021) Focal Frequency Loss for Image Reconstruction and Synthesis. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, 10-17 October 2021, 13919-13929. https://doi.org/10.1109/iccv48922.2021.01366 |
| [17] | Fuoli, D., Van Gool, L. and Timofte, R. (2021) Fourier Space Losses for Efficient Perceptual Image Super-Resolution. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, 10-17 October 2021, 2340-2349. https://doi.org/10.1109/iccv48922.2021.00236 |
| [18] | Ho, J., Jain, A. and Abbeel, P. (2020) Denoising Diffusion Probabilistic Models. Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, 6-12 December 2020, 6840-6851. |
| [19] | Kawar, B., Elad, M., Ermon, S. and Song, J. (2022) Denoising Diffusion Restoration Models. Advances in Neural Information Processing Systems 35, New Orleans, 28 November-9 December 2022, 23593-23606. https://doi.org/10.52202/068431-1714 |
| [20] | Chung, H., Kim, J., McCann, M.T., Klasky, M.L. and Ye, J.C. (2023) Diffusion Posterior Sampling for General Noisy Inverse Problems. International Conference on Learning Representations, Kigali, 1-5 May 2023, 30. https://openreview.net/forum?id=OnD9zGAGT0k |
| [21] | Wang, Y., Yu, J. and Zhang, J. (2023) Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model. International Conference on Learning Representations, Kigali, 1-5 May 2023, 31. https://openreview.net/forum?id=mRieQgMtNTQ |
| [22] | Hinton, G., Vinyals, O. and Dean, J. (2015) Distilling the Knowledge in a Neural Network. https://arxiv.org/abs/1503.02531 |
| [23] | Shen, Y., Zhang, Z., Sabuncu, M.R. and Sun, L. (2021) Real-Time Uncertainty Estimation in Computer Vision via Uncertainty-Aware Distribution Distillation. 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa, 3-8 January 2021, 707-716. https://doi.org/10.1109/wacv48630.2021.00075 |
| [24] | Lai, W.S., Huang, J.B., Ahuja, N. and Yang, M.H. (2017) Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 21-26 July 2017, 5835-5843. https://doi.org/10.1109/cvpr.2017.618 |
| [25] | Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M., et al. (2021) Multi-Stage Progressive Image Restoration. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, 20-25 June 2021, 14816-14826. https://doi.org/10.1109/cvpr46437.2021.01458 |
| [26] | Chen, Z., Badrinarayanan, V., Lee, C.Y. and Rabinovich, A. (2018) GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks. Proceedings of the 35th International Conference on Machine Learning, PMLR, Vol. 80, 794-803. |
| [27] | Zhang, H., Chen, D. and Wang, C. (2022) Confidence-Aware Multi-Teacher Knowledge Distillation. 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore, 23-27 May 2022, 4498-4502. https://doi.org/10.1109/icassp43922.2022.9747534 |
| [28] | Hu, C., Wu, H., Li, X., Ma, C., Chen, X., Yan, J., Wang, B. and Liu, X. (2024) Less or More from Teacher: Exploiting Trilateral Geometry for Knowledge Distillation. International Conference on Learning Representations, Vienna, 7 May 2024, 16. https://openreview.net/forum?id=OZitfSXpdT |