|
|
求解非凸优化问题的带惯性项Majorized Bregman交替方向乘子法
|
Abstract:
对非凸两分块优化问题，提出一种带惯性的Majorized Bregman交替方向乘子法。该算法在迭代中结合了目标函数的极大化线性技术和Bregman距离来简化子问题的求解，同时通过引入惯性项加快收敛效果。在适当条件下证明了算法的收敛性质。初步数值实验结果表明该算法是有效的。
A Majorized Bregman alternating direction method of multipliers (Bregman-ADMM) with inertial terms is proposed for nonconvex two-block optimization problems. The algorithm combines a linearization technique for the objective function and the Bregman distance in each iteration to simplify subproblem solutions, while accelerating the convergence rate through inertial terms. The convergence of the algorithm is established under appropriate conditions. Preliminary numerical experiments demonstrate the effectiveness of the proposed algorithm.
| [1] | Schizas, I.D., Ribeiro, A. and Giannakis, G.B. (2007) Consensus in AD Hoc WSNs with Noisy Links—Part I: Distributed Estimation of Deterministic Signals. IEEE Transactions on Signal Processing, 56, 350-364. https://doi.org/10.1109/TSP.2007.906734 |
| [2] | Afonso, M.V., Bioucas-Dias, J.M. and Figueiredo, M.A.T. (2010) An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems. IEEE Transactions on Image Processing, 20, 681-695. https://doi.org/10.1109/TIP.2010.2076294 |
| [3] | Dhar, S., Yi, C., Ramakrishnan, N. and Shah, M. (2015) ADMM Based Scalable Machine Learning on Spark. 2015 IEEE International Conference on Big Data (Big Data), Santa Clara, 29 October 2015-1 November 2015, 1174-1182. https://doi.org/10.1109/bigdata.2015.7363871 |
| [4] | Yang, Q., Chen, G. and Wang, T. (2020) ADMM-Based Distributed Algorithm for Economic Dispatch in Power Systems with Both Packet Drops and Communication Delays. IEEE/CAA Journal of Automatica Sinica, 7, 842-852. https://doi.org/10.1109/jas.2020.1003156 |
| [5] | 邓钊, 晁绵涛, 简金宝. 非凸两分块问题乘子交替方向法的收敛性分析[J]. 广西科学, 2016, 23(5): 422-427. |
| [6] | Wang, Y., Yin, W. and Zeng, J. (2019) Global Convergence of ADMM in Nonconvex Nonsmooth Optimization. Journal of Scientific Computing, 78, 29-63. https://doi.org/10.1007/s10915-018-0757-z |
| [7] | Wang, F., Cao, W. and Xu, Z. (2018) Convergence of Multi-Block Bregman ADMM for Nonconvex Composite Problems. Science China Information Sciences, 61, Article No. 12201. https://doi.org/10.1007/s11432-017-9367-6 |
| [8] | 陈建华, 彭建文, 罗洪林. 求解非凸两分块优化问题的Majorized Bregman交替方向乘子法[J]. 重庆师范大学学报(自然科学版), 2023, 40(5): 1-10. |
| [9] | Polyak, B.T. (1964) Some Methods of Speeding up the Convergence of Iteration Methods. USSR Computational Mathematics and Mathematical Physics, 4, 1-17. https://doi.org/10.1016/0041-5553(64)90137-5 |
| [10] | Ochs, P., Chen, Y., Brox, T. and Pock, T. (2014) iPiano: Inertial Proximal Algorithm for Nonconvex Optimization. SIAM Journal on Imaging Sciences, 7, 1388-1419. https://doi.org/10.1137/130942954 |
| [11] | Boţ, R.I., Csetnek, E.R. and László, S.C. (2016) An Inertial Forward-Backward Algorithm for the Minimization of the Sum of Two Nonconvex Functions. EURO Journal on Computational Optimization, 4, 3-25. https://doi.org/10.1007/s13675-015-0045-8 |
| [12] | Xu, J. and Chao, M. (2022) An Inertial Bregman Generalized Alternating Direction Method of Multipliers for Nonconvex Optimization. Journal of Applied Mathematics and Computing, 68, 1-27. https://doi.org/10.1007/s12190-021-01590-1 |
| [13] | Chao, M.T., Zhang, Y. and Jian, J.B. (2020) An Inertial Proximal Alternating Direction Method of Multipliers for Nonconvex Optimization. International Journal of Computer Mathematics, 98, 1199-1217. https://doi.org/10.1080/00207160.2020.1812585 |
| [14] | 刘浩洋, 户将, 李勇锋, 等. 最优化: 建模、算法与理论[M]. 北京: 高等教育出版社, 2020. |
| [15] | Wang, F., Xu, Z. and Xu, H.K. (2014) Convergence of Bregman Alternating Direction Method with Multipliers for Nonconvex Composite Problems. arXiv:1410.8625. |
| [16] | Gonçalves, M.L.N, Melo, J.G. and Monteiro, R.D.C. (2017) Convergence Rate Bounds for a Proximal ADMM with Over-Relaxation Step Size Parameter for Solving Nonconvex Linearly Constrained Problems. arXiv:1702.01850. |
| [17] | Boyd, S. (2010) Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Foundations and Trends® in Machine Learning, 3, 1-122. https://doi.org/10.1561/2200000016 |
| [18] | 李晶晶. 基于梯度的三种优化方法及比较[J]. 统计学与应用, 2024, 13(1): 21-29. |
| [19] | Chao, M., Deng, Z. and Jian, J. (2020) Convergence of Linear Bregman ADMM for Nonconvex and Nonsmooth Problems with Nonseparable Structure. Complexity, 2020, Article 6237942. https://doi.org/10.1155/2020/6237942 |
