全部 标题 作者
关键词 摘要

OALib Journal期刊
ISSN: 2333-9721
费用:99美元

查看量下载量

相关文章

更多...

A Symmetric Bregman Alternating Direction Method of Multipliers for Separable Nonconvex Minimization Problem

DOI: 10.4236/jamp.2025.133047, PP. 889-913

Keywords: Symmetric ADMM, Bregman ADMM, Nonconvex Optimization, Separable, Kurdyka-Lojasiewicz Inequality

Full-Text   Cite this paper   Add to My Lib

Abstract:

The alternating direction method of multipliers (ADMM) and its symmetric version are efficient for minimizing two-block separable problems with linear constraints. However, both ADMM and symmetric ADMM have limited versatility across various fields due to the requirement that the gradients of differentiable functions exhibit global Lipschitz continuity, a condition that is typically challenging to satisfy in nonconvex optimization problems. Recently, a novel Bregman ADMM that not only eliminates the need for global Lipschitz continuity of the gradient, but also ensures that Bregman ADMM can be degenerated to the classical ADMM has been proposed for two-block nonconvex optimization problems with linear constraints. Building on this, we propose a symmetric Bregman alternating direction method of multipliers, which can be degenerated into the symmetric ADMM and the Bregman ADMM, and thus further degenerated into the classical ADMM. Moreover, when solving separable nonconvex optimization problems, it does not require the global Lipschitz continuity of the gradients of differentiable functions. Furthermore, we demonstrate that under the Kurdyka-Lojasiewicz inequality and certain conditions, the iterative sequence generated by our algorithm converges to a critical point of the problem. In addition, we examine the convergence rate of the algorithm.

References

[1]  Goldstein, T., O’Donoghue, B., Setzer, S. and Baraniuk, R. (2014) Fast Alternating Direction Optimization Methods. SIAM Journal on Imaging Sciences, 7, 1588-1623.
https://doi.org/10.1137/120896219
[2]  Boyd, S. (2010) Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Foundations and Trends in Machine Learning, 3, 1-122.
https://doi.org/10.1561/2200000016
[3]  Yin, W., Osher, S., Goldfarb, D. and Darbon, J. (2008) Bregman Iterative Algorithms for-Minimization with Applications to Compressed Sensing. SIAM Journal on Imaging Sciences, 1, 143-168.
https://doi.org/10.1137/070703983
[4]  Figueiredo, M.A.T. and Bioucas-Dias, J.M. (2010) Restoration of Poissonian Images Using Alternating Direction Optimization. IEEE Transactions on Image Processing, 19, 3133-3145.
https://doi.org/10.1109/tip.2010.2053941
[5]  Goldstein, T., Bresson, X. and Osher, S. (2009) Geometric Applications of the Split Bregman Method: Segmentation and Surface Reconstruction. Journal of Scientific Computing, 45, 272-293.
https://doi.org/10.1007/s10915-009-9331-z
[6]  Ochs, P., Chen, Y., Brox, T. and Pock, T. (2014) iPiano: Inertial Proximal Algorithm for Nonconvex Optimization. SIAM Journal on Imaging Sciences, 7, 1388-1419.
https://doi.org/10.1137/130942954
[7]  Candes, E.J. and Tao, T. (2006) Near-Optimal Signal Recovery from Random Projections: Universal Encoding Strategies? IEEE Transactions on Information Theory, 52, 5406-5425.
https://doi.org/10.1109/tit.2006.885507
[8]  Beck, A. and Teboulle, M. (2009) Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems. IEEE Transactions on Image Processing, 18, 2419-2434.
https://doi.org/10.1109/tip.2009.2028250
[9]  Dai, W. and Milenkovic, O. (2009) Subspace Pursuit for Compressive Sensing Signal Reconstruction. IEEE Transactions on Information Theory, 55, 2230-2249.
https://doi.org/10.1109/tit.2009.2016006
[10]  Aharon, M., Elad, M. and Bruckstein, A. (2006) K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation. IEEE Transactions on Signal Processing, 54, 4311-4322.
https://doi.org/10.1109/tsp.2006.881199
[11]  Gabay, D. and Mercier, B. (1976) A Dual Algorithm for the Solution of Nonlinear Variational Problems via Finite Element Approximation. Computers & Mathematics with Applications, 2, 17-40.
https://doi.org/10.1016/0898-1221(76)90003-1
[12]  Glowinski, R. and Marroco, A. (1975) Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de Dirichlet non linéaires. Revue française dautomatique, informatique, recherche opérationnelle. Analyse numérique, 9, 41-76.
https://doi.org/10.1051/m2an/197509r200411
[13]  Boley, D. (2013) Local Linear Convergence of the Alternating Direction Method of Multipliers on Quadratic or Linear Programs. SIAM Journal on Optimization, 23, 2183-2207.
https://doi.org/10.1137/120878951
[14]  Han, D. and Yuan, X. (2013) Local Linear Convergence of the Alternating Direction Method of Multipliers for Quadratic Programs. SIAM Journal on Numerical Analysis, 51, 3446-3457.
https://doi.org/10.1137/120886753
[15]  He, B. and Yuan, X. (2012) On the Convergence Rate of the Douglas-Rachford Alternating Direction Method. SIAM Journal on Numerical Analysis, 50, 700-709.
https://doi.org/10.1137/110836936
[16]  Yang, W.H. and Han, D. (2016) Linear Convergence of the Alternating Direction Method of Multipliers for a Class of Convex Optimization Problems. SIAM Journal on Numerical Analysis, 54, 625-640.
https://doi.org/10.1137/140974237
[17]  Li, G. and Pong, T.K. (2015) Global Convergence of Splitting Methods for Nonconvex Composite Optimization. SIAM Journal on Optimization, 25, 2434-2460.
https://doi.org/10.1137/140998135
[18]  Hong, M., Luo, Z. and Razaviyayn, M. (2016) Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problems. SIAM Journal on Optimization, 26, 337-364.
https://doi.org/10.1137/140990309
[19]  Guo, K., Han, D.R. and Wu, T.T. (2016) Convergence of Alternating Direction Method for Minimizing Sum of Two Nonconvex Functions with Linear Constraints. International Journal of Computer Mathematics, 94, 1653-1669.
https://doi.org/10.1080/00207160.2016.1227432
[20]  Wu, Z., Li, M., Wang, D.Z.W. and Han, D. (2017) A Symmetric Alternating Direction Method of Multipliers for Separable Nonconvex Minimization Problems. Asia-Pacific Journal of Operational Research, 34, Article ID: 1750030.
https://doi.org/10.1142/s0217595917500300
[21]  Bauschke, H.H., Bolte, J. and Teboulle, M. (2017) A Descent Lemma Beyond Lipschitz Gradient Continuity: First-Order Methods Revisited and Applications. Mathematics of Operations Research, 42, 330-348.
https://doi.org/10.1287/moor.2016.0817
[22]  Dragomir, R., d’Aspremont, A. and Bolte, J. (2021) Quartic First-Order Methods for Low-Rank Minimization. Journal of Optimization Theory and Applications, 189, 341-363.
https://doi.org/10.1007/s10957-021-01820-3
[23]  Nesterov, Y. (2019) Implementable Tensor Methods in Unconstrained Convex Optimization. Mathematical Programming, 186, 157-183.
https://doi.org/10.1007/s10107-019-01449-1
[24]  Tan, L. and Guo, K. (2025) Bregman ADMM: A New Algorithm for Nonconvex Optimization with Linear Constraints. Journal of Nonlinear and Variational Analysis, 9, 176-196.
https://doi.org/10.23952/jnva.9.2025.2.02
[25]  Wang, H. and Banerjee, A. (2014) Bregman Alternating Direction Method of Multipliers. arXiv: 1306.3203.
[26]  Beck, A. (2017) First-Order Methods in Optimization. Society for Industrial and Applied Mathematics.
https://doi.org/10.1137/1.9781611974997
[27]  Bolte, J., Sabach, S., Teboulle, M. and Vaisbourd, Y. (2018) First Order Methods Beyond Convexity and Lipschitz Gradient Continuity with Applications to Quadratic Inverse Problems. SIAM Journal on Optimization, 28, 2131-2151.
https://doi.org/10.1137/17m1138558
[28]  Attouch, H., Bolte, J., Redont, P. and Soubeyran, A. (2010) Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Łojasiewicz Inequality. Mathematics of Operations Research, 35, 438-457.
https://doi.org/10.1287/moor.1100.0449
[29]  Attouch, H., Bolte, J. and Svaiter, B.F. (2011) Convergence of Descent Methods for Semi-Algebraic and Tame Problems: Proximal Algorithms, Forward-Backward Splitting, and Regularized Gauss–seidel Methods. Mathematical Programming, 137, 91-129.
https://doi.org/10.1007/s10107-011-0484-9
[30]  Bolte, J., Sabach, S. and Teboulle, M. (2013) Proximal Alternating Linearized Minimization for Nonconvex and Nonsmooth Problems. Mathematical Programming, 146, 459-494.
https://doi.org/10.1007/s10107-013-0701-9
[31]  Bauschke, H.H., Borwein, J.M. and Combettes, P.L. (2003) Bregman Monotone Optimization Algorithms. SIAM Journal on Control and Optimization, 42, 596-636.
https://doi.org/10.1137/s0363012902407120
[32]  Bregman, L.M. (1967) The Relaxation Method of Finding the Common Point of Convex Sets and Its Application to the Solution of Problems in Convex Programming. USSR Computational Mathematics and Mathematical Physics, 7, 200-217.
https://doi.org/10.1016/0041-5553(67)90040-7
[33]  Attouch, H. and Bolte, J. (2007) On the Convergence of the Proximal Algorithm for Nonsmooth Functions Involving Analytic Features. Mathematical Programming, 116, 5-16.
https://doi.org/10.1007/s10107-007-0133-5

Full-Text

Contact Us

service@oalib.com

QQ:3279437679

WhatsApp +8615387084133