The alternating direction method of multipliers (ADMM) and its symmetric version are efficient for minimizing two-block separable problems with linear constraints. However, both ADMM and symmetric ADMM have limited versatility across various fields due to the requirement that the gradients of differentiable functions exhibit global Lipschitz continuity, a condition that is typically challenging to satisfy in nonconvex optimization problems. Recently, a novel Bregman ADMM that not only eliminates the need for global Lipschitz continuity of the gradient, but also ensures that Bregman ADMM can be degenerated to the classical ADMM has been proposed for two-block nonconvex optimization problems with linear constraints. Building on this, we propose a symmetric Bregman alternating direction method of multipliers, which can be degenerated into the symmetric ADMM and the Bregman ADMM, and thus further degenerated into the classical ADMM. Moreover, when solving separable nonconvex optimization problems, it does not require the global Lipschitz continuity of the gradients of differentiable functions. Furthermore, we demonstrate that under the Kurdyka-Lojasiewicz inequality and certain conditions, the iterative sequence generated by our algorithm converges to a critical point of the problem. In addition, we examine the convergence rate of the algorithm.
References
[1]
Goldstein, T., O’Donoghue, B., Setzer, S. and Baraniuk, R. (2014) Fast Alternating Direction Optimization Methods. SIAMJournalonImagingSciences, 7, 1588-1623. https://doi.org/10.1137/120896219
[2]
Boyd, S. (2010) Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. FoundationsandTrendsinMachineLearning, 3, 1-122. https://doi.org/10.1561/2200000016
[3]
Yin, W., Osher, S., Goldfarb, D. and Darbon, J. (2008) Bregman Iterative Algorithms for-Minimization with Applications to Compressed Sensing. SIAMJournalonImagingSciences, 1, 143-168. https://doi.org/10.1137/070703983
[4]
Figueiredo, M.A.T. and Bioucas-Dias, J.M. (2010) Restoration of Poissonian Images Using Alternating Direction Optimization. IEEETransactionsonImageProcessing, 19, 3133-3145. https://doi.org/10.1109/tip.2010.2053941
[5]
Goldstein, T., Bresson, X. and Osher, S. (2009) Geometric Applications of the Split Bregman Method: Segmentation and Surface Reconstruction. JournalofScientificComputing, 45, 272-293. https://doi.org/10.1007/s10915-009-9331-z
[6]
Ochs, P., Chen, Y., Brox, T. and Pock, T. (2014) iPiano: Inertial Proximal Algorithm for Nonconvex Optimization. SIAMJournalonImagingSciences, 7, 1388-1419. https://doi.org/10.1137/130942954
[7]
Candes, E.J. and Tao, T. (2006) Near-Optimal Signal Recovery from Random Projections: Universal Encoding Strategies? IEEETransactionsonInformationTheory, 52, 5406-5425. https://doi.org/10.1109/tit.2006.885507
[8]
Beck, A. and Teboulle, M. (2009) Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems. IEEETransactionsonImageProcessing, 18, 2419-2434. https://doi.org/10.1109/tip.2009.2028250
[9]
Dai, W. and Milenkovic, O. (2009) Subspace Pursuit for Compressive Sensing Signal Reconstruction. IEEETransactionsonInformationTheory, 55, 2230-2249. https://doi.org/10.1109/tit.2009.2016006
[10]
Aharon, M., Elad, M. and Bruckstein, A. (2006) K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation. IEEETransactionsonSignalProcessing, 54, 4311-4322. https://doi.org/10.1109/tsp.2006.881199
[11]
Gabay, D. and Mercier, B. (1976) A Dual Algorithm for the Solution of Nonlinear Variational Problems via Finite Element Approximation. Computers&MathematicswithApplications, 2, 17-40. https://doi.org/10.1016/0898-1221(76)90003-1
[12]
Glowinski, R. and Marroco, A. (1975) Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de Dirichlet non linéaires. Revuefrançaised’automatique, informatique, rechercheopérationnelle.Analysenumérique, 9, 41-76. https://doi.org/10.1051/m2an/197509r200411
[13]
Boley, D. (2013) Local Linear Convergence of the Alternating Direction Method of Multipliers on Quadratic or Linear Programs. SIAMJournalonOptimization, 23, 2183-2207. https://doi.org/10.1137/120878951
[14]
Han, D. and Yuan, X. (2013) Local Linear Convergence of the Alternating Direction Method of Multipliers for Quadratic Programs. SIAMJournalonNumericalAnalysis, 51, 3446-3457. https://doi.org/10.1137/120886753
[15]
He, B. and Yuan, X. (2012) On the Convergence Rate of the Douglas-Rachford Alternating Direction Method. SIAMJournalonNumericalAnalysis, 50, 700-709. https://doi.org/10.1137/110836936
[16]
Yang, W.H. and Han, D. (2016) Linear Convergence of the Alternating Direction Method of Multipliers for a Class of Convex Optimization Problems. SIAMJournalonNumericalAnalysis, 54, 625-640. https://doi.org/10.1137/140974237
[17]
Li, G. and Pong, T.K. (2015) Global Convergence of Splitting Methods for Nonconvex Composite Optimization. SIAMJournalonOptimization, 25, 2434-2460. https://doi.org/10.1137/140998135
[18]
Hong, M., Luo, Z. and Razaviyayn, M. (2016) Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problems. SIAMJournalonOptimization, 26, 337-364. https://doi.org/10.1137/140990309
[19]
Guo, K., Han, D.R. and Wu, T.T. (2016) Convergence of Alternating Direction Method for Minimizing Sum of Two Nonconvex Functions with Linear Constraints. InternationalJournalofComputerMathematics, 94, 1653-1669. https://doi.org/10.1080/00207160.2016.1227432
[20]
Wu, Z., Li, M., Wang, D.Z.W. and Han, D. (2017) A Symmetric Alternating Direction Method of Multipliers for Separable Nonconvex Minimization Problems. Asia-PacificJournalofOperationalResearch, 34, Article ID: 1750030. https://doi.org/10.1142/s0217595917500300
[21]
Bauschke, H.H., Bolte, J. and Teboulle, M. (2017) A Descent Lemma Beyond Lipschitz Gradient Continuity: First-Order Methods Revisited and Applications. MathematicsofOperationsResearch, 42, 330-348. https://doi.org/10.1287/moor.2016.0817
[22]
Dragomir, R., d’Aspremont, A. and Bolte, J. (2021) Quartic First-Order Methods for Low-Rank Minimization. JournalofOptimizationTheoryandApplications, 189, 341-363. https://doi.org/10.1007/s10957-021-01820-3
[23]
Nesterov, Y. (2019) Implementable Tensor Methods in Unconstrained Convex Optimization. MathematicalProgramming, 186, 157-183. https://doi.org/10.1007/s10107-019-01449-1
[24]
Tan, L. and Guo, K. (2025) Bregman ADMM: A New Algorithm for Nonconvex Optimization with Linear Constraints. Journal of Nonlinear and Variational Analysis, 9, 176-196. https://doi.org/10.23952/jnva.9.2025.2.02
[25]
Wang, H. and Banerjee, A. (2014) Bregman Alternating Direction Method of Multipliers. arXiv: 1306.3203.
[26]
Beck, A. (2017) First-Order Methods in Optimization. Society for Industrial and Applied Mathematics. https://doi.org/10.1137/1.9781611974997
[27]
Bolte, J., Sabach, S., Teboulle, M. and Vaisbourd, Y. (2018) First Order Methods Beyond Convexity and Lipschitz Gradient Continuity with Applications to Quadratic Inverse Problems. SIAMJournalonOptimization, 28, 2131-2151. https://doi.org/10.1137/17m1138558
[28]
Attouch, H., Bolte, J., Redont, P. and Soubeyran, A. (2010) Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Łojasiewicz Inequality. MathematicsofOperationsResearch, 35, 438-457. https://doi.org/10.1287/moor.1100.0449
[29]
Attouch, H., Bolte, J. and Svaiter, B.F. (2011) Convergence of Descent Methods for Semi-Algebraic and Tame Problems: Proximal Algorithms, Forward-Backward Splitting, and Regularized Gauss–seidel Methods. MathematicalProgramming, 137, 91-129. https://doi.org/10.1007/s10107-011-0484-9
[30]
Bolte, J., Sabach, S. and Teboulle, M. (2013) Proximal Alternating Linearized Minimization for Nonconvex and Nonsmooth Problems. MathematicalProgramming, 146, 459-494. https://doi.org/10.1007/s10107-013-0701-9
Bregman, L.M. (1967) The Relaxation Method of Finding the Common Point of Convex Sets and Its Application to the Solution of Problems in Convex Programming. USSRComputationalMathematicsandMathematicalPhysics, 7, 200-217. https://doi.org/10.1016/0041-5553(67)90040-7
[33]
Attouch, H. and Bolte, J. (2007) On the Convergence of the Proximal Algorithm for Nonsmooth Functions Involving Analytic Features. MathematicalProgramming, 116, 5-16. https://doi.org/10.1007/s10107-007-0133-5