|
|
一种用于求解等式约束鞍点问题的递归神经网络
|
Abstract:
鞍点问题在工程优化、经济决策、信号处理、智能控制等诸多领域中具有广泛的应用背景，本文针对等式约束下的鞍点问题，提出了一种新型递归神经网络模型。通过构建相应的动力学方程，分析了该神经网络在Lyapunov意义下的稳定性，并证明了其平衡点与原问题最优解的等价性。
Saddle point problems have broad applications in various fields such as engineering optimization, economic decision-making, signal processing, and intelligent control. This paper proposes a novel recurrent neural network model for solving equality-constrained saddle point problems. By constructing the corresponding dynamic equations, the stability of the neural network in the Lyapunov sense is analyzed, and the equivalence between its equilibrium point and the optimal solution of the original problem is proven.
| [1] | Ben-Tal, A., Ghaoui, L.E. and Nemirovski A. (2009) Robust Optimization. Princeton University Press. |
| [2] | Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y. (2014) Generative Adversarial Nets. Neural Information Processing Systems, 27, 2672-2680. |
| [3] | Boyd, S. (2011) Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Foundations and Trends® in Machine Learning, 3, 1-122. https://doi.org/10.1561/2200000016 |
| [4] | Zafar, M.B., Valera, I., Gomez Rodriguez, M. and Gummadi, K.P. (2017) Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment. Proceedings of the 26th International Conference on World Wide Web, Perth, 3-7 April 2017, 1171-1180. https://doi.org/10.1145/3038912.3052660 |
| [5] | Madry, A., Makelov, A., Schmidt, L., Tsipras, D. and Vladu, A. (2018) Towards Deep Learning Models Resistant to Adversarial Attacks. International Conference on Learning Representations, Vancouver. |
| [6] | Arrow, K.J. and Debreu, G. (1954) Existence of an Equilibrium for a Competitive Economy. Econometrica, 22, 265-290. https://doi.org/10.2307/1907353 |
| [7] | Arrow, K.J., Hurwicz, L. and Uzawa, H. (1958) Studies in Linear and Non-Linear Programming. Stanford University Press. |
| [8] | Daskalakis, C., Ilyas, A., Syrgkanis, V. and Zeng, H.Y. (2018) Training GANs with Optimism. International Conference on Learning Representations, Vancouver. |
| [9] | Cheng, Z., Ma, J., Wang, W., Zhu, Z., de Silva, C.W. and Lee, T.H. (2024) Alternating Direction Method of Multipliers-Based Parallel Optimization for Multi-Agent Collision-Free Model Predictive Control. IEEE Transactions on Artificial Intelligence, 5, 4176-4191. https://doi.org/10.1109/tai.2024.3364127 |
| [10] | Gao, X.-B., Liao, L.-Z. and Xue, W. (2004) A Neural Network for a Class of Convex Quadratic Minimax Problems with Constraints. IEEE Transactions on Neural Networks, 15, 622-628. https://doi.org/10.1109/tnn.2004.824405 |
| [11] | Gao, X. and Liao, L. (2006) A Novel Neural Network for a Class of Convex Quadratic Minimax Problems. Neural Computation, 18, 1818-1846. https://doi.org/10.1162/neco.2006.18.8.1818 |
| [12] | Ghasabi-Oskoei, H., Malek, A. and Ahmadi, A. (2007) Novel Artificial Neural Network with Simulation Aspects for Solving Linear and Quadratic Programming Problems. Computers & Mathematics with Applications, 53, 1439-1454. https://doi.org/10.1016/j.camwa.2006.10.022 |
| [13] | Gao, X.B. and Li, C.P. (2017) A New Neural Network for Convex Quadratic Minimax Problems with Box and Equality Constraints. Computers & Chemical Engineering, 104, 1-10. https://doi.org/10.1016/j.compchemeng.2017.03.022 |
| [14] | Wang, Z.D., Liu, Y.R., Li, M.Z. and Liu, X.H. (2006) Stability Analysis for Stochastic Cohen-Grossberg Neural Networks with Mixed Time Delays. IEEE Transactions on Neural Networks, 17, 814-820. https://doi.org/10.1109/tnn.2006.872355 |
| [15] | 喻昕, 陈昭蓉. 一类非光滑非凸优化问题的神经网络方法[J]. 计算机应用研究, 2019, 36(9): 2575-2578. |
| [16] | 喻昕, 林植良. 解决一类非光滑伪凸优化问题的新型神经网络[J]. 计算机科学, 2022, 49(5): 227-234. |
| [17] | Hopfield, J.J. (1982) Neural Networks and Physical Systems with Emergent Collective Computational Abilities. Proceedings of the National Academy of Sciences, 79, 2554-2558. https://doi.org/10.1073/pnas.79.8.2554 |
| [18] | 王雨嫣, 廖柏林, 彭晨, 等. 递归神经网络研究综述[J]. 吉首大学学报(自然科学版), 2021, 42(1): 41-48. |
| [19] | Xia, Y. and Wang, J. (2005) A Recurrent Neural Network for Solving Nonlinear Convex Programs Subject to Linear Constraints. IEEE Transactions on Neural Networks, 16, 379-386. https://doi.org/10.1109/tnn.2004.841779 |
| [20] | Liu, Q.S. and Wang, J. (2011) A One-Layer Recurrent Neural Network for Constrained Nonsmooth Optimization. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 41, 1323-1333. https://doi.org/10.1109/tsmcb.2011.2140395 |
| [21] | Tank, D. and Hopfield, J. (1986) Simple ‘Neural’ Optimization Networks: An A/D Converter, Signal Decision Circuit, and a Linear Programming Circuit. IEEE Transactions on Circuits and Systems, 33, 533-541. https://doi.org/10.1109/tcs.1986.1085953 |
| [22] | Kennedy, M.P. and Chua, L.O. (1988) Neural Networks for Nonlinear Programming. IEEE Transactions on Circuits and Systems, 35, 554-562. https://doi.org/10.1109/31.1783 |
| [23] | Zhang, S. and Constantinides, A.G. (1992) Lagrange Programming Neural Networks. IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, 39, 441-452. https://doi.org/10.1109/82.160169 |
| [24] | Xia, Y. and Wang, J. (2004) A General Projection Neural Network for Solving Monotone Variational Inequalities and Related Optimization Problems. IEEE Transactions on Neural Networks, 15, 318-328. https://doi.org/10.1109/tnn.2004.824252 |
| [25] | Qin, S., Bian, W. and Xue, X. (2013) A New One-Layer Recurrent Neural Network for Nonsmooth Pseudoconvex Optimization. Neurocomputing, 120, 655-662. https://doi.org/10.1016/j.neucom.2013.01.025 |
| [26] | Gao, X.B. (2003) Exponential Stability of Globally Projected Dynamic Systems. IEEE Transactions on Neural Networks, 14, 426-431. https://doi.org/10.1109/tnn.2003.809409 |
| [27] | Murray, R.M., Li, Z. and Sastry, S.S. (2017) A Mathematical Introduction to Robotic Manipulation. 1st Edition, CRC Press. https://doi.org/10.1201/9781315136370-1 |
| [28] | 袁亚湘, 孙文瑜. 最优化理论与方法[M]. 北京: 科学出版社, 1997. |
| [29] | 张从军. 集值分析与经济应用[M]. 北京: 科学出版社, 2004. |
| [30] | 唐昊. 不等式约束鞍点问题的罚函数方法[D]: [硕士学位论文]. 重庆: 重庆交通大学, 2024. |
