U-D Factorization-Based Nonlinear Programming Method and its Application in Neural Network Training
非线性规划U-D分解方法及其在神经网络训练中的应用

To solve convergence rate problems of often used DFP and BFCS methods, the stable construction of inverse Hassian matrix are presented. To get high numerical stability and computational efficiency, U-D factorization-based DFP and BFGS algorithms are developed. In the new methods the positive definiteness of the inverse matrix H is ensured and both the stability and convergence of the algorithm is improved. By using rank-one U-D factorization updates of H, the numerical accuracy and efficiency are increased. Operational counts for computing H show that the efficiency of the new algorithm is increased by 20% and the storages of matrix H is reduced by 50%. Results of several numerical example show that the optimization problems can be solved by using the programming methods presented in this paper and accurate results may be obtained.