Training Pi-sigma neural network by stochastic simple point online gradient algorithm with Lagrange multiplier method
Pi-sigma神经网络的乘子法随机单点在线梯度算法*

When the on-line gradient algotithm is used for training Pi-sigma neural netrork, there is a problem that the chosen weights may be very small, resulting in a very slow convergence. The shortcoming can be overcome by the penalty method, but there are the difficulties in numerical solution, caused by the facts that the penalty factor must approach infinity and the absolute value of penalty term is nondifferentiable. Based on Lagrange multipler algorithm, this paper proposed a stochastic simple point on-line gradient algorithm to overcome the deficiencies of small weights and penalty function. Using the optimized theory method, transformed the restrained question into the non-constraint question. Proved the convergence rate and stability of the algorithm. The simulated experimental results indicate that the algorithm is efficient.