This paper investigates the solution of Ordinary Differential Equations (ODEs) with initial conditions using Regression Based Algorithm (RBA) and compares the results with arbitrary- and regression-based initial weights for different numbers of nodes in hidden layer. Here, we have used feed forward neural network and error back propagation method for minimizing the error function and for the modification of the parameters (weights and biases). Initial weights are taken as combination of random as well as by the proposed regression based model. We present the method for solving a variety of problems and the results are compared. Here, the number of nodes in hidden layer has been fixed according to the degree of polynomial in the regression fitting. For this, the input and output data are fitted first with various degree polynomials using regression analysis and the coefficients involved are taken as initial weights to start with the neural training. Fixing of the hidden nodes depends upon the degree of the polynomial. For the example problems, the analytical results have been compared with neural results with arbitrary and regression based weights with four, five, and six nodes in hidden layer and are found to be in good agreement. 1. Introduction Differential equations play vital role in various fields of engineering and science. The exact solution of differential equations may not be always possible [1]. So various types of well known numerical methods such as Euler, Runge-kutta, Predictor-Corrector, finite element, and finite difference methods, are used for solving these equations. Although these numerical methods provide good approximations to the solution, but these may be challenging for higher dimension problems. In recent years, many researchers tried to find new methods for solving differential equations. As such here Artificial Neural Network (ANN) based models are used to solve ordinary differential equations with initial conditions. Lee and Kang [2] first introduced a method to solve first order differential equation using Hopfield neural network models. Then, another approach by Meade and Fernandez [3, 4] has been proposed for both linear and nonlinear differential equations using -splines and feed forward neural network. Artificial neural networks based on Broyden-Fletcher-Goldfarb-Shanno (BFGS) optimization technique for solving ordinary and partial differential equations have been excellently presented by Lagaris et al. [5]. Also Lagaris et al. [6] investigated neural network methods for boundary value problems with irregular boundaries.
References
[1]
H. J. Ricardo, A Modern Introduction to Differential Equations, Elsevier, 2nd edition, 2009.
[2]
H. Lee and I. S. Kang, “Neural algorithm for solving differential equations,” Journal of Computational Physics, vol. 91, no. 1, pp. 110–131, 1990.
[3]
A. J. Meade Jr. and A. A. Fernandez, “The numerical solution of linear ordinary differential equations by feedforward neural networks,” Mathematical and Computer Modelling, vol. 19, no. 12, pp. 1–25, 1994.
[4]
A. J. Meade Jr. and A. A. Fernandez, “Solution of nonlinear ordinary differential equations by feedforward neural networks,” Mathematical and Computer Modelling, vol. 20, no. 9, pp. 19–44, 1994.
[5]
I. E. Lagaris, A. Likas, and D. I. Fotiadis, “Artificial neural networks for solving ordinary and partial differential equations,” IEEE Transactions on Neural Networks, vol. 9, no. 5, pp. 987–1000, 1998.
[6]
I. E. Lagaris, A. C. Likas, and D. G. Papageorgiou, “Neural-network methods for boundary value problems with irregular boundaries,” IEEE Transactions on Neural Networks, vol. 11, no. 5, pp. 1041–1049, 2000.
[7]
D. R. Parisi, M. C. Mariani, and M. A. Laborde, “Solving differential equations with unsupervised neural networks,” Chemical Engineering and Processing, vol. 42, no. 8-9, pp. 715–721, 2003.
[8]
A. Malek and R. Shekari Beidokhti, “Numerical solution for high order differential equations using a hybrid neural network-Optimization method,” Applied Mathematics and Computation, vol. 183, no. 1, pp. 260–271, 2006.
[9]
B. Choi and J.-H. Lee, “Comparison of generalization ability on solving differential equations using backpropagation and reformulated radial basis function networks,” Neurocomputing, vol. 73, no. 1–3, pp. 115–118, 2009.
[10]
H. S. Yazdi, M. Pakdaman, and H. Modaghegh, “Unsupervised kernel least mean square algorithm for solving ordinary differential equations,” Neurocomputing, vol. 74, no. 12-13, pp. 2062–2071, 2011.
[11]
N. Selvaraju and J. Abdul Samant, “Solution of matrix Riccati differential equation for nonlinear singular system using neural networks,” International Journal of Computer Applications, vol. 29, pp. 48–54, 2010.
[12]
S. He, K. Reif, and R. Unbehauen, “Multilayer neural networks for solving a class of partial differential equations,” Neural Networks, vol. 13, no. 3, pp. 385–396, 2000.
[13]
M. Kumar and N. Yadav, “Multilayer perceptrons and radial basis function neural network methods for the solution of differential equations: a survey,” Computers and Mathematics with Applications, vol. 62, no. 10, pp. 3796–3811, 2011.
[14]
I. G. Tsoulos, D. Gavrilis, and E. Glavas, “Solving differential equations with constructed neural networks,” Neurocomputing, vol. 72, no. 10–12, pp. 2385–2391, 2009.
[15]
L. Jianyu, L. Siwei, Q. Yingjian, and H. Yaping, “Numerical solution of elliptic partial differential equation using radial basis function neural networks,” Neural Networks, vol. 16, no. 5-6, pp. 729–734, 2003.
[16]
Y. Shirvany, M. Hayati, and R. Moradian, “Multilayer perceptron neural networks with novel unsupervised training method for numerical solution of the partial differential equations,” Applied Soft Computing Journal, vol. 9, no. 1, pp. 20–29, 2009.
[17]
N. Mai-Duy and T. Tran-Cong, “Numerical solution of differential equations using multiquadric radial basis function networks,” Neural Networks, vol. 14, no. 2, pp. 185–199, 2001.
[18]
T. Leephakpreeda, “Novel determination of differential-equation solutions: universal approximation method,” Journal of Computational and Applied Mathematics, vol. 146, no. 2, pp. 443–457, 2002.
[19]
C. Franke and R. Schaback, “Solving partial differential equations by collocation using radial basis functions,” Applied Mathematics and Computation, vol. 93, no. 1, pp. 73–82, 1998.
[20]
N. Smaoui and S. Al-Enezi, “Modelling the dynamics of nonlinear partial differential equations using neural networks,” Journal of Computational and Applied Mathematics, vol. 170, no. 1, pp. 27–58, 2004.
[21]
I. G. Tsoulos and I. E. Lagaris, “Solving differential equations with genetic programming,” Genetic Programming and Evolvable Machines, vol. 7, no. 1, pp. 33–54, 2006.
[22]
K. S. McFall and J. R. Mahan, “Artificial neural network method for solution of boundary value problems with exact satisfaction of arbitrary boundary conditions,” IEEE Transactions on Neural Networks, vol. 20, no. 8, pp. 1221–1233, 2009.
[23]
S. A. Hoda and H. A. Nagla, “Neural network methods for mixed boundary value problems,” International Journal of Nonlinear Science, vol. 11, pp. 312–316, 2011.
[24]
J. M. Zurada, Introduction to Artificial Neural Network, West Publishing, 1994.
[25]
S. Haykin, Neural Networks a Comprehensive Foundation, Prentice Hall, New York, NY, USA, 1999.
[26]
S. Chakraverty, V. P. Singh, and R. K. Sharma, “Regression based weight generation algorithm in neural network for estimation of frequencies of vibrating plates,” Computer Methods in Applied Mechanics and Engineering, vol. 195, no. 33–36, pp. 4194–4202, 2006.
[27]
V. P. Singh, S. Chakraverty, R. K. Sharma, and G. K. Sharma, “Modeling vibration frequencies of annular plates by regression based neural network,” Applied Soft Computing Journal, vol. 9, no. 1, pp. 439–447, 2009.
[28]
S. Mall and S. Chakraverty, “Regression Based Neural network training for the solution of ordinary differential equations,” International Journal of Mathematical Modelling and Numerical Optimisation, vol. 4, pp. 136–149, 2013.
