|
A Continuous Dynamical Systems Approach to Gauss-Newton MinimizationDOI: 10.4236/oalib.1101028, PP. 1-10 Subject Areas: Ordinary Differential Equation, Dynamical System, Numerical Mathematics Keywords: Gauss-Newton Method, Unconstrained Minimization, Dynamical Systems, Ordinary Differential Equations Abstract In
this paper we show how the iterative Gauss-Newton method for minimizing a
function can be reformulated as a solution to a continuous, autonomous
dynamical system. We investigate the properties of the solutions to a
one-parameter ODE initial value problem that involves the gradient and Hessian
of the function. The equation incorporates an eigenvalue shift conditioner,
which is a non-negative continuous function of the state. It enforces positive
definiteness on a modified Hessian. Assuming the existence of a unique global
minimum, the existence of a bounded connected sub-level set of the function and
that the Hessian is non-zero in the interior of this set, our main results are:
1) existence of local solutions to the ODE
initial value problem; 2) construction of a global solution by
recursive extension of local solutions; 3) convergence of the global solution to the
minimizing state for all initial values contained in the interior of the
bounded level set; 4) eventual exact exponential decay of the
gradient magnitude independent of the particular function and number of its
variables. The results of a numerical experiment on the Rosenbrock Banana using
a constant step-size 4th order Runge-Kutta method are presented and we point
toward the direction of future research. Danchick, R. (2014). A Continuous Dynamical Systems Approach to Gauss-Newton Minimization. Open Access Library Journal, 1, e1028. doi: http://dx.doi.org/10.4236/oalib.1101028. References
comments powered by Disqus |