New versions of the Hestenes-Stiefel nonlinear conjugate gradient method based on the secant condition for optimization

Based on the secant condition often satisfied by quasi-Newton methods, two new versions of the Hestenes-Stiefel (HS) nonlinear conjugate gradient method are proposed, which are descent methods even with inexact line searches. The search directions of the proposed methods have the form d k = - θkg k + βkHSd k-1, or d k = -g k + βkHSd k-1+ θky k-1. When exact line searches are used, the proposed methods reduce to the standard HS method. Convergence properties of the proposed methods are discussed. These results are also extended to some other conjugate gradient methods such as the Polak-Ribiére-Polyak (PRP) method. Numerical results are reported.