A Symmetric Rank-One Quasi-Newton Method for Nonnegative Matrix Factorization

As is well known, the nonnegative matrix factorization (NMF) is a dimension reduction method that has been widely used in image processing, text compressing, signal processing, and so forth. In this paper, an algorithm on nonnegative matrix approximation is proposed. This method is mainly based on a relaxed active set and the quasi-Newton type algorithm, by using the symmetric rank-one and negative curvature direction technologies to approximate the Hessian matrix. The method improves some recent results. In addition, some numerical experiments are presented in the synthetic data, imaging processing, and text clustering. By comparing with the other six nonnegative matrix approximation methods, this method is more robust in almost all cases. 1. Introduction An NMF problem is to decompose a nonnegative matrix into two nonnegative matrix and , such that approximate to as well as possible. To measure the distance between and , there are many methods such as the Kullback Leibler divergence, and Bregman divergence, and Frobenius divergence. Due to the favorable property of the Frobenius divergence, many methods are presented based on it. The Frobenius divergence may be described as follows: subject to , for all and . In the last decade, numerous methods have been proposed to deal with the NMF problem in (1). Most of these methods can be classified into two classes, that is, alternating one-step gradient descent and alternating least squares.(i)The alternating one-step gradient is to alternate and with one step. The most well known is Lee's multiplicative update algorithm [1], which alternates and by the following rules: suppose we have obtained the th matrix and , then (ii)For the frame work of the alternating least squares, the procedure is as follows [2]:(1)initialize with a nonnegative matrix;(2)solve the following problems repeatedly until a stopping criterion is satisfied: where is fixed, and where is fixed. Obviously, the above two methods both satisfy that Equation (5) insures that the above numerical methods converge to the solution of (1). Since (3) and (4) may be regarded as the subproblems of the NMF problem (1). Next we mainly focus on how to solve (4), while (3) is similar to (4), due to the symmetric property of Euclidear distance. There exist some methods to solve (4), such as the projected Newton method [3], quasi-Newton method [4–6], the active set method [7–9], and so forth. Note that the Frobenius norm of a matrix is just the sum of the Eucldean distance over columns (or rows). Then solving (4) in fact is equal to solving a series of the