Kernel Projection Algorithm for Large-Scale SVM Problems

Support Vector Machine (SVM) has become a very effective method in statistical machine learning and it has proved that training SMV is to solve Nearest Point pair Problem (NPP) between two disjoint closed convex sets. Later Keerthi pointed out that it is difficult to apply classical excellent geometric algorithms direcly to SVM and so designed a new geometric algorithm for SVM. In this article, a new algorithm for geometrically solving SVM, Kernel Projection Algorithm, is presented based on the theorem on fixed-points of projection mapping. This new algorithm makes it easy to apply classical geometric algorithms to solving SVM and is more understandable than Keerthi’s. Experiments show that the new algorithm can also handle large-scale SVM problems. Geometric algorithms for SVM, such as Keerthi’s algorithm, require that two closed convex sets be disjoint and otherwise the algorithms are meaningless. In this article, this requirement will be guaranteed in theory by using the theoretic result on universal kernel functions. This research is supported by the NKBRSF of China (Grant No. G1998030508), the National Natural Science Foundation of in China (Grant No. 60175032) and the Pilot Program of the Knowledge Innovation Project of Chinese Academy of Sciences. WANG Jiaqi received his B.S. degree from Beijing Polytechnic University in 1998. He is currently a graduate student in Institute of Automation, Chinese Academy of Sciences, P.R. China. His research interests are data mining, machine learning and kernel method. TAO Qing received the M.S. degree in mathematics from Southwest Normal University in 1989 and the Ph.D. degree from the University of Science and Technology of China in 1999. From June 1999 to June 2001, he was a Postdoctoral Fellow in the University of Science and Technology of China. He is currently a Postdoctoral Fellow in Institute of Automation, Chinese Academy of Sciences. His research interests are neural networks, nonlinear function analysis and SVM theory. WANG Jue is a professor in Institute of Automation, Chinese Academy of Science. His research interests are ANN, GA, multi-agent system, machine learning and data mining.