Feature selection is an NP-hard problem from the viewpoint of algorithm design and it is one of the main open problems in pattern recognition. In this paper, we propose a new evolutionary-incremental framework for feature selection. The proposed framework can be applied on an ordinary evolutionary algorithm (EA) such as genetic algorithm (GA) or invasive weed optimization (IWO). This framework proposes some generic modifications on ordinary EAs to be compatible with the variable length of solutions. In this framework, the solutions related to the primary generations have short length. Then, the length of solutions may be increased through generations gradually. In addition, our evolutionary-incremental framework deploys two new operators called addition and deletion operators which change the length of solutions randomly. For evaluation of the proposed framework, we use that for feature selection in the application of face recognition. In this regard, we applied our feature selection method on a robust face recognition algorithm which is based on the extraction of Gabor coefficients. Experimental results show that our proposed evolutionary-incremental framework can select a few number of features from existing thousands features efficiently. Comparison result of the proposed methods with the previous methods shows that our framework is comprehensive, robust, and well-defined to apply on many EAs for feature selection. 1. Introduction In many pattern recognition problems, hundreds or maybe thousands of features are extracted from data to recognize the patterns in those data. Pattern recognition is a search process in the feature space; thus, when the number of extracted features is increased, computational complexity of search process is usually increased exponentially. Therefore, reduction of feature space is one of the main phases in many pattern recognition systems, especially for real-time systems. One of the approaches for reduction of feature dimensions is PCA (principal component analysis) based methods. PCA tries to remove correlations between features and transfer data to a new uncorrelated space. The new space is built by a linear combination of original feature space and it is orthogonal. Although this approach can reduce dimension of feature spaces, this method does not preserve the essence of original feature spaces in the new space. In other words, unlike the original feature space, the new feature space is usually not meaningful, because it is a linear combination of original features. In [1], a review and comparison of linear and
References
[1]
L. van der Maaten, E. Postma, and J. van den Herik, Dimensionality Reduction: A Comparative Review, Tilburg University, 2009.
[2]
J. Rissanen, “A universal prior for integers and estimation by minimum description length,” The Annals of Statistics, vol. 11, no. 2, pp. 416–431, 1983.
[3]
G. Chandrashekar and F. Sahin, “A survey on feature selection methods,” Computers & Electrical Engineering, vol. 40, no. 1, pp. 16–28, 2014.
[4]
X. Wu, K. Yu, W. Ding, H. Wang, and X. Zhu, “Online feature selection with streaming features,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 5, pp. 1178–1192, 2013.
[5]
H. Liu and R. Setiono, “Incremental feature selection,” Applied Intelligence, vol. 9, no. 3, pp. 217–230, 1998.
[6]
J. H. Holland, Adaption in Natural and Artificial Systems, The University of Michigan Press, Ann Arbor, Mich, USA, 1975.
[7]
T. Murata and H. Ishibuchi, “MOGA: multi-objective genetic algorithms,” in Proceedings of the IEEE International Conference on Evolutionary Computation, pp. 289–294, Perth, Australia, December 1995.
[8]
M. Morita, R. Sabourin, F. Bortolozzi, and C. Y. Suen, “Unsupervised feature selection using multi-objective genetic algorithms for handwritten word recognition,” in Proceedings of the International Conference on Document Analysis and Recognition, pp. 666–670, August 2003.
[9]
Y. Liu, G. Wang, H. Chen, H. Dong, X. Zhu, and S. Wang, “An improved particle swarm optimization for feature selection,” Journal of Bionic Engineering, vol. 8, no. 2, pp. 191–200, 2011.
[10]
M. H. Sigari and C. Lucas, “Near optimal feature selection using evolutionary algorithms for face eecognition: PSO (particle swarm intelligence), ICA (imperialist competitive algorithm) and IWO (invasive weed optimization),” in Proceedings of the 19th Iranian Conference on Electrical Engineering, Tehran, Iran, 2011.
[11]
S. Wu, Y. Hu, W. Wang, X. Feng, and W. Shu, “Application of global optimization methods for feature selection and machine learning,” Mathematical Problems in Engineering, vol. 2013, Article ID 241517, 8 pages, 2013.
[12]
H. Josiński, D. Kostrzewa, A. Michalczuk, and A. ？witoński, “The expanded invasive weed optimization metaheuristic for solving continuous and discrete optimization problems,” The Scientific World Journal, vol. 2014, Article ID 831691, 14 pages, 2014.
[13]
A. R. Mehrabian and C. Lucas, “A novel numerical optimization algorithm inspired from weed colonization,” Ecological Informatics, vol. 1, no. 4, pp. 355–366, 2006.
[14]
L. Wiskott, J. M. Fellous, N. Krüger, and C. von der Malsburg, “Face recognition by elastic bunch graph matching,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 775–779, 1997.
[15]
M. H. Sigari, “Best wavelength selection for gabor wavelet using GA for EBGM algorithm,” in Proceedings of the International Conference on Machine Vision (ICMV '07), pp. 35–39, Islamabad, Pakistan, December 2007.
[16]
K. Venkatesh, S. C. Satapathy, J. V. R. Murthy, and P. V. G. D. Prasad Reddy, “Hybridized improved genetic algorithm with variable length chromosome for image clustering,” International Journal of Computer Science and Network Security, vol. 7, no. 11, pp. 121–131, 2007.
[17]
S. Vijendra and S. Laxman, “Subspace clustering of high-dimensional data: an evolutionary approach,” Applied Computational Intelligence and Soft Computing, vol. 2013, Article ID 863146, 12 pages, 2013.
