The study is concerned with data and feature reduction in fuzzy modeling. As these reduction activities are advantageous to fuzzy models in terms of both the effectiveness of their construction and the interpretation of the resulting models, their realization deserves particular attention. The formation of a subset of meaningful features and a subset of essential instances is discussed in the context of fuzzy-rule-based models. In contrast to the existing studies, which are focused predominantly on feature selection (namely, a reduction of the input space), a position advocated here is that a reduction has to involve both data and features to become efficient to the design of fuzzy model. The reduction problem is combinatorial in its nature and, as such, calls for the use of advanced optimization techniques. In this study, we use a technique of particle swarm optimization (PSO) as an optimization vehicle of forming a subset of features and data (instances) to design a fuzzy model. Given the dimensionality of the problem (as the search space involves both features and instances), we discuss a cooperative version of the PSO along with a clustering mechanism of forming a partition of the overall search space. Finally, a series of numeric experiments using several machine learning data sets is presented. 1. Introduction In fuzzy modeling, the two main approaches for generating the rules rely on knowledge acquisition from human experts and knowledge discovery from data [1, 2]. In recent years, knowledge discovery from data or data-driven fuzzy modeling has become more important [2–4]. In many cases, the ability to develop models efficiently is hampered by the dimensionality of the input space as well as the number of data. If we are concerned with rule-based models, the high-dimensionality of the feature space along with the topology of the rules gives rise to the curse of dimensionality [1, 4]. The number of rules increases exponentially and is equal to , where is the number of features and stands for the number of fuzzy sets defined for each feature. The factors that contribute most to the accuracy of the data-driven fuzzy modeling are associated with the size of the input space and the decomposition of the input data. A Large number of data points or instances in a continuous input-output domain exhibit a significant impact on fuzzy models. It is well known that more training data will not always lead to a better performance for data-driven models. Large amount of training data have important implications on the modeling capabilities. Since the number of
References
[1]
W. Pedrycz and F. Gomide, Fuzzy Systems Engineering, John Wiley & Sons, Hoboken, NJ, USA, 2007.
[2]
G. Castellano, C. Castiello, A. M. Fanelli, and C. Mencar, “Knowledge discovery by a neuro-fuzzy modeling framework,” Fuzzy Sets and Systems, vol. 149, no. 1, pp. 187–207, 2005.
[3]
Y. Jin, “Fuzzy modeling of high-dimensional systems: complexity reduction and interpretability improvement,” IEEE Transactions on Fuzzy Systems, vol. 8, no. 2, pp. 212–221, 2000.
[4]
Q. Zhang and M. Mahfouf, “A hierarchical Mamdani-type fuzzy modelling approach with new training data selection and multi-objective optimisation mechanisms: a special application for the prediction of mechanical properties of alloy steels,” Applied Soft Computing Journal, vol. 11, no. 2, pp. 2419–2443, 2011.
[5]
G. E. Tsekouras, “On the use of the weighted fuzzy c-means in fuzzy modeling,” Advances in Engineering Software, vol. 36, no. 5, pp. 287–300, 2005.
[6]
A. G. Di Nuovo, M. Palesi, and V. Catania, “Multi-objective evolutionary fuzzy clustering for high-dimensional problems,” in Proceedings of the IEEE International Conference on Fuzzy Systems, pp. 1–6, July 2007.
[7]
M. Setnes, R. Babu？ka, U. Kaymak, and H. R. Van Nauta Lemke, “Similarity measures in fuzzy rule base simplification,” IEEE Transactions on Systems, Man, and Cybernetics B, vol. 28, no. 3, pp. 376–386, 1998.
[8]
M. Y. Chen and D. A. Linkens, “Rule-base self-generation and simplification for data-driven fuzzy models,” Fuzzy Sets and Systems, vol. 142, no. 2, pp. 243–265, 2004.
[9]
H. Wang, S. Kwong, Y. Jin, W. Wei, and K. F. Man, “Multi-objective hierarchical genetic algorithm for interpretable fuzzy rule-based knowledge extraction,” Fuzzy Sets and Systems, vol. 149, no. 1, pp. 149–186, 2005.
[10]
F. J. Berlanga, A. J. Rivera, M. J. del Jesus, and F. Herrera, “GP-COACH: genetic Programming-based learning of COmpact and ACcurate fuzzy rule-based classification systems for High-dimensional problems,” Information Sciences, vol. 180, no. 8, pp. 1183–1200, 2010.
[11]
J. Alcalá-Fdéz, R. Alcalá, and F. Herrera, “A fuzzy associative classification system with genetic rule selection for high-dimensional problems,” in Proceedings of the 4th International Workshop on Genetic and Evolutionary Fuzzy Systems (GEFS '10), pp. 33–38, March 2010.
[12]
Y. Chen, B. Yang, A. Abraham, and L. Peng, “Automatic design of hierarchical Takagi-Sugeno type fuzzy systems using evolutionary algorithms,” IEEE Transactions on Fuzzy Systems, vol. 15, no. 3, pp. 385–397, 2007.
[13]
M. R. Delgado, F. V. Zuben, and F. Gomide, “Coevolutionary genetic fuzzy systems: a hierarchical collaborative approach,” Fuzzy Sets and Systems, vol. 141, no. 1, pp. 89–106, 2004.
[14]
N. Xiong and L. Litz, “Reduction of fuzzy control rules by means of premise learning—method and case study,” Fuzzy Sets and Systems, vol. 132, no. 2, pp. 217–231, 2002.
[15]
A. E. Gaweda, J. M. Zurada, and R. Setiono, “Input selection in data-driven fuzzy modeling,” in Proceedings of the 10th IEEE International Conference on Fuzzy Systems, vol. 3, pp. 1251–1254, December 2001.
[16]
M. L. Hadjili and V. Wertz, “Takagi-Sugeno fuzzy modeling incorporating input variables selection,” IEEE Transactions on Fuzzy Systems, vol. 10, no. 6, pp. 728–742, 2002.
[17]
R. ？indelá？ and R. Babu？ka, “Input selection for nonlinear regression models,” IEEE Transactions on Fuzzy Systems, vol. 12, no. 5, pp. 688–696, 2004.
[18]
M. H. F. Zarandi, I. B. Türk？en, and B. Rezaee, “A systematic approach to fuzzy modeling for rule generation from numerical data,” in Proceedings of the Annual Meeting of the North American Fuzzy Information Processing Society (NAFIPS '04), pp. 768–773, June 2004.
[19]
H. Du and N. Zhang, “Application of evolving Takagi-Sugeno fuzzy model to nonlinear system identification,” Applied Soft Computing Journal, vol. 8, no. 1, pp. 676–686, 2008.
[20]
S. N. Ghazavi and T. W. Liao, “Medical data mining by fuzzy modeling with selected features,” Artificial Intelligence in Medicine, vol. 43, no. 3, pp. 195–206, 2008.
[21]
Y. Zhang, X. B. Wu, Z. Y. Xing, and W. L. Hu, “On generating interpretable and precise fuzzy systems based on Pareto multi-objective cooperative co-evolutionary algorithm,” Applied Soft Computing Journal, vol. 11, no. 1, pp. 1284–1294, 2011.
[22]
F. Wan, H. Shang, L. X. Wang, and Y. X. Sun, “How to determine the minimum number of fuzzy rules to achieve given accuracy: a computational geometric approach to SISO case,” Fuzzy Sets and Systems, vol. 150, no. 2, pp. 199–209, 2005.
[23]
I. Guyon and A. Elisseeff, “An introduction to variable and feature selection,” Journal of Machine Learning Research, vol. 3, pp. 1157–1182, 2003.
[24]
H. Liu and L. Yu, “Toward integrating feature selection algorithms for classification and clustering,” IEEE Transactions on Knowledge and Data Engineering, vol. 17, no. 4, pp. 491–502, 2005.
[25]
H. Liu and H. Motoda, Computational Methods of Feature Selection, Chapman & Hall/CRC, Boca Raton, Fla USA, 2008.
[26]
J. A. Olvera-López, J. A. Carrasco-Ochoa, J. F. Martínez-Trinidad, and J. Kittler, “A review of instance selection methods,” Artificial Intelligence Review, vol. 34, no. 2, pp. 133–143, 2010.
[27]
H. Liu and H. Motoda, Instance Selection and Construction for Data Mining, Kluwer Academic Publishers, Boston, Mass USA, 2001.
[28]
H. Ishibuchi, T. Nakashima, and M. Nii, “Genetic-Algorithm-Based instance and feature selection,” in Instance Selection and Construction for Data Mining, H. Lui and H. Motoda, Eds., pp. 95–112, Kluwer Academic Publishers, Boston, Mass, USA, 2001.
[29]
J. Derrac, S. García, and F. Herrera, “IFS-CoCo: instance and feature selection based on cooperative coevolution with nearest neighbor rule,” Pattern Recognition, vol. 43, no. 6, pp. 2082–2105, 2010.
[30]
A. Hertz and D. Kobler, “Framework for the description of evolutionary algorithms,” European Journal of Operational Research, vol. 126, no. 1, pp. 1–12, 2000.
[31]
J. R. Cano, F. Herrera, and M. Lozano, “Using evolutionary algorithms as instance selection for data reduction in KDD: an experimental study,” IEEE Transactions on Evolutionary Computation, vol. 7, no. 6, pp. 561–575, 2003.
[32]
J. Kennedy and R. Eberhart, “Particle swarm optimization,” in Proceedings of the IEEE International Conference on Neural Networks, pp. 1942–1948, Perth, Australia, December 1995.
[33]
F. van den Bergh and A. P. Engelbrecht, “A cooperative approach to particle swarm optimization,” IEEE Transactions on Evolutionary Computation, vol. 8, no. 3, pp. 225–239, 2004.
[34]
Y. Shi and R. C. Eberhart, “Empirical study of particle Swarm Optimization,” Congress on Evolutionary Computing, vol. 3, pp. 1945–1950, 1999.
