This paper develops a process whereby a high-dimensional clustering problem is solved using a neural network and a low-dimensional cluster diagram of the results is produced using the Mapper method from topological data analysis. The low-dimensional cluster diagram makes the neural network's solution to the high-dimensional clustering problem easy to visualize, interpret, and understand. As a case study, a clustering problem from a diabetes study is solved using a neural network. The clusters in this neural network are visualized using the Mapper method during several stages of the iterative process used to construct the neural network. The neural network and Mapper clustering diagram results for the diabetes study are validated by comparison to principal component analysis. 1. Introduction Topological data analysis (TDA) is an emerging field of mathematics that focuses on constructing topological models for data and calculating algebraic invariants of such models [1–3]. The fundamental idea is to use methods from topology to determine shapes or patterns in high-dimensional data sets [4]. One method from TDA called Mapper constructs a low-dimensional topological model for a data set from the clusters in the level sets of a function on the data set [5]. This topological model for is a cluster diagram that shows the clusters in the level sets of (i.e., clusters in the layers of a stratification of ) and how clusters in adjacent, overlapping level sets are connected (i.e., how the neighboring layers are glued together). The topological model built in this way is analogous to how Morse theory is used to construct a cell decomposition of a manifold using sublevel sets of a Morse function on the manifold [5–7]. The resolution of the cluster diagram produced by Mapper can be adjusted by changing the level sets by varying the number, size, and shape of the regions used to cover the image of the function . Further, the Mapper method allows for different clustering algorithms to be used. The most important step for obtaining a useful topological model from Mapper is finding a function that solves a particular clustering problem of interest for a data set . This study examines the case when the function is a neural network. A feedforward, multilayer perceptron artificial neural network (hereafter called a neural network) is a function constructed by an iterative process in order to approximate a training function between two finite sets of points and called the inputs and target outputs, where and . In a context where a target output value represents the
References
[1]
G. Carlsson, “Topology and data,” Bulletin of the American Mathematical Society, vol. 46, no. 2, pp. 255–308, 2009.
[2]
H. Edelsbrunner and J. Harer, Computational Topology: An Introduction, American Mathematical Society, Providence, RI, USA, 2010.
[3]
A. Zomorodian, Topology for Computing, Cambridge University Press, New York, NY, USA, 2005.
[4]
P. Y. Lum, G. Singh, A. Lehman et al., “Extracting insights from the shape of complex data using topology,” Scientific Reports, vol. 3, article 1236, 2013.
[5]
G. Singh, F. Mémoli, and G. Carlsson, “Topological methods for the analysis of high dimensional data sets and 3D object recognition,” in Eurographics Symposium on Point-Based Graphics (Prague '07), pp. 91–100.
[6]
H. Adams, A. Atanasov, and G. Carlsson, “Morse theory in topological data analysis,” http://arxiv.org/abs/1112.1993.
[7]
C. Marzban and U. Yurtsever, “Baby morse theory in data analysis,” in Proceedings of the Workshop on Knowledge Discovery, Modeling and Simulation (KDMS '11), pp. 15–21, August 2011.
[8]
K.-L. Du, “Clustering: a neural network approach,” Neural Networks, vol. 23, no. 1, pp. 89–107, 2010.
[9]
J. Herrero, A. Valencia, and J. Dopazo, “A hierarchical unsupervised growing neural network for clustering gene expression patterns,” Bioinformatics, vol. 17, no. 2, pp. 126–136, 2001.
[10]
M. Hagan, H. Demuth, and M. Beale, Neural Network Design, PWS Publishing, Boston, Mass, USA, 1995.
[11]
R. Marks and R. Reed, Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks, Denver, Bradford, UK, 1999.
[12]
R. Rojas, Neural Networks: A Systematic Introduction, Springer, New York, NY, USA, 1996.
[13]
G. M. Reaven and R. G. Miller, “An attempt to define the nature of chemical diabetes using a multidimensional analysis,” Diabetologia, vol. 16, no. 1, pp. 17–24, 1979.
[14]
R. Miller, “Discussion—projection pursuit,” Annals of Statistics, vol. 13, no. 2, pp. 510–513, 1985.
[15]
S. Walczak, “Methodological triangulation using neural networks for business research,” Advances in Artificial Neural Systems, vol. 2012, Article ID 517234, 12 pages, 2012.
[16]
M. Nicolau, A. J. Levine, and G. Carlsson, “Topology based data analysis identifies a subgroup of breast cancers with a unique mutational profile and excellent survival,” Proceedings of the National Academy of Sciences of the United States of America, vol. 108, no. 17, pp. 7265–7270, 2011.
[17]
G. R. Bowman, X. Huang, Y. Yao et al., “Structural insight into RNA hairpin folding intermediates,” Journal of the American Chemical Society, vol. 130, no. 30, pp. 9676–9678, 2008.
[18]
J. Mao and A. K. Jain, “Artificial neural networks for feature extraction and multivariate data projection,” IEEE Transactions on Neural Networks, vol. 6, no. 2, pp. 296–317, 1995.
[19]
M. A. Kramer, “Nonlinear principal component analysis using autoassociative neural networks,” AIChE Journal, vol. 37, no. 2, pp. 233–243, 1991.
[20]
D. De Ridder and R. P. W. Duin, “Sammon's mapping using neural networks: a comparison,” Pattern Recognition Letters, vol. 18, no. 11-13, pp. 1307–1316, 1997.
[21]
D. K. Agrafiotis and V. S. Lobanov, “Nonlinear mapping networks,” Journal of Chemical Information and Computer Sciences, vol. 40, no. 6, pp. 1356–1362, 2000.
[22]
W. Pedrycz, “Conditional fuzzy clustering in the design of radial basis function neural networks,” IEEE Transactions on Neural Networks, vol. 9, no. 4, pp. 601–612, 1998.
[23]
G. Cybenko, “Approximation by superpositions of a sigmoidal function,” Mathematics of Control, Signals, and Systems, vol. 2, no. 4, pp. 303–314, 1989.
[24]
K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural Networks, vol. 2, no. 5, pp. 359–366, 1989.
[25]
P. G. Goerss and J. F. Jardine, Simplicial Homotopy Theory, Birkh？user, Basel, Switzerland, 2009.
[26]
P. J. Huber, “Projection pursuit,” The Annals of Statistics, vol. 13, no. 2, pp. 435–475, 1985.
[27]
D. F. Andrews and A. M. Herzberg, Data: A Collection of Problems from Many Fields for the Student and Research Worker, Springer, New York, NY, USA, 1985.
[28]
C. R. Rao, “The use and interpretation of principal component analysis in applied research,” Sankhya Series A, vol. 26, pp. 329–358, 1964.
[29]
R. J. Bolton and W. J. Krzanowski, “A characterization of principal components for projection pursuit,” The American Statistician, vol. 53, no. 2, pp. 108–109, 1999.
[30]
C. Croux, P. Filzmoser, and M. R. Oliveira, “Algorithms for Projection-Pursuit robust principal component analysis,” Chemometrics and Intelligent Laboratory Systems, vol. 87, no. 2, pp. 218–225, 2007.
[31]
Y. LeCun, L. Bottou, G. Orr, and K. Müller, “Efficient backprop,” in Neural Networks: Tricks of the Trade, Springer, New York, NY, USA, 1998.
[32]
D. Müllner and G. Singh, “Mapper 1d for matlab,” 2013, http://comptop.stanford.edu/programs/.
M. Halkidi, Y. Batistakis, and M. Vazirgiannis, “On clustering validation techniques,” Journal of Intelligent Information Systems, vol. 17, no. 2-3, pp. 107–145, 2001.
