The stochastic configuration network (SCN) is an incremental neural network with fast convergence, efficient learning and strong generalization ability, and is widely used in fields such as medical data analysis. However, SCN is mainly used for supervised learning and its performance is limited in the case of scarce labeled data. To this end, this paper proposes semi-supervised SCN (MR-SCN) in combination with manifold regularization to make full use of unlabeled data to improve the model performance. Experimental results show that MR-SCN can still maintain high classification accuracy with a small number of labeled samples, which is better than LapRLS, SS-ELM and LapSVM, and the training time is shorter, showing good learning ability and computational efficiency.
References
[1]
Tavernier, J., Simm, J., Meerbergen, K., Wegner, J.K., Ceulemans, H. and Moreau, Y. (2019) Fast Semi-Supervised Discriminant Analysis for Binary Classification of Large Data Sets. Pattern Recognition, 91, 86-99. https://doi.org/10.1016/j.patcog.2019.02.015
[2]
Wang, S., Lu, J., Gu, X., Du, H. and Yang, J. (2016) Semi-Supervised Linear Discriminant Analysis for Dimension Reduction and Classification. Pattern Recognition, 57, 179-189. https://doi.org/10.1016/j.patcog.2016.02.019
[3]
Peng, J., Estrada, G., Pedersoli, M. and Desrosiers, C. (2020) Deep Co-Training for Semi-Supervised Image Segmentation. Pattern Recognition, 107, Article 107269. https://doi.org/10.1016/j.patcog.2020.107269
[4]
Zhang, B., Bai, B. and Su, J. (2007) Semi-Supervised Text Classification Based on Self-Training EM Algorithm. Journal-National University of Defense Technology, 29, 65-69.
[5]
Song, Z., Yang, X., Xu, Z. and King, I. (2023) Graph-Based Semi-Supervised Learning: A Comprehensive Review. IEEE Transactions on Neural Networks and Learning Systems, 34, 8174-8194. https://doi.org/10.1109/tnnls.2022.3155478
[6]
Chong, Y., Ding, Y., Yan, Q. and Pan, S. (2020) Graph-Based Semi-Supervised Learning: A Review. Neurocomputing, 408, 216-230. https://doi.org/10.1016/j.neucom.2019.12.130
[7]
Zhao, H., Zheng, J., Deng, W. and Song, Y. (2020) Semi-Supervised Broad Learning System Based on Manifold Regularization and Broad Network. IEEE Transactions on Circuits and Systems I: Regular Papers, 67, 983-994. https://doi.org/10.1109/tcsi.2019.2959886
[8]
Belkin, M., Niyogi, P. and Sindhwani, V. (2006) Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples. Journal of Machine Learning Research, 7, 2399-2434.
[9]
Huang, G., Song, S., Gupta, J.N.D. and Cheng Wu, (2014) Semi-Supervised and Unsupervised Extreme Learning Machines. IEEE Transactions on Cybernetics, 44, 2405-2417. https://doi.org/10.1109/tcyb.2014.2307349
[10]
Li, K., Zhang, J., Xu, H., Luo, S. and Li, H. (2013) A Semi-Supervised Extreme Learning Machine Method Based on Co-Training. Journal of Computer Information Systems, 9, 207-214.
[11]
Krizhevsky, A., Sutskever, I. and Hinton, G.E. (2017) ImageNet Classification with Deep Convolutional Neural Networks. Communications of the ACM, 60, 84-90. https://doi.org/10.1145/3065386
[12]
Medsker, L.R. and Jain, L. (2001) Recurrent Neural Networks: Design and Applications. CRC Press.
[13]
Pao, Y.-H. and Takefuji, Y. (1992) Functional-Link Net Computing: Theory, System Architecture, and Functionalities. Computer, 25, 76-79. https://doi.org/10.1109/2.144401
[14]
Pao, Y., Park, G. and Sobajic, D.J. (1994) Learning and Generalization Characteristics of the Random Vector Functional-Link Net. Neurocomputing, 6, 163-180. https://doi.org/10.1016/0925-2312(94)90053-1
[15]
Schmidt, W.F., Kraaijveld, M.A. and Duin, R.P.W. (1992) Feedforward Neural Networks with Random Weights. Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol. II. ConferenceB: Pattern Recognition Methodology and Systems, The Hague, 30 August-3 September 1992, 1-4. https://doi.org/10.1109/icpr.1992.201708
[16]
Wang, D. and Li, M. (2017) Stochastic Configuration Networks: Fundamentals and Algorithms. IEEE Transactions on Cybernetics, 47, 3466-3479. https://doi.org/10.1109/tcyb.2017.2734043
[17]
Wang, Q., Dai, W., Lu, Q., Fu, X. and Ma, X. (2022) A Sparse Learning Method for SCN Soft Measurement Model. Control and Decision, 37, 3171-3182.
[18]
Zhao, L., Zou, S., Guo, S. and Huang, M. (2020) Ball Mill Load Condition Recognition Model Based on Regularized Stochastic Configuration Networks. Control Engineering of China, 27, 1-7.
[19]
Li, W., Zeng, Z., Qu, H. and Sun, C. (2019) A Novel Fiber Intrusion Signal Recognition Method for OFPS Based on SCN with Dropout. Journal of Lightwave Technology, 37, 5221-5230. https://doi.org/10.1109/jlt.2019.2930624
[20]
Wang, D. and Li, M. (2017) Robust Stochastic Configuration Networks with Kernel Density Estimation for Uncertain Data Regression. Information Sciences, 412, 210-222. https://doi.org/10.1016/j.ins.2017.05.047
[21]
Cao, W., Xie, Z., Li, J., Xu, Z., Ming, Z. and Wang, X. (2021) Bidirectional Stochastic Configuration Network for Regression Problems. Neural Networks, 140, 237-246. https://doi.org/10.1016/j.neunet.2021.03.016
[22]
Dai, W., Li, D., Zhou, P. and Chai, T. (2019) Stochastic Configuration Networks with Block Increments for Data Modeling in Process Industries. Information Sciences, 484, 367-386. https://doi.org/10.1016/j.ins.2019.01.062
[23]
Dai, W., Zhou, X., Li, D., Zhu, S. and Wang, X. (2022) Hybrid Parallel Stochastic Configuration Networks for Industrial Data Analytics. IEEE Transactions on Industrial Informatics, 18, 2331-2341. https://doi.org/10.1109/tii.2021.3096840
[24]
Wang, D. and Li, M. (2018) Deep Stochastic Configuration Networks with Universal Approximation Property. 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, 8-13 July 2018, 1-8. https://doi.org/10.1109/ijcnn.2018.8489695
[25]
Pratama, M. and Wang, D. (2019) Deep Stacked Stochastic Configuration Networks for Lifelong Learning of Non-Stationary Data Streams. Information Sciences, 495, 150-174. https://doi.org/10.1016/j.ins.2019.04.055
[26]
Ai, W. and Wang, D. (2020) Distributed Stochastic Configuration Networks with Cooperative Learning Paradigm. Information Sciences, 540, 1-16. https://doi.org/10.1016/j.ins.2020.05.112
[27]
Fierimonte, R., Scardapane, S., Uncini, A. and Panella, M. (2017) Fully Decentralized Semi-Supervised Learning via Privacy-Preserving Matrix Completion. IEEE Transactions on Neural Networks and Learning Systems, 28, 2699-2711. https://doi.org/10.1109/tnnls.2016.2597444
