This is the first paper to be written on the theory of structural learning. The first section outlines the overall concept; the second section proposes the logical universe as the environment for structural learning; the third, fourth and fifth sections detail the theories of the Central Diagram Processor, Software Diagram and Diagram Thinking, with the Central Diagram Processor acting as the machine’s brain, the Software Diagram as its body, and the Diagram Thinking as its life activities; finally, the sixth section explains existing research achievements in machine life and structural learning. The main purpose of this paper is to recommend the idea of Structural Learning for Machine Life to human society.
References
[1]
Liu, D.H. (2024) Logical Structure School I (Five Fundamental Theories). Intelligent Control and Automation, 15, 125-159. https://doi.org/10.4236/ica.2024.154008
[2]
Liu, D.H. (2024) The Fundamental Theory of Artificial Intelligence—Logic Structure and Logic Engineering. Intelligent Control and Automation, 15, 28-62. https://doi.org/10.4236/ica.2024.151003
[3]
Liu, D.H. (2024) Logical Structure School II (Nine Applied Theories). Intelligent Control and Automation, 15, 160-214. https://doi.org/10.4236/ica.2024.154009
[4]
Chen, J., Zhou, D., Tang, Y., Yang, Z., Cao, Y. and Gu, Q. (2020) Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks. Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, Yokohama, 11-17 July 2020, 3267-3275. https://doi.org/10.24963/ijcai.2020/452
[5]
Dauphin, Y.N., Pascanu, R., Gulcehre, C., et al. (2014) Identifying and Attacking the Saddle Point Problem in High Dimensional Nonconvex Optimization. NeurIPS 2014, Montréal, 8-13 December 2014, 2933-2941.
[6]
Wang, Y.X., Ramanan, D. and Hebert, M. (2017) Learning to Model the Tail. Proceedings of the 31stInternational Conference on Neural Information Processing Systems, Long Beach, 4-9 December 2017, 7032-7042.
[7]
Cao, K., Wei, C., Gaidon, A., et al. (2019) Learning Imbalanced Datasets with Label Distribution Aware Margin Loss. Proceedings of the 33rd International Conference on Neural Information Processing Systems, Vancouver, 8-14 December 2019, 1567-1578.
[8]
Duchi, J., Hazan, E. and Singer, Y. (2011) Adaptive Subgradient Methods for Online Learning and Stochastic Optimization. Journal of Machine Learning Research, 12, 2121-2159.
[9]
Yang, Z., Dai, Z., Yang, Y., et al. (2019) XLNet: Generalized Autoregressive Pretraining for Language Understanding. arXiv: 1906.08237.
[10]
Bartlett, P.L. (1998) The Sample Complexity of Pattern Classification with Neural Networks: The Size of the Weights Is More Important than the Size of the Network. IEEE Transactions on Information Theory, 44, 525-536. https://doi.org/10.1109/18.661502
[11]
Neyshabur, B., Tomioka, R. and Srebro, N. (2015) Norm-Based Capacity Control in Neural Networks. arXiv: 1503.00036.
Long, P.M. and Sedghi, H. (2019) Generalization Bounds for Deep Convolutional Neural Networks. arXiv: 1905.12600.
[14]
Wei, C. and Ma, T. (2019) Data-Dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation. arXiv: 1905.03684
[15]
Dziugaite, G.K. and Roy, D.M. (2017) Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data. arXiv:1703.11008
[16]
Kang, B., Xie, S., Rohrbach, M., et al. (2019) Decoupling Representation and Classifier for Long-Tailed Recognition. arXiv: 1910.09217
[17]
Hinton, G. (2012) Lecture 6D: A Separate, Adaptive Learning Rate for Each Connection. Slides of Lecture Neural Networks for Machine Learning. Technical Report.
[18]
Li, H., Xu, Z., Taylor, G., et al. (2018) Visualizing the Loss Landscape of Neural Nets. Proceedings ofthe 32nd International Conference on Neural Information Processing Systems, Montréal, 3-8 December 2018, 6391-6401.
[19]
Tang, K., Huang, J. and Zhang, H. (2020) Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect. arXiv: 2009.12991.
[20]
Vaswani, A., Shazeer, N., Parmar, N., et al. (2017) Attention Is All You Need. Advances in NeuralInformation Processing Systems, Long Beach, 4-9 December 2017, 5998-6008.
[21]
Neyshabur, B., Bhojanapalli, S., Mcallester, D., et al. (2017) Exploring Generalization in Deep Learning. arXiv: 1706.08947.
[22]
Yao, Z., Gholami, A., Keutzer, K., et al. (2018) Hessian-Based Analysis of Large Batch Training and Robustness to Adversaries. Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montréal, 3-8 December 2018, 4954-4964.
[23]
Ioffe, S. and Szegedy, C. (2015) Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. Proceedings of the 32nd International Conference on International Conference on Machine Learning, 37, 448-456.
[24]
Madry, A., Makelov, A., Schmidt, L., et al. (2017) Towards Deep Learning Models Resistant to Adversarial Attacks. arXiv: 1706.06083.
