|
|
复杂数据背景下的高斯过程回归模型
|
Abstract:
高斯过程回归模型是一种基于贝叶斯推断的非参数模型，它以概率论为基础，通过在模型中明确地引入随机性，将研究者的先验知识与从观察数据中学习到的知识进行有机融合，并通过贝叶斯推断来减小不确定性。高斯过程回归模型所具有的天然的可解释性、灵活性以及稳健性，决定了其在统计学习领域发挥了重要且不可替代的作用，被广泛应用于各个领域。近年来，随着大数据时代的到来，现实数据不断趋于复杂化、非结构化以及实时化，催生了该模型在可扩展性以及模型结构更新等方面的快速发展。本文对近十年来高斯过程回归模型在大数据领域的拓展算法以及模型的改进方法进行分析总结，概述了各个方法的优缺点，并对高斯过程回归模型的未来研究方向进行展望。
Gaussian process regression model is a nonparametric model based on Bayesian inference. It is based on probability theory. By explicitly introducing randomness into the model, it organically integrates researchers’ prior knowledge and knowledge learned from observation data, and reduces uncertainty through Bayesian inference. The natural interpretability, flexibility and robustness of Gaussian process regression model determine that it plays an important and irreplaceable role in the field of statistical learning and is widely used in various fields. In recent years, with the advent of the big data era, the real data has become increasingly complex, unstructured and real-time, which has led to the rapid development of the model in terms of scalability and model structure update. This paper analyzes and summarizes the expanding algorithms and improved methods of the Gaussian process regression model in the field of big data in the past decade, summarizes the advantages and disadvantages of each method, and looks forward to the future research direction of the Gaussian process regression model.
| [1] | 张江帆. 基于高斯过程回归模型的锂电池数据处理[D]: [硕士学位论文]. 北京: 北京交通大学, 2017. |
| [2] | 郭忠义, 李永华, 李关辉, 彭志勇, 张宁, 于振中. 装备系统剩余使用寿命预测技术研究进展[J]. 南京航空航天大学学报, 2022, 54(3): 341-364. |
| [3] | Lally, N. and Hartman, B. (2018) Estimating Loss Reserves Using Hierarchical Bayesian Gaussian Process Regression with Input Warping. Insurance: Mathematics and Economics, 82, 124-140.
https://doi.org/10.1016/j.insmatheco.2018.06.008 |
| [4] | 余柏杨, 吕宏强, 周岩, 罗振兵, 刘学军. 基于机器学习的高速复杂流场流动控制效果预测分析[J]. 实验流体力学, 2022, 36(3): 44-54. |
| [5] | 梁智, 孙国强, 俞娜燕, 倪晓宇, 沈海平, 卫志农. 基于高斯过程回归和粒子滤波的短期风速预测[J]. 太阳能学报, 2020, 41(3): 45-51. |
| [6] | Velásquez, R.M.A. and Lara, J.V.M. (2020) Forecast and Evaluation of COVID-19 Spreading in USA with Reduced-Space Gaussian Process Regression. Chaos, Solitons & Fractals, 136, Article ID: 109924.
https://doi.org/10.1016/j.chaos.2020.109924 |
| [7] | 何志昆, 刘光斌, 赵曦晶, 王明昊. 高斯过程回归方法综述[J]. 控制与决策, 2013, 28(8): 1121-1129+1137. |
| [8] | 牛文杰. 薄板样条法和泛克里金法在理论和应用方面的比较[J]. 工程图学学报, 2010, 31(4): 123-129. |
| [9] | Preparata, F.P. and Shamos, M.I. (2012) Computational Geometry: An Introduction. Springer Science & Business Media, New York. |
| [10] | Lawrence, N., Seeger, M. and Herbrich, R. (2002) Fast Sparse Gaussian Process Methods: The Informative Vector Machine. Advances in Neural Information Processing Systems 15, Vancouver, 9-14 December 2002, 625-632. |
| [11] | Seeger, M. (2003) Bayesian Gaussian Process Models: PAC-Bayesian Generalisation Error Bounds and Sparse Approximations. University of Edinburgh, Edinburgh. |
| [12] | Keerthi, S. and Chu, W. (2005) A Matching Pursuit Approach to Sparse Gaussian Process Regression. Proceedings of the 18th International Conference on Neural Information Processing Systems, Vancouver, 5-8 December 2005, 643-650. |
| [13] | Melkumyan, A. and Ramos, F.T. (2009) A Sparse Covariance Function for Exact Gaussian Process Inference in Large Datasets. 21st International Joint Conference on Artificial Intelligence, Pasadena, 11-17 July 2009, 1936-1942. |
| [14] | Williams, C.K.I. and Seeger, M. (2001) Using the Nystrom Method to Speed Up Kernel Machines. Proceedings of the 13th International Conference on Neural Information Processing Systems, Denver, January 2000, 661-667. |
| [15] | Huggins, J.H., Campbell, T., Kasprzak, M. and Broderick, T. (2019) Scalable Gaussian Process Inference with Finite-Data Mean and Variance Guarantees. The 22nd International Conference on Artificial Intelligence and Statistics, Naha, 16-18 April 2019, 796-805. |
| [16] | Cheng, C.A. and Boots, B. (2017) Variational Inference for Gaussian Process Models with Linear Complexity. Annual Conference on Neural Information Processing Systems 2017, Long Beach, 4-9 December 2017, 5190-5200. |
| [17] | Choudhury, A., Nair, P.B. and Keane, A.J. (2002) A Data Parallel Approach for Large-Scale Gaussian Process Modeling. Proceedings of the 2002 SIAM International Conference on Data Mining, Arlington, 11-13 April 2002, 95-111.
https://doi.org/10.1137/1.9781611972726.6 |
| [18] | Gramacy, R.B. and Lee, H.K. (2008) Bayesian Treed Gaussian Process Models with an Application to Computer Modeling. Journal of the American Statistical Association, 103, 1119-1130. https://doi.org/10.1198/016214508000000689 |
| [19] | Urtasun, R. and Darrell, T. (2008) Sparse Probabilistic Regression for Activity-Independent Human Pose Inference. 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, 23-28 June 2008, 1-8.
https://doi.org/10.1109/CVPR.2008.4587360 |
| [20] | Chen, T. and Ren, J. (2009) Bagging for Gaussian Process Regression. Neurocomputing, 72, 1605-1610.
https://doi.org/10.1016/j.neucom.2008.09.002 |
| [21] | Chen, K., Xu, L. and Chi, H. (1999) Improved Learning Algorithms for Mixture of Experts in Multiclass Classification. Neural Networks, 12, 1229-1252. https://doi.org/10.1016/S0893-6080(99)00043-X |
| [22] | Bishop, C.M. and Svenskn, M. (2002) Bayesian Hierarchical Mixtures of Experts. In: D’Ambrosio, B., Smets, P. and Bonissone, P.P., Uncertainty in Artificial Intelligence, Morgan Kaufmann Publishers Inc., Burlington, MA, USA, 57-64. |
| [23] | Nguyen, T.N.A., Bouzerdoum, A. and Phung, S.L. (2016) Variational Inference for Infinite Mixtures of Sparse Gaussian Processes through KL-Correction. 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, 20-25 March 2016, 2579-2583. https://doi.org/10.1109/ICASSP.2016.7472143 |
| [24] | Duvenaud, D., Lloyd, J., Grosse, R., et al. (2013) Structure Discovery in Nonparametric Regression through Compositional Kernel Search. International Conference on Machine Learning, Atlanta, 16-21 June 2013, 1166-1174. |
| [25] | Steinruecken, C., Smith, E., Janz, D., Lloyd, J. and Ghahramani, Z. (2019) The Automatic Statistician. In: Hutter, F., Kotthoff, L. and Vanschoren, J., Eds., Automated Machine Learning, Springer, Cham, 161-173.
https://doi.org/10.1007/978-3-030-05318-5_9 |
| [26] | Kronberger, G, and Kommenda, M. (2013) Evolution of Covariance Functions for Gaussian Process Regression Using Genetic Programming. International Conference on Computer Aided Systems Theory, Las Palmas de Gran Canaria, 10-15 February 2013, 308-315. https://doi.org/10.1007/978-3-642-53856-8_39 |
| [27] | Hensman, J., Fusi, N. and Lawrence, N.D. (2013) Gaussian Processes for Big Data. Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence, 12-14 July 2013, arXiv preprint arXiv:1309.6835. |
| [28] | Kim, H. and The, Y.W. (2018) Scaling up the Automatic Statistician: Scalable Structure Discovery Using Gaussian Processes. International Conference on Artificial Intelligence and Statistics, Lanzarote, 9-11 April 2018, 575-584. |
| [29] | Chugh, T., Rahat, A. and Palar, P.S. (2019) Trading-Off Data Fit and Complexity in Training Gaussian Processes with Multiple Kernels. International Conference on Machine Learning, Optimization, and Data Science, Siena, 10-13 September 2019, 579-591. https://doi.org/10.1007/978-3-030-37599-7_48 |
| [30] | Palar, P.S. and Shimoyama, K. (2019) Kriging with Composite Kernel Learning for Surrogate Modeling in Computer Experiments. AIAA Scitech 2019 Forum, 7-11 January 2019, San Diego, 2209. https://doi.org/10.2514/6.2019-2209 |
