Model averaging has attracted increasing attention in recent years for
the analysis of high-dimensional data. By weighting several competing
statistical models suitably, model averaging attempts to achieve stable and
improved prediction. To obtain a better understanding of the available model
averaging methods, their properties and the relationships between them, this
paper is devoted to make a review on some recent progresses in high-dimensional
model averaging from the frequentist perspective. Some future research topics
are also discussed.
References
[1]
Bühlmann, P. and van de Geer, S. (2011) Statistics for High-Dimensional Data: Methods, Theory and Applications. Springer, Berlin.
https://doi.org/10.1007/978-3-642-20192-9
[2]
Li, X. and Xu, R. (2009) High-Dimensional Data Analysis in Cancer Research. Springer, Berlin. https://doi.org/10.1007/978-0-387-69765-9
[3]
Tibshirani, R. (1996) Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society: Series B, 58, 268-288.
[4]
Fan, J. and Li, R. (2001) Variable Selection via Nonconcave Penalized Likelihood and Its Oracle Properties. Journal of the American Statistical Association, 96, 1348-1360. https://doi.org/10.1198/016214501753382273
[5]
Zou, H. and Hastie, T. (2005) Regularization and Variable Selection via the Elastic Net. Journal of the Royal Statistical Society: Series B, 67, 301-320.
https://doi.org/10.1111/j.1467-9868.2005.00503.x
[6]
Zhang, C. (2010) Nearly Unbiased Variable Selection under Minimax Concave Penalty. The Annals of Statistics, 38, 894-942. https://doi.org/10.1214/09-AOS729
[7]
Wang, H., Zhang, X. and Zou, G. (2009) Frequentist Model Averaging Estimation: A Review. Journal of Systems Science and Complexity, 22, 732-748.
https://doi.org/10.1007/s11424-009-9198-y
[8]
Liang, H., Zou, G., Wan, A.T.K. and Zhang, X. (2011) Optimal Weight Choice for Frequentist Model Average Estimators. Journal of the American Statistical Association, 106, 1053-1066. https://doi.org/10.1198/jasa.2011.tm09478
[9]
Fragoso, T.M. and Neto, F.L. (2015) Bayesian Model Averaging: A Systematic Review and Conceptual Classification. arXiv:1509.08864.
[10]
Akaike, H. (1979) A Bayesian Extension of the Minimum AIC Procedure of Autoregressive Model Fitting. Biometrika, 66, 237-242.
https://doi.org/10.1093/biomet/66.2.237
[11]
Hoeting, J., Madigan, D., Raftery, A. and Volinsky, C. (1999) Bayesian Model Averaging. Statistical Science, 14, 382-401.
[12]
Hansen, B.E. (2007) Least Squares Model Averaging. Econometrica, 75, 1175-1189.
https://doi.org/10.1111/j.1468-0262.2007.00785.x
[13]
Wan, A., Zhang, X. and Zou, G. (2010) Least Squares Model Averaging by Mallows Criterion. Journal of Econometrics, 156, 277-283.
https://doi.org/10.1016/j.jeconom.2009.10.030
[14]
Hansen, B.E. and Racine, J. (2012) Jackknife Model Averaging. Journal of Econometrics, 167, 38-46. https://doi.org/10.1016/j.jeconom.2011.06.019
[15]
Zhang, X.Y., Wan, A.T.K. and Zou, G.H. (2013) Model Averaging by Jackknife Criterion in Models with Dependent Data. Journal of Econometrics, 174, 82-94.
https://doi.org/10.1016/j.jeconom.2013.01.004
[16]
Ando, T. and Li, K.C. (2014) A Model-Averaging Approach for High-Dimensional Regression. Journal of the American Statistical Association, 109, 254-265.
https://doi.org/10.1080/01621459.2013.838168
[17]
Ando, T. and Li, K.C. (2017) A Weight-Relaxed Model Averaging Approach for High-Dimensional Generalized Linear Models. The Annals of Statistics, 45, 2654-2679. https://doi.org/10.1214/17-AOS1538
[18]
Lin, B., Wang, Q., Zhang, J. and Pang, Z. (2017) Stable Prediction in High-Dimensional Linear Models. Statistics and Computing, 27, 1401-1412.
https://doi.org/10.1007/s11222-016-9694-6
[19]
Zhang, X., Zou, G. and Liang, H. (2014) Model Averaging and Weight Choice in Linear Mixed-Effects Models. Biometrika, 101, 205-218.
https://doi.org/10.1093/biomet/ast052
[20]
Schomaker, M., Wan, A.T.K. and Heumann, C. (2010) Frequentist Model Averaging with Missing Observations. Computational Statistics and Data Analysis, 54, 3336-3347. https://doi.org/10.1016/j.csda.2009.07.023
