In this paper, we propose a Smooth Quantile Boost Classification (SQBC) algorithm for binary classification problem. The SQBC algorithm directly uses a smooth function to approximate the “check function” of the quantile regression. Compared to other boost-based classification algorithms, the proposed algorithm is more accurate, flexible and robust to noisy predictors. Furthermore, the SQBC algorithm also can work well in high dimensional space. Extensive numerical experiments show that our proposed method has better performance on randomly simulations and real data.
References
[1]
Freund, Y. and Schapire, R. (1997) A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting. Journal of Computer and System Sciences, 55, 119-139. http://dx.doi.org/10.1006/jcss.1997.1504
[2]
Friedman, J., Hastie, T. and Tibshirani, R. (2000) Additive Logistic Regression: A Statistical View of Boosting. Annals of Statistics, 28, 337-407. http://dx.doi.org/10.1214/aos/1016218223
[3]
Friedman, J. (2001) Greedy Function Approximation: A Gradient Boosting Machine. Annals of Statistics, 29, 1189-1232. http://dx.doi.org/10.1214/aos/1013203451
[4]
Mason, L., Baxter, J., Bartlett, P.L. and Frean, M. (2000) Boosting Algorithms as Gradient Descent. Proceedings of Advances in Neural Information Processing Systems (NIPS), 8, 479-485.
[5]
Bühlmann, P. and Yu, B. (2003) Boosting with the L2-Loss: Regression and Classification. Journal of the American Statistical Association, 98, 324-340. http://dx.doi.org/10.1198/016214503000125
[6]
Kriegler, B. and Berk, R. (2007) Boosting the Quantile Distribution: A Cost-Sensitive Statistical Learning Procedure. Technical Report, Department of Statitics, University of California, Los Angeles.
[7]
Zheng, S.F. (2010) Boosting Based Conditional Quantile Estimation for Regression and Binary Classification. Advances in Soft Computing, 6438, 67-79. http://dx.doi.org/10.1007/978-3-642-16773-7_6
[8]
Zheng, S.F. (2011) Gradient Descent Algorithms for Quantile Regression with Smooth Approximation. International Journal of Machine Learning and Cybernetics, 2, 191-207. http://dx.doi.org/10.1007/s13042-011-0031-2
[9]
Koenker, R. and Bassett, G. (1978) Regression Quantiles. Econometrica, 46, 33-50. http://dx.doi.org/10.2307/1913643
[10]
Koenker, R. (2005) Quantile Regression. Cambridge University Press, New York. http://dx.doi.org/10.1017/CBO9780511754098
[11]
Hunter, D.R. and Lange, K. (2000) Quantile Regression via an MM Algorithm. Journal of Computational and Graphical Statistics, 9, 60-77.
[12]
Chen, C. and Mangasarian, O.L. (1996) A Class of Smoothing Functions for Nonlinear and Mixed Complementarity Problems. Computational Optimization and Applications, 5, 97-138. http://dx.doi.org/10.1007/BF00249052
[13]
Kordas, G. (2006) Smoothed Binary Regression Quantiles. Journal of Applied Econometrics, 21, 387-407. http://dx.doi.org/10.1002/jae.843
[14]
Dettling, M. and Bühlmann, P. (2003) Boosting for Tumor Classification with Gene Expression Data. Bioinformatics, 19, 1061-1069. http://dx.doi.org/10.1093/bioinformatics/btf867
[15]
Bühlmann, P. and Hothorn, T. (2007) Boosting Algorithms: Regularization, Prediction and Model Fitting. Statistical Science, 22, 477-505. http://dx.doi.org/10.1214/07-STS242
[16]
Manski, C.F. (1985) Semiparametric Analysis of Discrete Response: Asymptotic Properties of the Maximum Score Estimator. Journal of Economics, 27, 313-333. http://dx.doi.org/10.1016/0304-4076(85)90009-0
[17]
Chen, T.Q. and Guestrin, C. (2016) XGBoost: A Scalable Tree Boosting System. arXiv:1603.02754v3.
[18]
Pierdzioch, C., Risse, M. and Rohloff, S. (2015) A Quantile-Boosting Approach to Forecasting Gold Returns. North American Journal of Economics Finance, 35, 38-55. http://dx.doi.org/10.1016/j.najef.2015.10.015
