In the presence of multicollinearity, ridge regression techniques result in estimated coefficients that are biased but have smaller variance than Ordinary Least Squares estimators and may, therefore, have a smaller Mean Squares Error (MSE). The ridge solution is to supplement the data by stochastically shrinking the estimates toward zero. In this study, we propose a new estimator to reduce the effect of multicollinearity and improve the estimation. We show by a simulation study that the MSE of the suggested estimator is lower than other estimators of the ridge and the OLS estimators.
References
[1]
Montgomery, D.C. (1982) Economic Design of an X Control Chart. Journal of Quality Technology, 14, 40-43. https://doi.org/10.1080/00224065.1982.11978782
[2]
Montgomery, D.C., Peck, E.A. and Vining, G.G. (2006) Introduction to Linear Regression Analysis. John Wiley & Sons, Hoboken.
[3]
Hoerl, A.E. and Kennard, R.W. (1970) Ridge Regression: Biased Estimation for Non-Orthogonal Problems. Technometrics, 12, 55-67.
https://doi.org/10.1080/00401706.1970.10488634
[4]
Hoerl, A.E., Kennard, R.W. and Baldwin, K.F. (1975) Ridge Regression: Some Simulation. Communications in Statistics—Theory and Methods, 4, 105-124.
https://doi.org/10.1080/03610917508548342
[5]
Gibbons, D.G. (1981) A Simulation Study of Some Ridge Estimators. Journal of the American Statistical Association, 76, 131-139.
https://doi.org/10.1080/01621459.1981.10477619
[6]
Saleh, A.K. and Kibria, B.M. (1993) Performance of Some New Preliminary Test Ridge Regression Estimators and Their Properties. Communications in Statistics—Theory and Methods, 22, 2747-2764.
https://doi.org/10.1080/03610929308831183
[7]
Kibria, B.M.G. (2003) Performance of Some New Ridge Regression Estimators. Communications in Statistics—Theory and Methods, 32, 419-435.
https://doi.org/10.1081/SAC-120017499
[8]
Khalaf, G. and Shukur, G. (2005) Choosing Ridge Parameters for Regression Problems. Communications in Statistics—Theory and Methods, 34, 1177-1182.
https://doi.org/10.1081/STA-200056836
[9]
Dorugade, A.V. and Kashid, D.N. (2010) Alternative Method for Choosing Ridge Parameter for Regression. International Journal of Applied Mathematical Sciences, 4, 447-456.
[10]
Khalaf, G. (2013) A Comparison between Biased and Unbiased Estimators. Journal of Modern Applied Statistical Methods, 12, 293-303.
https://doi.org/10.22237/jmasm/1383279360
[11]
Khalaf, G. and Iguernane, M. (2014) Ridge Regression and Ill-Conditioning. Journal of Modren Applied Statistical Methods, 13, 355-363.
https://doi.org/10.22237/jmasm/1414815420
[12]
Fujii, K. (2018) Least Squares Method from the View Point of Deep Learning. Advances in Pure Mathematics, 8, 485-493. https://doi.org/10.4236/apm.2018.85027
[13]
Yuzbasi, B., Arashi, M. and Ejaz Ahmed, S. (2020) Shrinkage Estimation Strategies in Generalized Ridge Regression Models: Low/High-Dimension Regime. International Statistical Review, 88, 229-251. https://doi.org/10.1111/insr.12351
[14]
Tsagris, M. and Pandis, N. (2021) Multicollinearity. American Journal of Orthodontics and Dentofacial Orthopedics, 159, 695-696.
https://doi.org/10.1016/j.ajodo.2021.02.005
