In the presence of multicollinearity, ridge regression techniques result in estimated coefficients that are biased but have smaller variance than Ordinary Least Squares estimators and may, therefore, have a smaller Mean Squares Error (MSE). The ridge solution is to supplement the data by stochastically shrinking the estimates toward zero. In this study, we propose a new estimator to reduce the effect of multicollinearity and improve the estimation. We show by a simulation study that the MSE of the suggested estimator is lower than other estimators of the ridge and the OLS estimators.
Cite this paper
Khalaf, G. (2022). Improving the Ordinary Least Squares Estimator by Ridge Regression. Open Access Library Journal, 9, e8738. doi: http://dx.doi.org/10.4236/oalib.1108738.
Montgomery, D.C. (1982) Economic Design of an X Control Chart. Journal of Quality Technology, 14, 40-43. https://doi.org/10.1080/00224065.1982.11978782
Hoerl, A.E., Kennard, R.W. and Baldwin, K.F. (1975) Ridge Regression: Some Simulation. Communications in Statistics—Theory and Methods, 4, 105-124.
https://doi.org/10.1080/03610917508548342
Gibbons, D.G. (1981) A Simulation Study of Some Ridge Estimators. Journal of the American Statistical Association, 76, 131-139.
https://doi.org/10.1080/01621459.1981.10477619
Saleh, A.K. and Kibria, B.M. (1993) Performance of Some New Preliminary Test Ridge Regression Estimators and Their Properties. Communications in Statistics—Theory and Methods, 22, 2747-2764.
https://doi.org/10.1080/03610929308831183
Kibria, B.M.G. (2003) Performance of Some New Ridge Regression Estimators. Communications in Statistics—Theory and Methods, 32, 419-435.
https://doi.org/10.1081/SAC-120017499
Khalaf, G. and Shukur, G. (2005) Choosing Ridge Parameters for Regression Problems. Communications in Statistics—Theory and Methods, 34, 1177-1182.
https://doi.org/10.1081/STA-200056836
Dorugade, A.V. and Kashid, D.N. (2010) Alternative Method for Choosing Ridge Parameter for Regression. International Journal of Applied Mathematical Sciences, 4, 447-456.
Khalaf, G. (2013) A Comparison between Biased and Unbiased Estimators. Journal of Modern Applied Statistical Methods, 12, 293-303.
https://doi.org/10.22237/jmasm/1383279360
Khalaf, G. and Iguernane, M. (2014) Ridge Regression and Ill-Conditioning. Journal of Modren Applied Statistical Methods, 13, 355-363.
https://doi.org/10.22237/jmasm/1414815420
Fujii, K. (2018) Least Squares Method from the View Point of Deep Learning. Advances in Pure Mathematics, 8, 485-493. https://doi.org/10.4236/apm.2018.85027
Yuzbasi, B., Arashi, M. and Ejaz Ahmed, S. (2020) Shrinkage Estimation Strategies in Generalized Ridge Regression Models: Low/High-Dimension Regime. International Statistical Review, 88, 229-251. https://doi.org/10.1111/insr.12351
Tsagris, M. and Pandis, N. (2021) Multicollinearity. American Journal of Orthodontics and Dentofacial Orthopedics, 159, 695-696.
https://doi.org/10.1016/j.ajodo.2021.02.005
