Home Comparing Ridge Regression Estimators: Exploring Both New and Old Methods
Article
Licensed
Unlicensed Requires Authentication

Comparing Ridge Regression Estimators: Exploring Both New and Old Methods

  • R. Lakshmi EMAIL logo and T. A. Sajesh ORCID logo
Published/Copyright: February 10, 2025
Become an author with De Gruyter Brill

Abstract

Ridge regression presents a method to tackle multicollinearity issues. Several estimators and predictors for the estimation of biasing parameter k have been extensively detailed in scholarly literature. We offer a thorough analysis of both conventional and emerging methods aimed at precisely determining the ridge parameter k. Our investigation provides valuable insights into the properties of these estimators and their practical efficacy in various applications. Proposed estimators for the parameter k are assessed using Monte Carlo simulations and a real-world example, with a focus on evaluating their performance based on Mean Squared Error (MSE). Our estimator, in conjunction with others, showcases commendable performance, as indicated by the results.

MSC 2020: 62J07; 62J05; 97K80

References

[1] S. Ali, H. Khan, I. Shah, M. M. Butt and M. Suhail, A comparison of some new and old robust ridge regression estimators, Comm. Statist. Simulation Comput. 50 (2021), no. 8, 2213–2231. 10.1080/03610918.2019.1597119Search in Google Scholar

[2] M. Alkhamisi, G. Khalaf and G. Shukur, Some modifications for choosing ridge parameters, Comm. Statist. Theory Methods 35 (2006), no. 10–12, 2005–2020. 10.1080/03610920600762905Search in Google Scholar

[3] M. A. Alkhamisi and G. Shukur, Developing ridge parameters for SUR model, Comm. Statist. Theory Methods 37 (2008), no. 3–5, 544–564. 10.1080/03610920701469152Search in Google Scholar

[4] Y. Asar, A. Karaibrahimoğlu and A. Genç, Modified ridge regression parameters: A comparative Monte Carlo study, Hacet. J. Math. Stat. 43 (2014), no. 5, 827–841. Search in Google Scholar

[5] S. Chand and B. M. G. Kibria, A new ridge type estimator and its performance for the linear regression model: Simulation and application, Hacet. J. Math. Stat. 53 (2024), no. 3, 837–850. 10.15672/hujms.1359446Search in Google Scholar

[6] S. Chatterjee and A. S. Hadi, Regression Analysis by Example, John Wiley and Sons, New York, 2013. Search in Google Scholar

[7] C. Daniel and F. S. Wood, Fitting Equations to Data: Computer Analysis of Multifactor Data, John Wiley and Sons, New York, 1980. Search in Google Scholar

[8] I. S. Dar, S. Chand, M. Shabbir and B. M. G. Kibria, Condition-index based new ridge regression estimator for linear regression model with multicollinearity, Kuwait J. Sci. 50 (2023), no. 2, 91–96. 10.1016/j.kjs.2023.02.013Search in Google Scholar

[9] I. Dawoud, A. F. Lukman and A. R. Haadi, A new biased regression estimator: Theory, simulation and application, Sci. African 15 (2022), Article ID e01100. 10.1016/j.sciaf.2022.e01100Search in Google Scholar

[10] A. V. Dorugade, Improved ridge estimator in linear regression with multicollinearity, heteroscedastic errors and outliers, J. Mod. Appl. Statist. Methods 15 (2016), 362–381. 10.22237/jmasm/1478002860Search in Google Scholar

[11] L. Firinguetti, G. Kibria and R. Araya, Study of partial least squares and ridge regression methods, Comm. Statist. Simulation Comput. 46 (2017), no. 8, 6631–6644. 10.1080/03610918.2016.1210168Search in Google Scholar

[12] D. G. Gibbons, A simulation study of some ridge estimators, J. Amer. Statist. Assoc. 76 (1981), no. 373, 131–139. 10.1080/01621459.1981.10477619Search in Google Scholar

[13] R. R. Hocking, F. M. Speed and M. J. Lynn, A class of biased estimators in linear regression, Technometrics 18 (1976), no. 4, 425–437. 10.1080/00401706.1976.10489474Search in Google Scholar

[14] A. E. Hoerl, R. W. Kannard and K. F. Baldwin, Ridge regression: Some simulations, Comm. Statist. Theory Methods 4 (1975), no. 2, 105–123. 10.1080/03610927508827232Search in Google Scholar

[15] A. E. Hoerl and R. W. Kennard, Ridge regression—1980. Advances, algorithms, and applications, Amer. J. Math. Manag. Sci. 1 (1981), no. 1, 5–83. 10.1080/01966324.1981.10737061Search in Google Scholar

[16] S. Kaçiranlar, S. Sakallioglu, F. Akdeniz, G. P. H. Styan and H. J. Werner, A new biased estimator in linear regression and a detailed analysis of the widely-analysed dataset on Portland cement, Sankhya B 61 (1999), no. 3, 443–459. Search in Google Scholar

[17] G. Khalaf, A proposed ridge parameter to improve the least square estimator, J. Mod. Appl. Statist. Methods 11 (2012), no. 2, Paper No. 15. 10.22237/jmasm/1351743240Search in Google Scholar

[18] G. Khalaf, K. Månsson and G. Shukur, Modified ridge regression estimators, Comm. Statist. Theory Methods 42 (2013), no. 8, 1476–1487. 10.1080/03610926.2011.593285Search in Google Scholar

[19] G. Khalaf and G. Shukur, Choosing ridge parameter for regression problems, Comm. Statist. Theory Methods 34 (2005), no. 5, 1177–1182. 10.1081/STA-200056836Search in Google Scholar

[20] B. M. G. Kibria, Performance of some new ridge regression estimators, Comm. Statist. Simulation Comput. 32 (2003), no. 2, 419–435. 10.1081/SAC-120017499Search in Google Scholar

[21] B. M. G. Kibria, More than hundred (100) estimators for estimating the shrinkage parameter in a linear and generalized linear ridge regression models, J. Econ. Statist. 2 (2022), no. 2, 233–252. Search in Google Scholar

[22] J. F. Lawless and P. Wang, A simulation study of ridge and other regression estimators, Comm. Statist. Theory Methods 5 (1976), no. 4, 307–323. 10.1080/03610927608827353Search in Google Scholar

[23] K. Liu, Using Liu-type estimator to combat collinearity, Comm. Statist. Theory Methods 32 (2003), no. 5, 1009–1020. 10.1081/STA-120019959Search in Google Scholar

[24] A. F. Lukman and A. Olatunji, Newly proposed estimator for ridge parameter: An application to the Nigerian economy, Pakistan J. Statist. 34 (2018), no. 2, 91–98. Search in Google Scholar

[25] S. Mermi, A. Göktaş and Ö. Akkuş, Are most proposed ridge parameter estimators skewed and do they have any effect on MSE values?, J. Stat. Comput. Simul. 91 (2021), no. 10, 2074–2093. 10.1080/00949655.2021.1883611Search in Google Scholar

[26] G. Muniz and B. M. G. Kibria, On some ridge regression estimators: An empirical comparisons, Comm. Statist. Simulation Comput. 38 (2009), no. 3–5, 621–630. 10.1080/03610910802592838Search in Google Scholar

[27] S. Singh and D. S. Tracy, Ridge-regression using scrambled responses, Metrika 41 (1999), no. 2, 147–157. Search in Google Scholar

[28] M. Suhail, S. Chand and B. M. G. Kibria, Quantile based estimation of biasing parameters in ridge regression model, Comm. Statist. Simulation Comput. 49 (2020), no. 10, 2732–2744. 10.1080/03610918.2018.1530782Search in Google Scholar

[29] H. Woods, H. H. Steinour and H. R. Starke, Effect of composition of Portland cement on heat evolved during hardening, Indust. Eng. Chem. 24 (1932), no. 11, 1207–1214. 10.1021/ie50275a002Search in Google Scholar

Received: 2024-11-05
Revised: 2025-01-03
Accepted: 2025-01-03
Published Online: 2025-02-10
Published in Print: 2025-05-01

© 2025 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 14.9.2025 from https://www.degruyterbrill.com/document/doi/10.1515/eqc-2024-0043/html?lang=en
Scroll to top button