Post-shrinkage strategies for nonlinear semiparametric regression models in low and high-dimensional settings
-
S. Ejaz Ahmed
Abstract
This paper considers semiparametric estimation strategies for the nonlinear semiparametric regression model (NSRM) under the sparsity assumption by modifying the Gauss–Newton method for both low- and high-dimensional data scenarios. In the low-dimensional case, coefficients are partitioned into two parts that represent nonzero (strong signals) and sparse coefficients. In the high-dimensional case, a weighted-ridge approach is employed, and coefficients are partitioned into three parts, adding weak signals as well. Shrinkage estimators are then obtained in both cases. More importantly, in this paper, we assume that a nonlinear structure is present in the parametric component of the model, which makes the direct application of penalized least squares to the NSRM impossible. To solve this problem, we employ the iterative Gauss–Newton method to obtain the final NSRM estimators. We provide both theoretical and practical details for the suggested estimators. Asymptotic results are derived for both low- and high-dimensional cases. We conduct an extensive simulation study to evaluate the performance of the estimators in a practical setting. Moreover, we substantiate our findings with data examples from two distinct breast cancer datasets: the Breast Cancer in the United States (BCUS) and Wisconsin datasets. By demonstrating the effectiveness of our introduced estimators in these particular biostatistical contexts, our numerical study provides support for the theoretical efficacy of shrinkage estimators, suggesting their potential relevance to breast cancer research and biostatistical methodologies.
Funding source: Türkiye Bilimsel ve Teknolojik Arastırma Kurumu
Award Identifier / Grant number: 1059B142100231
Acknowledgments
The research of Professor S. Ejaz Ahmed was supported by the Natural Sciences and the Engineering Research Council (NSERC) of Canada.
-
Research ethics: Not applicable.
-
Informed consent: Not applicable.
-
Author contributions: Conceptualization: S.E.A. & D.A., Supervision: S.E.A., Design: E.Y. & D.A., Funding: E.Y., Analysis: D.A. & E.Y., Application and Coding: E.Y. & D.A., Review: S.E.A.
-
Use of Large Language Models, AI and Machine Learning Tools: None declared.
-
Conflict of interest: The authors do not have conflict of interest.
-
Research funding: This paper is funded by the program of Scientific and Technological Research Council of Turkey BIDEB-2214-A with the Project No: 1059B142100231.
-
Data availability: Breast Cancer Wisconsin dataset: https://www.kaggle.com/datasets/uciml/breast-cancer-wisconsin-data Breast Cancer in the United States (BCUS): https://data.world/deviramanan2016/nki-breast-cancer-data.
References
1. Seber, GA, Wild, CJ. Nonlinear regression. Hoboken, New Jersey: John Wiley & Sons; 2003, vol 62:1238 p.Search in Google Scholar
2. Huet, S, Bouvier, A, Poursat, MA, Jolivet, E, Bouvier, AM. Statistical tools for nonlinear regression: a practical guide with S-PLUS and R examples. New York: Springer; 2004:233 p.Search in Google Scholar
3. Han, X, Xie, WF, Fu, Z, Luo, W. Nonlinear systems identification using dynamic multitime scale neural networks. Neurocomputing 2011;74:3428–39. https://doi.org/10.1016/j.neucom.2011.06.007.Search in Google Scholar
4. AdilabdAlkareem, Z, Kalaf, BA. Comparison different estimation methods for the parameters of non-linear regression. Iraqi J Sci 2022;63:1662–80. https://doi.org/10.24996/ijs.2022.63.4.24.Search in Google Scholar
5. Bertoli, W, Oliveira, RP, Achcar, JA. A new semiparametric regression framework for analyzing non-linear data. Analytics 2022;1:15–26. https://doi.org/10.3390/analytics1010002.Search in Google Scholar
6. Espinoza, M, Suykens, JA, De Moor, B. Kernel based partially linear models and nonlinear identification. IEEE Trans Autom Control 2005;50:1602–6. https://doi.org/10.1109/tac.2005.856656.Search in Google Scholar
7. Ahmed, SE, Ahmed, F, Yüzbaşı, B. Post-shrinkage strategies in statistical and machine learning for high dimensional data. CRC Press; 2023.10.1201/9781003170259Search in Google Scholar
8. Gao, X, Ahmed, SE, Feng, Y. Post selection shrinkage estimation for high‐dimensional data analysis. Appl Stochastic Models Bus Ind 2017;33:97–120. https://doi.org/10.1002/asmb.2193.Search in Google Scholar
9. Ahmed, SE. Penalty, shrinkage, and pretest strategies: variable selection and estimation. New York: Springer; 2014:2014 p.10.1007/978-3-319-03149-1Search in Google Scholar
10. Ahmed, SE, Nicol, CJ. An application of shrinkage estimation to the nonlinear regression model. Comput Stat Data Anal 2012;56:3309–21. https://doi.org/10.1016/j.csda.2010.07.022.Search in Google Scholar
11. Yüzbaşı, B, Ahmed, SE, Aydın, D. Ridge-type pretest and shrinkage estimations in partially linear models. Stat Pap 2020;61:869–98. https://doi.org/10.1007/s00362-017-0967-8.Search in Google Scholar
12. Aydın, D, Ahmed, SE, Yılmaz, E. Estimation of semiparametric regression model with right-censored high-dimensional data. J Stat Comput Simul 2019;89:985–1004. https://doi.org/10.1080/00949655.2019.1572757.Search in Google Scholar
13. Ruppert, D, Wand, MP, Carroll, RJ. Semiparametric regression (No. 12). Cambridge: Cambridge University Press; 2003.10.1017/CBO9780511755453Search in Google Scholar
14. Ahmed, SE, Doksum, KA, Hossain, S, You, J. Shrinkage, pretest and absolute penalty estimators in partially linear models. Aust N Z J Stat 2007;49:435–54. https://doi.org/10.1111/j.1467-842x.2007.00493.x.Search in Google Scholar
15. Li, R, Liang, H. Variable selection in semiparametric regression modeling. Ann Stat 2008;36:261. https://doi.org/10.1214/009053607000000604.Search in Google Scholar PubMed PubMed Central
16. Meloun, M, Militký, J. Statistical data analysis: a practical guide. New Delhi: Woodhead Publishing Limited; 2011.10.1533/9780857097200Search in Google Scholar
17. Hurvich, CM, Simonoff, JS, Tsai, CL. Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. J R Stat Soc B 1998;60:271–93.10.1111/1467-9868.00125Search in Google Scholar
Supplementary Material
This article contains supplementary material (https://doi.org/10.1515/ijb-2024-0011).
© 2025 Walter de Gruyter GmbH, Berlin/Boston
