Home Shrinkage estimation applied to a semi-nonparametric regression model
Article
Licensed
Unlicensed Requires Authentication

Shrinkage estimation applied to a semi-nonparametric regression model

  • Hossein Zareamoghaddam EMAIL logo , Syed E. Ahmed and Serge B. Provost
Published/Copyright: August 10, 2020
Become an author with De Gruyter Brill
Submit Manuscript Author Information
The International Journal of Biostatistics
From the journal The International Journal of Biostatistics Volume 17 Issue 1

Abstract

Stein-type shrinkage techniques are applied to the parametric components of a semi-nonparametric regression model recently proposed by (Ma et al. 2015: 285–303). On the basis of an uncertain prior information (restrictions) about the parameters of interest, shrinkage techniques are shown to improve the accuracy of the model. The effectiveness of the proposed estimators are corroborated by a simulation study.

Keywords: local linear regression; multiple simple regression; semi-nonparametric model; shrinkage
Corresponding author: Hossein Zareamoghaddam, Department of Statistical and Actuarial Sciences, The University of Western Ontario, London, Ontario, Canada, E-mail:

Funding source: Natural Sciences and Engineering Research Council of Canada

Acknowledgments

The financial support of the Natural Sciences and Engineering Research Council of Canada is gratefully acknowledged. We would also like to express our sincere thanks to the two reviewers and an editor whose valuable comments and suggestions significantly improved the presentation of our study.

  1. Author contribution: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.

  2. Research funding: This research was funded by Natural Sciences and Engineering Research Council of Canada.

  3. Conflict of interest statement: The authors declare no conflicts of interest regarding this article.

Appendix

The following Lemma will be used to prove the asymptotic results:

Lemma 5.1

. (Judge and Bock [17]) Let thep-vector xhave a Np(μx, Σx)distribution. Then, for a measurable function ofϕ, one has

(27)E[xϕ(x′x)]=μxE[ϕ(χp+22(Δ2))]
(28)E[xx′ϕ(x′x)]=∑xE[ϕ(χp+22(Δ2))]+μxμ′xE[ϕ(χp+42(Δ2))]
whereΔ2=μ′x∑x−1μx.

A1: Proof of Theorem 4.1

It is straightforward to find the bias of β^U, α^U, β^R and α^R by making use of Theorem 3.1. As for the estimates, we have

b(β^S)= E[β^U−β−cLn−1Hβ^U]=−c(n−1)HβE[ϕ(χp+2−2(Δ2))]
b(α^S)= E[α^U−α+cLn−1TnHβ^U]=c(n−1)TnHβE[ϕ(χp+2−2(Δ2))]
b(β^PS)= E[β^R−β+(1−cLn−1)Hβ^UI(Ln>c)]   =−Hβ{Gn+1,m(c1;Δ2)+c1E[Fn+1,m−1(Δ2)I(Fn+1,m(Δ2)>c1)]}
b(α^PS)= E[α^U−α+{1−(1−cLn−1)I(Ln>c)}TnHβ^U]   =c1TnHβ {E[Fn+1,m−1(Δ2)]−E[Fn+1,m−1(Δ2)I(Fn+1,m(Δ2)>c1)]}      +TnHβ Gn+1,m(c1;Δ2)
b(β^PT)= E[β^U−β−Hβ^U I(Ln<Cα)]=−Hβ Gn+1,m(ℓα;Δ2)
b(α^PT)= E[α^U−α+TnHβ^U I(Ln<Cα)]=TnHβ Gn+1,m(ℓα;Δ2)

A2: Proof of Theorem 4.2

Likewise, the MSE of β^U, α^U, β^R and α^R are easily derived from Theorem 3.1. The MSE of other estimators are obtained as follows:

MSE(β^S)= E[(β^U−c Ln−1Hβ^U−β)(β^U−c Ln−1Hβ^U−β)]     = E[(β^U−β)(β^U−β)′]−2cHE[Ln−1β^U(β^U−β)′]      +c2HE[Ln−2β^Uβ^^U]H′     =σ2D22−c(n−1)σ2HD22{2E[χn+1−2(Δ2)]−(n−3)E[χn+1−4(Δ2)]}     +c(n2−1)(Hββ′H′)E[[χn+3−4(Δ2)]
MSE(α^S)= E[(α^U+c Ln−1TnHβ^U−α)(α^U+c Ln−1TnHβ^U−α)′]      = σ2D11−c(n−1)σ2TnHD22H′Tn{2Δ2E[χn+1−2(Δ2)]−(n−3)E[χn+1−4(Δ2)]}    +c(n2−1)(TnHββ’H’Tn’)E[χn+3−4(Δ2)]
MSE(β^PT)= E[(β^U−Hβ^U I(Ln<Cα)−β)(β^U−Hβ^U I(Ln<Cα)−β)′]       = E[(β^U−β)(β^U−β)′]−2HE[β^U I(Ln<Cα)(β^U−β)′] +HE[I(Ln<Cα)β^Uβ^^U]H′  =σ2D22−σ2HD22Gn+1,m(ℓα;Δ2)+Hββ′H′ ×{2Gn+1,m(ℓα;Δ2)−Gn+3,m(ℓα∗;Δ2)}
MSE(α^PT)= E[(α^U+TnHβ^U I(Ln<Cα)−α)(α^U+TnHβ^U I(Ln<Cα)−α)′]      = σ2D11+2TnH E[β^U I(Ln<Cα)(α^U−α)′]     +TnHE[I(Ln<Cα)β^Uβ^’U]H′T′n       = σ2D11−σ2(D11−D11∗)Gn+1,m(ℓα;Δ2)+TnHββ′H′T′n     ×{2Gn+1,m(ℓα;Δ2)−Gn+3,m(ℓα∗;Δ2)}
MSE(β^PS)= E[(β^U−(1−c Ln−1) I(Ln>c)Hβ^U−β)(β^U−(1−c Ln−1) I(Ln>c)Hβ^U−β)′]      = MSE(β^S)−(σ2HD22−2Hββ’H’)E[(1−c1Fn+1,m−1(Δ2)IFn+1,m(Δ2)<c1)]      −Hββ’H’E[(1−c2Fn+3,m−1(Δ2))I(Fn+3,m(Δ2)<c2)]
MSE(α^PS)= E[(α^U+{1−(1−c Ln−1) I(Ln>c)}TnHβ^U−α)      ×(α^U+{1−(1−c Ln−1) I(Ln>c)}TnHβ^U−α)′]      = MSE(α^S)−(σ2TnHD22H′T′n−2TnHββ′H′T′n)E[(1−c1Fn+1,m−1(Δ2))      ×I(Fn+1,m(Δ2)<c1)]−TnHββ′H′T′nE[(1−c2Fn+3,m−1(Δ2))I(Fn+3,m(Δ2)<c2)]

For more detailed information about the derivations of the MSE’s of the pretest and shrinkage estimators, one can refer to [18].

A3: Proof of Theorem 4.3

By making use of the properties of the trace of a matrix and the definition of the risk, that is, R(β^∗;W)=E((β^∗−β)′W(β^∗−β) | M)=tr(WMSE(β^∗)), the risk expressions follow directly from the MSE expressions given in Theorem 4.2.

References

1. Ma, W, Feng, Y, Chen, K, Ying, Z. Functional and parametric estimation in a semi- and nonparametric model with application to mass-spectrometry data. Int J Biostat 2015;11:285–303. https://doi.org/10.1515/ijb-2014-0066.Search in Google Scholar

2. Fan, J, Gijbels, I. Local polynomial modelling and its applications. London: Chapman & Hall; 1996.Search in Google Scholar

3. Robinson, PM. Root-N consistent semiparametric regression. Econometrica 1988;55:931–54. https://doi.org/10.2307/1912705.Search in Google Scholar

4. Stone, CJ. Optimal global rates of convergence for nonparametric regression. Ann Math Stat 1982;10:1040–53. https://doi.org/10.1214/aos/1176345969.Search in Google Scholar

5. Ruppert, D, Wand, M, Carroll, R. Semiparametric regression. New York: Cambridge University Press; 2003.10.1017/CBO9780511755453Search in Google Scholar

6. Begun, J, Hall, WJ, Huang, WM, Wellner, JA. Information and asymptotic efficiency in parametric-nonparametric models. Ann Math Stat 1983;11:432–52. https://doi.org/10.1214/aos/1176346151.Search in Google Scholar

7. Roy, P, Truntzer, C, Maucort-Boulch, D, Jouve, T, Molinari, N. Protein mass spectra data analysis for clinical biomarker discovery: a global review. Briefings Bioinf 2011;12:176–86. https://doi.org/10.1093/bib/bbq019.Search in Google Scholar

8. Yasui, Y, Pepe, M, Thompson, ML, Adam, BL, Wright, GLJr., Qu, Y, et al.. A data-analytic strategy for protein biomarker discovery: profiling of high-dimensional proteomic data for cancer detection. Biostatistics 2003;4:449–63. https://doi.org/10.1093/biostatistics/4.3.449.Search in Google Scholar

9. Guilhaus, M. Principles and instrumentation in time-of-flight mass spectrometry. J Mass Spectrom 1995;30:1519–32. https://doi.org/10.1002/jms.1190301102.Search in Google Scholar

10. Baggerly, KA, Morris, JS, Coombes, KR. Reproducibility of SELDI-TOF protein patterns in serum: comparing datasets from different experiments. Bioinformatics 2004;20:777–85. https://doi.org/10.1093/bioinformatics/btg484.Search in Google Scholar

11. Baggerly, KA, Morris, JS, Wang, J, Gold, D, Xiao, LC, Coombes, KR. A comprehensive approach to the analysis of MALDI-TOF proteomics spectra from serum samples. Proteomics 2003;3:1667–72. https://doi.org/10.1002/pmic.200300522.Search in Google Scholar

12. Ahmed, SE, Krzanowski, WJ. Biased estimation in a simple multivariate regression model. Comput Stat Data Anal 2004;45:689–96. https://doi.org/10.1016/s0167-9473(03)00088-4.Search in Google Scholar

13. Ahmed, SE. Improved estimation in a multivariate regression model. Comput Stat Data Anal 1994;17:537–54. https://doi.org/10.1016/0167-9473(94)90147-3.Search in Google Scholar

14. Chitsaz, S, Ahmed, SE. Shrinkage estimation for the regression parameter matrix in multivariate regression model. J Stat Comput Simulat 2012a;82:309–23. https://doi.org/10.1080/00949655.2011.648938.Search in Google Scholar

15. Chitsaz, S, Ahmed, SE. An improved estimation in regression parameter matrix in multivariate regression model. Commun Stat Theor Methods 2012b;41:2305–20. https://doi.org/10.1080/03610926.2012.664672.Search in Google Scholar

16. Ehsanes Saleh, AK. Theory and preliminary test and Stein-type estimation with applications. New York: Wiley; 2006.10.1002/0471773751Search in Google Scholar

17. Judge, GG, Bock, ME. The statistical implication of pretest and Stein-rule estimators in econometrics. Amsterdam: North Holland; 1978.Search in Google Scholar

18. Ali, AM. Interface of preliminary test approach and empirical Bayes approach to shrinkage estimation, PhD Thesis. Canada: Carleton University; 1990.10.22215/etd/1990-01814Search in Google Scholar

Received: 2018-10-23
Accepted: 2020-05-01
Published Online: 2020-08-10

© 2020 Walter de Gruyter GmbH, Berlin/Boston

Articles in the same Issue

  1. Frontmatter
  2. Research Articles
  3. The method of envelopes to concisely calculate semiparametric efficient scores under parametric restrictions
  4. A machine learning-based approach for estimating and testing associations with multivariate outcomes
  5. Shrinkage estimation applied to a semi-nonparametric regression model
  6. Multivariate quasi-beta regression models for continuous bounded data
  7. Integrative analysis with a system of semiparametric projection non-linear regression models
  8. Seemingly unrelated regression with measurement error: estimation via Markov Chain Monte Carlo and mean field variational Bayes approximation
  9. Alternatives to the Kaplan–Meier estimator of progression-free survival
  10. Two-stage receiver operating-characteristic curve estimator for cohort studies
  11. Estimating the area under a receiver operating characteristic curve using partially ordered sets
  12. Modelling ethnic differences in the distribution of insulin resistance via Bayesian nonparametric processes: an application to the SABRE cohort study
  13. Co-localization analysis in fluorescence microscopy via maximum entropy copula
https://doi.org/10.1515/ijb-2018-0109
Keywords for this article
local linear regression; multiple simple regression; semi-nonparametric model; shrinkage

Articles in the same Issue

  1. Frontmatter
  2. Research Articles
  3. The method of envelopes to concisely calculate semiparametric efficient scores under parametric restrictions
  4. A machine learning-based approach for estimating and testing associations with multivariate outcomes
  5. Shrinkage estimation applied to a semi-nonparametric regression model
  6. Multivariate quasi-beta regression models for continuous bounded data
  7. Integrative analysis with a system of semiparametric projection non-linear regression models
  8. Seemingly unrelated regression with measurement error: estimation via Markov Chain Monte Carlo and mean field variational Bayes approximation
  9. Alternatives to the Kaplan–Meier estimator of progression-free survival
  10. Two-stage receiver operating-characteristic curve estimator for cohort studies
  11. Estimating the area under a receiver operating characteristic curve using partially ordered sets
  12. Modelling ethnic differences in the distribution of insulin resistance via Bayesian nonparametric processes: an application to the SABRE cohort study
  13. Co-localization analysis in fluorescence microscopy via maximum entropy copula
Sign up now to receive a 20% welcome discount
Institutional Access
How does access work?
Have an idea on how to improve our website?
Please write us.
© 2025 De Gruyter Brill
Downloaded on 18.10.2025 from https://www.degruyterbrill.com/document/doi/10.1515/ijb-2018-0109/html
Scroll to top button