Bayesian Subset Selection for Reproductive Dispersion Linear Models

1 Introduction

Reproductive dispersion distribution family proposed by Jorgensen[1], which is a general distribution family and includes normal distribution, Gumbel distribution and exponential family distribution as its special case, has been received a lot of attention in recent years. For example, Tang et al.[2] proposed a nonlinear reproductive dispersion model and studied its influence diagnostics; Chen et al.[3] discussed influence diagnostics of semiparametric nonlinear reproductive dispersion models; Fu et al.[4] investigated local influence analysis for nonlinear structural equation models with non-ignorable missing outcomes from a reproductive dispersion model. In particular, Chen and Tang[5] developed a Bayesian approach to analyze semiparametric reproductive dispersion mixed-effects models on the basis of P-spline estimates of nonparametric components by using the Gibbs sampler[6] and the Metropolis-Hastings algorithm[7–8]. Recently, Tang and Zhao[9] presented semiparametric Bayesian analysis of nonlinear reproductive dispersion mixed models for longitudinal data.

Various Bayesian methods have been proposed for selecting suitable subset models, those classical methods include Bayeisan information criterion[10], asymptotic information criterion[11], Bayes factor[12] and pseudo-Bayes factor[13], etc. In particular, Mitchell and Beauchamp[14] presented a Bayesian variable selection method assuming the prior distribution of each regression coefficient is a mixture of a point mass at 0 and a diffuse uniform distribution elsewhere. Furthermore, George and McCulloch[15] developed a variable selection procedure via Gibbs sampling. The reversible jump MCMC[16] was proposed for investigating model uncertainty. Nott and Leonte[17] proposed sampling schemes for Bayesian selection in generalized linear models based on the Swendsen-Wang algorithm for the Ising model[18]. Hans[19] addressed model uncertainty for the Bayesian lasso regression by computing the marginal posterior probabilities for small model space. Recently, Lykou and Ntzoufras[20] studied how to select variables using Bayesian lasso. However, to our knowledge, there is little work done on Bayesian subset selection for reproductive dispersion distribution family. Motivated by the idea of Kuo and Mallick[21], the main purpose of this paper is to develop a novel Bayesian approach to select promising subsets under reproductive dispersion linear models by using MCMC technology.

This article is structured as follows. A reproductive dispersion linear model which includes model uncertainty is defined in Section 2. In Section 3, we explicitly describe a MCMC procedure that can identify the promising subsets of predictors under reproductive dispersion linear models. Several numerical examples are used to illustrate our proposed methodology in Section 4. A discussion is given in the final section. Technical details are given in the Appendix.

2 Model and notation

Suppose y_i (i = 1, 2, ⋯, n) is the observation for the ith individual, we assume that y₁, y₂, ⋯, y_n are independently distributed and has the following probability density function

where μ_i is the location parameter and may represent the mean of y_i, σ² ≜ τ⁻¹ ∈ 𝔹(𝔹⊂ℝ⁺) is referred to as the dispersion parameter which is known or can be estimated separately; a(⋅;⋅) is some suitable known function; d(y; μ) is a unit deviance on ℂ×Ω (Ω ⊆ ℂ ⊆ ℝ are open intervals) and satisfies d(y; y) = 0, ∀y ∈ ℂ and d(y; μ) > 0, ∀ y ≠ μ, and is twice continuously differentiable with respect to (y, μ) on ℂ × Ω. The model (2.1) is referred to as a reproductive dispersion distributional family[1] and includes normal distribution, double exponential distribution, Gumbel distribution and exponential family distribution as its special case. Motivated by the idea of Kuo and Mallick[21], we explore a simple method of subset selection. Instead of building a hierarchical model, we embed indicator variable in predictors that incorporates all 2^p submodels. Specifically, let γ_j (j = 1, 2, ⋯, p) be the indicator variable supported at two points 1 and 0. When γ_j = 1 we include the predictor in the predictor model. When γ_j = 0, we omit the predictor when building the model. Therefore we assume that the systematic part of the model is specified by

where g(⋅) is a known continuously differentiable link function. For simplicity, we only consider the case that g(⋅) is the canonical link in this article, i.e., g(μ_i) = μ_i, then (2.2) reduces to

The model defined by equations (2.1) and (2.3) is referred to as a reproductive dispersion linear model which includes model uncertainty.

Let β = (β₁, β₂, ⋯, β_p)^T, γ = (γ₁, γ₂, ⋯, γ_p)^T, θ = (β^T, τ)^T and x_i =(x_i1, x_i2, ⋯, x_ip)^T. We assume that the prior of θ is independent on γ and γ_j is mutually independent for j = 1, 2, ⋯, p. Under the above assumption, the prior density p(θ, γ) of (θ, γ) can be expressed as

Based on the recommendations given by Chen and Tang[5] and Kuo and Mallick[21], we consider the following prior distributions for parameters τ, β and γ_j(j = 1, 2, ⋯, p):

where a₁, a₂, β₀, H₀ and p_j are hyperparameters whose values are assumed to be given, Γ (a, b) denotes the gamma distribution with parameters a and b, N(β, Σ) denotes the multivariate normal distribution with mean β and covariance Σ, B(1, q) is the Bernoulli distribution with parameter q. In this paper, we set a₁ = a₂ = 0.01, β₀ = 0, H₀ = 100I_p and p_j = 0.5 in all numerical examples which may present noninformation priors, where I_p is the p × p identity matrix.

3 Bayesian subset selection

In this section, we propose and develop a MCMC procedure that can identify the promising subsets of predictors under reproductive dispersion linear models. Denote y = {y₁, y₂, ⋯, y_n}, x = {x₁, x₂, ⋯, x_n}. The Bayesian inference on θ and γ can be via the posterior distribution log p(θ, γ|y, x)∝ log p(y, x|θ,γ) + log p(θ, γ), where p(y, x|θ, γ) is the likelihood function of the observed data {y, x}. The likelihood p(y, x|θ, γ) involves intractable multiple high dimension integrals. Thus, it is rather difficult and tedious to work directly with this posterior density. Inspired by recent development in statistical computing, the Gibbs sampler[6] is used to generate a sequence of random observations from the joint posterior distribution p(θ, γ|y, x), which is proportional to ∏i=1np(y_i, x_i|θ, γ)p(θ, γ), i.e.,

In this algorithm, observations {θ, γ} are iteratively simulated from the following conditional distributions: p(θ|y, x, γ) and p(γ|y, x, θ). Then we identify the promising subset of predictors with high posterior probabilities of γ.

4 Numerical examples

In this section, several numerical examples are used to illustrate the preceding proposed method.

5 Conclusion and discussion

In this article, we extend the Bayesian variable selection methods proposed in [21] to analyze reproductive dispersion linear models which include model uncertainty. A full Bayesian approach is developed to investigate these models’ uncertainty by using Gibbs sampler and MH algorithm. Several numerical examples are used show the efficiency of the proposed Bayesian approach. The results show that the developed Bayesian procedure is highly efficient and computationally fast. Other Bayesian variable selection approaches under reproductive dispersion distribution family will be of great research interest. The authors intend to investigate it in the near future.