Some remarks on a pair of seemingly unrelated regression models

Definition 1.1

The vector ψ in (1.6) is said to be predictable under ℒ if there exists a matrix L ∈ ℝ^k×n such that E(Ly − ψ) = 0. In particular, the Gβ is said to be estimable under ℒ if there exists a matrix L ∈ ℝ^k×n such that E(Ly − Gβ) = 0.

Definition 1.2

Let ψ be defined in (1.6). If there exists a matrix L such that

holds in the Löwner partial ordering, the linear statistic Ly is defined to be the best linear unbiased predictor(BLUP) of ψ under ℒ, and is denoted by

If H = 0, or G = 0 in (1.6), then the Ly satisfying (1.10) is called the best linear unbiased estimator(BLUE) and the BLUP of Gβ and Hε under ℒ, respectively, and are denoted by

respectively.

BLUPs/BLUEs are well known objects of study in regression analysis because of their simple and optimality properties in statistical inferences, and are one of the prominent research objects in the field of statistics and applications. Because the BLUPs of ψ_i under ℒ₁ and ℒ₂, and the BLUPs of ψ_i under ℒ are not necessarily the same, it is natural to compare the BLUPs under these models, and establish possible connections for the BLUPs, such as,

This article aims at establishing necessary and sufficient conditions for the equalities to hold, and presents some consequences and applications of these equalities.

Lemma 2.1

([11]). Let A ∈ ℝ^m×n, B ∈ ℝ^m×k, C ∈ ℝ^l×n and D ∈ ℝ^l×k. Then

If ℛ(B) ⊆ ℛ(A) and ℛ(C′) ⊆ ℛ(A′), then

In addition, the following results hold.

1. r[A, B] = r(A) ⇔ ℛ(B) ⊆ ℛ(A) ⇔ AA⁺B = B ⇔ E_AB = 0.

2. rAC = r(A) ⇔ ℛ(C′) ⊆ ℛ(A′) ⇔ CA⁺A = C ⇔ C F_A = 0.

Lemma 2.2

([12]). The linear matrix equation AX = B is solvable for X if and only if r[A, B] = r(A), or equivalently, AA⁺B = B. In this case, the general solution of the equation can be written as X = A⁺ B + (I − A⁺A) U, where U is an arbitrary matrix.

In order to directly solve the matrix minimization problem in (1.10), we need the following known result.

Lemma 2.3

([9]). Let

where A ∈ ℝ^p×q, B ∈ ℝ^n×q, C ∈ ℝ^p×m and D ∈ ℝ^n×m are given, M ∈ ℝ^m×m is nnd, and the matrix equation LA = B is solvable. Then there always exists a solution L₀ of L₀A = B such that

holds for all solutions of LA = B. In this case, the matrix L₀ satisfying the above inequality is determined by the following solvable matrix equation

In this case, the general expression of L₀ and the corresponding f(L₀) and f(L) are given by

where F = BA⁺C + D, T = (A^⊥CMC′A^⊥)⁺, and U ∈ ℝ^n×p is arbitrary.

Theorem 3.1

Let ℒ₁ and ℒ₂ be as given in (1.1) and (1.2), respectively, and denote

Then, the parameter vectors ψ_i in (1.5) are predictable by y_i in ℒ₁ and ℒ₂, respectively, if and only if

In these cases,

The matrix equations in (3.6) are solvable under (3.5), and the general solutions L̂_i and the corresponding BLUP_ℒi(ψ_i) can be written as

where U_i ∈ ℝ^k_i×n_i are arbitrary, i = 1, 2. The corresponding f_i(L̂_i) and f_i(L_i) under (3.1)–(3.3) are given by

for i = 1, 2. Further, the following results hold.

1. r[X_i, Σ_ii Xi⊥ ] = r[X_i, Σ_ii], ℛ[X_i, Σ_ii Xi⊥ ] = ℛ[X_i, Σ_ii] and ℛ(X_i) ∩ ℛ(Σ_ii Xi⊥ ) = {0}, i = 1, 2.

2. L̂_i are unique if and only if r[X_i, Σ_ii] = n_i, i = 1, 2.

3. BLUP_ℒi(ψ_i) are unique with probability 1 if and only if y_i ∈ ℛ[X_i, Σ_ii] hold with probability 1, i = 1, 2.

4. The covariance matrices of BLUP_ℒi(ψ_i), as well as the covariance matrices between BLUP_ℒi(ψ_i) and ψ_i are unique, and satisfy the formulas

   Cov[BLUPLi(ψi)]=[Gi,CiXi⊥][Xi,ΣiiXi⊥]+Σii[Gi,CiXi⊥][Xi,ΣiiXi⊥]+′, (3.10)

   Cov{BLUPLi(ψi),ψi}=[Gi,CiXi⊥][Xi,ΣiiXi⊥]+Ci′, (3.11)

   Cov(ψi)−Cov[BLUPLi(ψi)]=HiTiΣ(HiTi)′−[Gi,CiXi⊥][Xi,ΣiiXi⊥]+Σii[Gi,CiXi⊥][Xi,ΣiiXi⊥]+′ (3.12)

   for i = 1, 2.

5. The BLUPs of ψ_i can be decomposed as the sums

   BLUPLi(ψi)=BLUELi(Giβi)+BLUPLi(Hiεi),i=1,2. (3.13)

6. If ψ₁ and ψ₂ are predictable under ℒ₁ and ℒ₂, respectively, then P₁ψ₁ and P₂ψ₂ are predictable under ℒ₁ and ℒ₂, respectively, and BLUP_ℒi(P_iψ_i) = P_iBLUP_ℒi(ψ_i) hold for any matrices P_i ∈ ℝ^t_i×k_i, i = 1, 2.

Proof

It can be seen from (1.1), (1.2), and (1.5) that

From Lemma 2.2, the matrix equations are solvable respectively if and only if (3.5) hold. In these cases, we see from Lemma 2.2 that the first parts of (3.6) are equivalent to finding solutions L̂_i of the solvable matrix equations L̂_iX_i = G_i such that

hold in the Löwner partial ordering. Further from Lemma 2.3, there always exist solutions L̂_i of L̂_iX_i = G_i such that (3.14) hold, and the L̂_i are determined by the matrix equations

thus establishing the matrix equations in (3.6). Solving the matrix equations by Lemma 2.2 gives the L̂_i in (3.7). Also from (3.3),

as required for (3.8) for i = 1, 2. Eq. (3.9) follows from Lemma 2.3.

Result (a) is well known. Results (b) and (c) follow directly from (3.7). Taking covariance matrices of (3.7) yields (3.10). From (3.4) and (3.7),

thus establishing (3.11) for i = 1, 2. The two equalities in (3.12) follow from (1.8) and (3.10). Results (e) and (f) are direct consequences of (3.7).□

We next derive the BLUPs of ψ and ψ_i under ℒ, respectively. Note from ℒ, (1.5) and (1.6) that

Then the expectations and covariance matrices of Ky − ψ and K_iy − ψ_i can be written as

for i = 1, 2. Our second main result is presented below.

Theorem 3.2

Let ℒ be as given in (1.4), and denote

Then, the parameter vector ψ in (1.6) is predictable by y in ℒ if and only if

In this case,

The matrix equation in (3.20) is solvable as well under (3.19), while the general forms of K̂ and the corresponding BLUP_ℒ(ψ) can be written as

where U ∈ ℝ^k×n is arbitrary. The corresponding g(K̂) and g(K) in (3.16) are given by

In particular, the parameter vectors ψ_i in (1.5) is predictable by y in ℒ if and only if

In this case,

The matrix equation in (3.25) is solvable as well under (3.24), while the general forms of K̂_i and the corresponding BLUP_ℒ(ψ_i) can be written as

where U_i ∈ ℝ^k_i×n are arbitrary, i = 1, 2. The corresponding g_i(K̂_i) and g_i(K_i) in (3.17) are given by

for i = 1, 2. Further, the following results hold.

1. r[X, ΣX^⊥] = r[X, Σ], ℛ[X, ΣX^⊥] = ℛ[X, Σ], and ℛ(X) ∩ ℛ(ΣX^⊥) = {0}.

2. K̂ is unique if and only if r[X, Σ] = n.

3. BLUP_ℒ(ψ) is unique with probability 1 if and only if y ∈ ℛ[X, Σ] holds with probability 1.

4. The following covariance matrix formulas

   Cov[BLUPL(ψ)]=[G,JX⊥][X,ΣX⊥]+Σ[G,JX⊥][X,ΣX⊥]+′,Cov{BLUPL(ψ),ψ}=[G,JX⊥][X,ΣX⊥]+J′,Cov(ψ)−Cov[BLUPL(ψ)]=HΣH′−[G,JX⊥][X,ΣX⊥]+Σ[G,JX⊥][X,ΣX⊥]+′,

   and

   Cov[BLUPL(ψi)]=[GiSi,JiX⊥][X,ΣX⊥]+Σ[GiSi,JiX⊥][X,ΣX⊥]+′,Cov{BLUPL(ψi),ψi}=[GiSi,JiX⊥][X,ΣX⊥]+Ji′,Cov(ψi)−Cov[BLUPL(ψi)]=HiTiΣ(HiTi)′−[GiSi,JiX⊥][X,ΣX⊥]+Σ[GiSi,JiX⊥][X,ΣX⊥]+′

   hold for i = 1, 2.

5. The BLUPs of ψ and ψ_i satisfy the following identities

   BLUPL(ψ)=BLUEL(Gβ)+BLUPL(Hε),BLUPL(ψi)=BLUEL(Giβi)+BLUPL(Hiεi),i=1,2.

6. Tψ is predictable under ℒ, then BLUP_ℒ(Tψ) = TBLUP_ℒ(ψ) holds for all matrices T ∈ ℝ^t×k.

Proof

It is obvious from (3.15) that

From Lemma 2.2, the matrix equations are solvable respectively if and only if (3.19) and (3.24) hold, respectively. In these cases, we see from Lemma 2.2 that the first parts of (3.20) and (3.25) are equivalent to finding solutions K̂ of the solvable matrix equations K̂X = G and K̂_i of the solvable matrix equations K̂_iX = G_iS_i such that

hold, respectively, in the Löwner partial ordering. Further from Lemma 2.3, there always exist solutions K̂ of K̂X = G and K̂_i of K̂_iX = G_iS_i such that (3.29) and (3.30) hold, respectively. Applying Lemma 2.3 to (3.29) and (3.30) leads to the conclusions in the theorem.

Theorem 4.1

Assume that ψ_i in (1.5) are predictable under ℒ₁ and ℒ₂, i.e., (3.5) holds, i = 1, 2. Then, they are predictable under ℒ as well. Also let BLUP_ℒi(ψ_i) and BLUP_ℒ(ψ_i) be as given in (3.7) and (3.26), respectively, i = 1, 2. Then, the following statements are equivalent:

1. BLUP_ℒ(ψ_i) = BLUP_ℒi(ψ_i), i = 1, 2.

2. rXiCov{yi,y}0X′GiCov{ψi,y}=rXiCov{yi,y}0X′,i=1,2.

3. rXiCov{yi,X⊥y}GiCov{ψi,X⊥y}=r[Xi,Cov{yi,X⊥y}],i=1,2.

4. ℛ([G_i, Cov{ψ_i, X^⊥y}]′) ⊆ ℛ([X_i, Cov{y_i, X^⊥y}]′), i = 1, 2.

Proof

If (a) holds, the coefficient matrices of BLUP_ℒi(ψ_i) and BLUP_ℒ(ψ_i) are the same, i.e., the coefficient matrices of BLUP_ℒi(ψ_i) satisfy (3.25)

Simplifying both sides by (2.1) and elementary block matrix operations, we obtain