On decompositions of estimators under a general linear model with partial parameter restrictions

1 Introduction

Consider a partitioned linear model with partial parameter restrictions

where

y is an n × 1 vector of observable response variables,

X = [X₁, …, X_k] is an n × p matrix of arbitrary rank,

X₁, …, X_k are k known n × p₁, …, n × p_k matrices with p = p₁ + ⋅ + p_k,

β=[β1⊤,…,βk⊤]⊤ and β₁, …, β_k are p₁ × 1, …, p_k × 1 vectors of fixed but unknown parameters, ε is an n × 1 vector of randomly distributed error terms,

E(⋅) and D(⋅) denote expectation and dispersion matrix,

Σ is an n × n known nonnegative definite matrix of arbitrary rank,

σ² is an arbitrary positive scaling factor,

A₁, …, A_k are given m₁ × p₁, …, m_k × p_k matrices, respectively, with m = m₁ + … + m_k,

b₁, …, b_k are m₁ × 1, …, m_k × 1 known vectors, respectively.

The system of linear equations in 𝓜 is often available as extraneous information for the unknown parameter vector β to satisfy which is an integral part of the constrained general linear model (CGLM) about the parameter space, and thus should ideally be utilized in any estimation procedure of the parameter space in (1). Associated with 𝓜 are the following k submodels

Obviously, these models can be considered as reduced versions of 𝓜 by deleting k − 1 regressors except X_iβ_i, i = 1, …, k. It has been realized that estimators of the unknown parameters in 𝓜 and 𝓜_i have some intrinsic connections, and people are interested in establishing certain additive decomposition of estimators under the partitioned model and its submodels.

For convenience of representation, denote

for i = 1, …, k. In this setting,

for i = 1, …, k.

CGLMs are usually handled by transforming into certain implicitly constrained model. The most popular transformations are based on model reduction, Lagrangian multipliers, or general solutions of matrix equations through generalized inverses of matrices. A well-known method of incorporating equality constraints in CGLMs is to merge the equations in 𝓜 as an implicitly restricted model

Also, merging the equations in 𝓜_i yields

Linear regression analysis is one of the most-used statistical methods, while linear models are the first type of regression models to be studied extensively in regression analysis, which have had a profound impact and play a central role in both theoretical and applied statistical science, see e.g. [1, 2, 3]. It has a long history in regression analysis to rewrite linear models as certain partitioned forms, and then to make estimation and statistical inference under the partitioned linear models. One of the main objectives in the statistical inference of linear models is to establish various estimators of the parameter spaces in the models and to characterize mathematical and statistical properties and features of these estimators under various model assumptions. In this approach statisticians are often interested in the connections of different estimators and especially in establishing possible equalities between estimators. There have been various attempts to establish additive decomposition equalities for estimators under linear models. Under the assumptions in (9) and (10), it is natural to consider relations among the best linear unbiased estimators (BLUEs) of X^β in (9) and X^iβi in (9) and (10). In this paper, we first verify or prove that under the assumptions that X₁β₁, …, X_kβ_k, X^1β1,…,X^kβk are estimable in (9), the BLUE of Xβ in M^ admits the following two additive decomposition identities

In view of the above observations, we propose the following two additive decomposition equalities for the BLUEs of Xβ and X^β in M^:

and then derive identifying conditions for the equalities to hold, respectively. These estimator decomposition identities have many different statistical interpretations and are not rare to see in statistical analysis of CGLMs. The problem on additive decompositions of BLUEs under general liner models was approached in [4, 5]. Zhang and Tian [6] recently investigated the above two decomposition identities for k = 2 by using some effective algebraic methods of dealing with additive decompositions of matrix expressions and ranks/ranges of matrices.

Before proceeding, we introduce the notation to the reader and explain its usage in this paper. ℝ^m×n stands for the collection of all m × n real matrices. The symbols A^⊤, r(A), and 𝓡(A) stand for the transpose, the rank, and the range (column space) of a matrix A ∈ ℝ^m×n, respectively; I_m denotes the identity matrix of order m. The Moore-Penrose inverse of A, denoted by A⁺, is defined to be the unique solution G satisfying the four matrix equations AGA = A, GAG = G, (AG)^⊤ = AG, and (GA)^⊤ = GA. Further, let P_A, E_A, and F_A stand for the three orthogonal projectors (symmetric idempotent matrices) P_A = AA⁺, E_A = A^⊥ = I_m − AA⁺, and F_A = I_n − A⁺A. Two symmetric matrices A and B of the same size are said to satisfy the inequality A ≽ B in the Löwner partial ordering if A–B is nonnegative definite. Further information about the orthogonal projectors P_A, E_A, and F_A with their applications in the linear statistical models can be found in [7, 8, 9]. Also, it is well known that the Löwner partial ordering is a surprisingly strong and useful property between two symmetric matrices. For more results about the Löwner partial ordering of symmetric matrices and applications in statistical analysis see, e.g., [8]. Generalized inverses of matrices are common tools to deal with singular matrices, which now are a fruitful and core part in current matrix theory and have profound impact in the field of statistics.

2 Some preliminaries in linear algebra

Statistical inference for linear models, as is well known, is entirely based on computations with the given vectors and matrices in the models, and formulas and algebraic tricks for handling matrices in linear algebra and matrix theory play an important role in the derivations of these estimators and the characterization of their performance. Because BLUEs of parameter spaces in linear models are calculated from given matrices and vectors in the models and are often represented by certain formulas composed by given matrices and vectors in linear models, the approach we take to the above problems is in fact to establish and characterize matrix equalities composed by matrices and their generalized inverses, and thus we need to use many influential and effective mathematical tools in order to characterize the above equalities of estimators and their covariance matrices under CGLMs. Many mathematical methods in statistical science require algebraical computations with vectors and matrices. In particular, formulas and algebraic techniques for handling matrices in linear algebra and matrix theory play important roles in the derivations and characterizations of estimators and their performances under linear models. As remarked in [10], a good starting point for the entry of matrices into statistics was in 1930s, while it is now a routine procedure to use given vectors, matrices and their generalized inverses in statistical models to formulate various estimators of parameter spaces in linear models and to make the corresponding statistical inferences.

As the study of additive decompositions of estimators in the contexts of linear regression models requires more effective mathematical analysis tools, it is forced toward algebraic questions that overlap with precise description and characterization of matrix decomposition identities in linear algebra. The scope of this section is to introduce various formulas for ranks of matrices in linear algebra suitable for establishing and characterizing various possible equalities for estimators under CGLMs. In this section, we first introduce some fundamental formulas for calculating ranks of matrices that will be used in the statistical analysis. Recall that the rank of matrix is conceptual foundation in matrix theory and is the most significant finite nonnegative integer in reflecting intrinsic properties of matrices, while the mathematical prerequisites for understanding the rank of matrix are minimal and do not go beyond elementary linear algebra. The intriguing connections between generalized inverses of matrices and rank formulas of matrices were recognized in 1970s, and a seminal work on establishing formulas for calculating matrices and their generalized inverses was presented in [11]. It has been known that matrix rank formulas are direct and effective tools of simplifying matrix expressions and equalities. The whole work in this paper is based on the effective use of the matrix rank methodology (MRM), which is a set of quantitative description techniques that encompass:

1. establishing non-trivial analytical formulas for calculating the maximum and minimum ranks of a matrix expression, and using the ranks to determine the singularity and nonsingularity of the matrix expression, the rank invariance of the matrix expression, the dimension of the row/column space of the matrix expression;

2. establishing formulas for calculating the rank of the difference of two matrix expressions, and using them to derive necessary and sufficient conditions for the two matrix expressions to be equal, i. e., proving matrix equality by matrix rank formulas;

3. characterizing relations between two linear subspaces, or two matrix sets by matrix rank formulas.

The above assertions show that there are important and peculiar consequences of establishing various formulas for calculating ranks of matrices from theoretical point of view. Thus, the MRM in fact provides us with a specified algebraic framework for tackling matrix expressions and matrix equalities, and gives a glimpse into a very broad and interesting field of matrix mathematics. But it was not until a few decades ago that the MRM was essentially recognized as an effective and influential tool in the field of mathematics and was extensively applied in matrix theory and applications. Because matrices are common objects in linear regression analysis, the advent of the MRM has greatly extended from the domain of matrix theory into statistical areas, some seminal work on the fundamental theory of the MRM and its applications in statistics can be found in e.g. in [11, 12, 13]. Some recent work on the MRM in the analysis of additive decompositions of BLUEs under linear models were presented in [4, 5, 6], while some contributions on MRM in the statistical analysis of CGLMs can be found in [14, 15, 16, 17, 18, 19 20, 21, 22, 23, 24].

In order to establish and characterize various possible equalities for estimators in the context of linear models and to simplify various matrix equalities composed by Moore–Penrose inverses of matrices, we will need the following well-known rank formulas involving Moore–Penrose inverses to make the paper self-contained.

With the support of the formulas in Lemmas 2.1–2.3, we are able to covert the problems in (11)–(14) into certain algebraic problems characterizing matrix equalities composed by the given matrices in the models and their generalized inverses, and to derive analytical solutions of the problems by using the methods of matrix equations, matrix rank formulas, and various skillful partitioned matrix calculations.

3 Estimability of parameter spaces under CGLMs

We take σ² = 1 in (1)–(10) for the convenience of presentation below, because it doesn’t play any role in the main results in this paper. In what follows, we assume that the model in (9) is consistent, i.e.,

see [27, 28]. We next introduce the definitions of the estimability of parameter spaces in CGLMs.

It is well known in statistical theory that the unbiasedness of linear statistics with respect to given parameter spaces in linear models is an important property. Considerable literature exists on estimability of parameter spaces in linear models; see e.g. [29 30 31 32 33 34 35 36 37 38] for some excellent expositions. We next present some classic and new results on the estimability of the parameter space in (9) and give their proofs.

Lemma 3.3(c) is easily verifiable for a given model matrix. In particular, they are satisfied under the condition r( X^ ) = p.

4 BLUEs’ computations

Theoretical and applied researches of a CGLM seek to develop various possible estimators of the parameter space in the CGLM. When there exist unbiased estimators for a given parameter space, there are usually many unbiased estimators for the parameter space. Thus, it is natural to seek such an unbiased estimator that has the smallest dispersion matrix among all the unbiased estimators, that is to say, the unbiasedness and smallest dispersion matrices of estimators are most intrinsic requirements in statistical analysis and inference. The concepts of BLUEs of parameter spaces in the contexts of (1)–(10) are given below.

Estimators of the parameter spaces in linear models are usually formulated from mathematical operations of the observed response vectors, the given model matrices, and the covariance matrices of the error terms in the models. Hence, the standard inference theory of linear statistical models can be established from the exact algebraic expressions of estimators, which is easily acceptable from both mathematical and statistical points of view. In fact, linear statistical models are the only type of statistical models that have complete and solid support from linear algebra and matrix theory. Observing that (9) is a special case of GLMs, the following lemma follows from the well-known results on the BLUEs under linear models; see e.g. [28, p. 282] and [39, p. 55].

Note that BLUEs of unknown parameters in linear models are defined from the requirements of both the unbiasedness and the smallest covariance matrices of linear statistics. In order to reveal more deep and fundamental properties and features of BLUEs of unknown parameters in (1), the so-called standard forms of the decomposition refer to the fact that the whole and partial mean vectors Xβ, X^ β, X_iβ_i, and X^ _iβ_i are the components in (1)–(10), while the BLUEs of Xβ and X^ β always exist under (9), as demonstrated in Lemma 4.2.

5 Direct additive decompositions of BLUEs under a CGLM

In this section, we give the analytical expressions of the BLUEs of X_iβ_i and X^ _iβ_i of interest, and present some of their statistical properties.

In what follows, we use { BLUEM^ (Kβ)} to denote the collection of all BLUEM^ (Kβ) in (28).

6 Additive decompositions of BLUEs under a full CGLM and its submodels

For convenience of representation, we adopt the notation in this section.

The misspecified BLUEs under the submodels in (10) are given below.

It should be pointed out that under the assumptions in (9), the k submodels in (10) are misspecified versions of (9). So that the estimators in (44) and (45) are not true BLUEs of X_iβ_i and X^ _iβ_i under the models in (10), that is to say, they neither are unbiased for X_iβ_i and X^ _iβ_i under (9), nor have the smallest covariance matrices in the Löwner sense. In such a case, the sums of the BLUEs may, however, be the BLUEs of Xβ and X^ β under some conditions. In this section, we derive some algebraical and statistical properties and features of the BLUEs under (9) and (10), and then give necessary and sufficient conditions for the equalities in (13) and (14) to hold. Although the results in the last section present exact formulas of BLUEs under various assumptions, we have to pay more attention to the mathematical manipulations hidden behind the BLUE formulas in order to establish the connections among the BLUEs. During this process, many skillful calculations of matrix ranks and elementary block matrix operations will be conducted in establishing and simplifying matrix equalities and expressions.

The following theorem gives a variety of properties of the two estimators in Lemma 6.1.