On the Use of the Helmert Transformation, and its Applications in Panel Data Econometrics

1 Introduction

The Helmert transformation is named after the German geodesist Friedrich Robert Helmert (1876) and has a long history of use in statistics (Sawkins 1940; Cramér 1946, p. 116; Kruskal 1946; Weatherburn 1961, p. 164; Brownlee 1965, p. 271; Rao 1973, p. 182 among others). One application (the application from now on for brevity) of the Helmert transformation in statistics is to find the joint distribution of the sample mean, and the sample variance, calculated from a sample from a normal population. Although the cited references are authoritative, the presentations there of the application are in our view often rather complicated, which inhibits readers’ understating of the topic. We present our version of how the Helmert transformation works in the application, and our version should be particularly suitable and accessible to either beginning undergraduate university students in statistics, or students at both undergraduate and postgraduate level in econometrics, or quantitative social and political sciences which use econometrics.

For example, Cramér (1946, p. 116) and Weatherburn (1961, p. 164) both follow Sawkins (1940), and introduce in abstract terms an orthogonal linear transformation having certain properties. Instead, we think that explicitly stating the transformation, and then directly establishing its key properties would have pedagogical advantages. Rao (1973, p. 182) similarly uses an orthogonal matrix, i.e. a Helmert matrix, which he defines in terms of certain abstract properties. We think there would be pedagogical advantage to instead explicitly display one such Helmert matrix, so that the reader can visualize what is going on, and then proceed to establishing its properties and what it exactly does in the context of the application.

Upon presenting our treatment of the application, which of course uses the vary same ideas of the Helmert transformation and the associated Helmert matrix as the sources mentioned above, we also briefly discuss the use of the Helmert transformation in panel data econometrics. In particular, the Helmert transformation can be employed to eliminate fixed effects in the estimation of fixed effects models. Therefore, our treatment has the advantage that it is very concrete and explicit compared to previous literature, and advocates for more extensive use of the Helmert transformation. Unlike previous authors, we firstly explicitly state the Helmert transformation and display the Helmert matrix, and then proceed to prove the implications of applying the Helmert transformation in the application.

Our proof is closest to Brownlee (1965, p. 271). Brownlee (1965, p. 271) starts with a formula for the sample variance and using it, derives new variables that possess certain desired properties. This derivation of the new variables is fairly involved. We instead start by defining the new variables and then using a simple induction argument, prove that these variables have the same variance as the original variables.

We use mathematical induction in our proof, and Stigler (1984) also uses mathematical induction in his proof, but the argument there is different. He first establishes the joint distribution of the sample mean and variance for a sample of just two observations and then via the induction argument, proves that this joint distribution extends to samples of larger size. Zehna (1991) asserts that Stigler’s proof is not completely rigorous, and relies three times throughout the proof on a faulty argument.

This article proceeds as follows. In Section 2 we introduce the Helmert transformation and derive the joint distribution of the sample mean and the sample variance. In Section 3 we introduce the Helmert matrix associated with the transformation, and provide some remarks on its properties. In Section 4 we suggest that the use of the Helmert transformation has been somewhat overlooked in within estimation of one way error component fixed effects models in panel data econometrics, and we briefly survey the extant applications of the Helmert transformation in panel data context. In the last section we conclude.

2 The Helmert Transformation and the Joint Distribution of the Sample Mean and the Sample Variance in Samples from a Normal Population

We have a set of T random variables x _t , t = 1, 2, …, T independently and identically distributed (i.i.d. from now on) as Normal(μ, σ ²), and we want to find the joint distribution of the sample mean x ̄ T = ∑ t = 1 T x t / T and the sample variance s 2 = ∑ t = 1 T ( x t − x ̄ T ) 2 / ( T − 1 ) .

Consider the Helmert transformation, which takes the set x _t , t = 1, 2, …, T and produces a new set of variables z _t , t = 1, 2, …, T as follows:

where x ̄ t − 1 = ∑ τ = 1 t − 1 x τ / ( t − 1 ) .

The new set of variables z _t have convenient properties. (In what follows, we liberally use the properties of the expectation E(⋅), variance Var(⋅), and covariance Cov(⋅, ⋅) operators, and we assume that the reader is familiar with these properties.)

Properties of the Helmert transformation:

1. z _t , t = 1, 2, …, T are mutually uncorrelated:   Cov ( z 1 , z t ) = Cov x ̄ T , ( x t − x ̄ t − 1 ) ( t − 1 ) / t   = ( 1 / T ) ( t − 1 ) / t ⋅ Cov ∑ τ = 1 T x τ , x t − ∑ τ = 1 t − 1 x τ / ( t − 1 )   = ( 1 / T ) ( t − 1 ) / t [ σ 2 − ( t − 1 ) σ 2 / ( t − 1 ) ] = 0 for t = 2, …, T. And,   Cov ( z t , z θ ) = Cov x t − ∑ τ = 1 t − 1 x τ / ( t − 1 ) , x θ − ∑ τ = 1 θ − 1 x τ / ( θ − 1 ) = −σ ²/(θ − 1) + (t − 1)σ ²/[(t − 1)(θ − 1)] = 0, where without loss of generality we take θ > t > 1. (Property 1 does not require normality; an i.i.d. distribution of the variables would be sufficient.)

2. z _t , t = 1, 2, …, T are linear combinations of x _t , t = 1, 2, …, T (which we have assumed, are jointly normally distributed) and hence z _t , t = 1, 2, …, T are jointly normally distributed as well.

3. E z 1 = E x ̄ T = μ ; E z t = E [ ( x t − x ̄ t − 1 ) ( t − 1 ) / t ] = ( t − 1 ) / t ⋅ ( μ − μ ) = 0 , for t = 2, 3, …, T.

4. Var z ₁ = σ ²/T, Var z _t = [σ ² + σ ²/(t − 1)](t − 1)/t = σ ², t = 2, …, T. So z ₂, …, z _T are homoskedastic.

5. ∑ t = 1 T ( x t − x ̄ T ) 2 = ∑ t = 2 T z t 2 = ∑ t = 2 T ( x t − x ̄ t − 1 ) 2 ( t − 1 ) / t , which we prove in the theorem below.

We will now use the properties of the Helmert transformation to deduce the joint distribution of the sample mean and the sample variance in samples from a normal population. We use the following three facts. First, linear combinations of jointly normal random variables are themselves jointly normally distributed (Cramér 1946, p. 213; Weatherburn 1961, p. 57). Second, for a set of jointly normal random variables, if they are uncorrelated, they are independent as well (e.g. Lancaster 1959; David 2009). There is a subtle point here, the set of variables need to be jointly normal, as one can construct counter examples where the marginal distributions are normal but the joint distribution is not normal, variables are uncorrelated, and yet they are not independent. Lancaster (1959) presents such a counter example, and precisely states the conditions in his Theorem 1. Third, the sum of k independent, squared, standard normal variables is distributed as χ ²(k), which is a distribution discovered and described by Helmert (1876). (Helmert (1876) discovered the χ ² distribution, however he did not observe that the sample mean and the sample variance are independent, see David (2009).)

3 The Helmert Matrix

The previous section is self contained, and using the Helmert transformation to deduce the joint distribution of the sample mean and the sample variance does not require any matrix algebra. However if there is need, or desire to do so, one can also relate the Helmert transformation from the previous section to what Lancaster (1965) calls a Helmert matrix in the strict sense.

Consider the following matrix

We can verify by direct multiplication that the rows of this matrix are orthogonal, that is, HoHo′ results in a diagonal matrix. We can also consider the rescaled version of Ho

where diag(v) is an operator that transforms a vector v into a diagonal matrix, and overall the operation diag(v) ⋅ Ho results in multiplying the ith row of the matrix Ho by the ith element of the vector v. With the choice of the first element in v as 1/T, we can see by direct multiplication that Hm′Hm is symmetric with one element repeated on the main diagonal and another element repeated everywhere off the main diagonal. HmHm′ is diagonal and is almost the identity matrix, only the upper left element is different from 1, the rest of HmHm′ coincides with the identity matrix.

If we instead choose the first element in v to be 1 / T ,

direct multiplication shows that Hn′Hn and HnHn′ are both the identity matrix, and we would call such a matrix Hn an orthonormal matrix.

We see that if we arrange the set x _t , t = 1, 2, …, T into a column vector x ≡ x 1 x 2 … x T ′ , and choose the first element of v to be 1/T, we will obtain the Helmert transformation from the previous section, z = Hm ⋅ x, where z ≡ z 1 z 2 … z T ′ . Because of the appealing aesthetics of Hn with the first element in v chosen to be 1 / T , i.e. a choice resulting in an orthonormal matrix Hn′Hn = HnHn′ = Identity, all authors we are aware of use this version of the Helmert matrix. However for our application all we need is that HmHm′ be a diagonal matrix with all the elements on the main diagonal below the first being equal to 1. In this situation, the elements of the vector z = Hm ⋅ x will be mutually uncorrelated and each element z _t , t = 2, …, T will be homoskedastic. Therefore, for our application choosing the first element in the vector v as 1/T serves perfectly fine.

4 The Helmert Transformation in Panel Data Models

We have derived the joint distribution of the sample mean and the sample variance with particular focus on the simplicity and concreteness of the derivation, and particular focus on the use of the Helmert transformation. The Helmert transformation has found its application in the “fixed effects” panel data model too. Consider the standard one way “fixed effects” panel data model (e.g. Wooldridge 2010, Ch.10; Hsiao 2014, Ch.3)

where the regressand y _it is a scalar, the regressor vector x _it is K × 1, y i t , x i t ′ are i.i.d. for i = 1, 2, …, I, i.e. the variables constitute an i.i.d. random sample in the cross section. The individual “fixed effects” (so called by convention) are time constant, random and potentially correlated with the regressor vector. The idiosyncratic error term ɛ _it is uncorrelated with the regressor x _jτ for all t, τ, i, j, i.e. the regressor x _it is strictly exogenous with respect to ɛ _it conditional on the fixed effects, and ɛ _it is i.i.d. both in the cross section and in the time series dimensions.

Consistent estimation of the parameter vector β in Eq. (3) under the assumptions that the fixed effects can be arbitrarily correlated with the regressors, and the regressors are strictly exogenous with respect to ɛ _it , conditional on the fixed effects, proceeds by eliminating the fixed effects. The conventional way of eliminating the fixed effects is by the within transformation, firstly averaging Eq. (3) across time, to obtain y ̄ i T = x ̄ i T ′ β + μ i + ε ̄ i T , where y ̄ i T = ∑ t = 1 T y i t / T , x ̄ i T = ∑ t = 1 T x i t / T and ε ̄ i T = ∑ t = 1 T ε i t / T . Then we subtract this averaged equation from Eq. (3) to eliminate the fixed effects and to obtain the estimating equation

Finally, we estimate Eq. (4) by ordinary least squares (OLS) over the I ⋅ T pooled observations to obtain the within estimator

The within estimator is well studied, well understood and a basic building block in the panel data econometrics literature. However the within transformation introduces strong correlation between the transformed errors in the estimating Eq. (4), because all the transformed errors ( ε i t − ε ̄ i T ) for a cross sectional unit i share the same ε ̄ i T . This makes residual diagnostic checks and residual analysis awkward and difficult, for example, if we wanted to test that the errors ɛ _it are indeed i.i.d.

On the other hand if we apply the Helmert transformation in Eq. (1) on each variable in the fixed effects model Eq. (3) and for each cross sectional unit i separately, to construct transformed variables corresponding to z ₂, z ₃, …, z _T which have mean 0, then we again eliminate the fixed effects μ _i :

i = 1, 2, …, I, t = 2, …, T, where y ̄ i t − 1 = ∑ τ = 1 t − 1 y i τ / ( t − 1 ) , x ̄ i t − 1 = ∑ τ = 1 t − 1 x i τ / ( t − 1 ) , ε ̄ i t − 1 = ∑ τ = 1 t − 1 ε i τ / ( t − 1 ) .

We can proceed with the OLS estimation of the Helmert-transformed estimating Eq. (6),

which is the best linear unbiased estimator, because the errors in the Helmert-transformed estimating equation ( ε i t − ε ̄ i t − 1 ) ( t − 1 ) / t are uncorrelated and homoskedastic, (and normal if the original ɛ _it were normal to start with). By invoking Property 6 of the Helmert transformation, one can verify that the estimator in Eq. (7) coincides with the within estimator in Eq. (4). However, as an added benefit, we can apply any residual diagnostics and checks that we might have in mind, because the Helmert-transformed errors in the estimating equation have the same stochastic properties as the original errors in the structural model. To the best of our knowledge, the existing literature on panel data models has not exploited this simplification in the analysis afforded by the Helmert transformation.

The use of the Helmert transformation in the context of panel models and, in particular, dynamic panel models has been popularized by Arellano and Bover (1995) and Arellano (2003). (See also Alvarez and Arellano 2003.) It is also described in Hansen (2022, Section 17.43). A dynamic panel model is like the one given in Eq. (3), but it additionally contains the lagged values of the dependent variable as regressors. For example, if y _it (directly) depends only on its own value in the previous period, then the model is

Arellano and Bover (1995) suggest to transform the variables in the following way:

i = 1, 2, …, I, t = 1, …, T − 1, where y ̄ i t + 1 = ∑ τ = t + 1 T y i τ / ( T − t ) , x ̄ i t + 1 = ∑ τ = t + 1 T x i τ / ( T − t ) . Observe that this is the same Helmert transformation in Eq. (1) but with the variables ordered in the reverse order according to the time index. Arellano and Bover (1995) refer to this transformation as “the forward orthogonal deviation” as opposed to “the backward orthogonal deviation” which is the one displayed in Eq. (6).

Both forward and backward orthogonal deviations produce uncorrelated and homoskedastic errors in the transformed model, provided that the original errors ɛ _it are i.i.d. However, the forward orthogonal deviation has the following advantage in the dynamic panel model. When transforming the variables to remove the fixed effects, it introduces correlation between the transformed lagged dependent variable and the transformed error term. Therefore, one needs to use an instrumental variables estimator to obtain consistent estimates of α and β. With the forward orthogonal deviation, the past values of the dependent variable y _i1, …, y _it−1 are valid instruments for ( y i t − 1 − y ̄ i t ) ( T − t − 1 ) / ( T − t ) , which is the new lagged dependent variable after the variable transformation. Furthermore, Hayakawa (2009a, 2009b) shows that the instrumental variables estimator is more efficient if the instruments themselves are constructed using backward orthogonal deviations.

5 Conclusion

We revisit the Helmert transformation, and we provide a simple and useful induction-based derivation of the joint distribution of the sample mean and the sample variance in samples from independently and identically distributed normal random variables. Our derivation is concrete and should be appealing to students of statistics and econometrics. We also suggest one fruitful application of the Helmert transformation in panel data econometrics – residual based tests in fixed effects models. We briefly review the applications of the Helmert transformation in panel data context, where the transformation is more commonly known as “the forward/backward orthogonal deviations operator”.