Moment-Based Estimation of Linear Panel Data Models with Factor-Augmented Errors

1 Introduction

The prevalence of panel data in modern economics has led theorists and practitioners to pay more attention to unobserved heterogeneity. A popular representation of unobserved effects is the linear factor structure ∑ j = 1 p f t j γ j i where f _tj is a time-varying macro effect or “common factor” and γ _ji is an individually heterogeneous response or “factor loading”. Except under highly-specific circumstances, the usual within transformation is insufficient in controlling for these unobserved effects. Many factor estimators require the number of time periods T to grow large with the number of cross-sectional units N. As the vast majority of microeconometric data sets have only a few time periods, the recent literature assumes T is fixed while N goes to infinity.

One of the most popular approaches is the common correlated effects (CCE) estimator of Pesaran (2006). He assumes an additional reduced form model where the covariates exhibit a pure factor model. The pooled CCE estimator comes from the OLS regression that estimates unit-specific slopes on the cross-sectional averages of the dependent and independent variables. CCE is similar to a fixed effects treatment that seeks to eliminate the factors and remove a source of both endogeneity and cross-sectional dependence. Consistency and asymptotic normality was originally proved for sequences of N and T going to infinity but recent work extends the CCE framework to a fixed-T setting. De Vos and Everaert (2021) derive a fixed-T consistency correction for the dynamic CCE estimator but requires T → ∞ for asymptotic normality. Westerlund, Petrova, and Norkute (2019) provide the first asymptotic normality derivation of pooled CCE when T is fixed and N → ∞.

Despite its theoretical rigor and practicality, the CCE estimator does not use all fundamental moment conditions of the model. This fact implies that the pure factor structure in the covariates cannot improve efficiency for the CCE estimator because it is irrelevant from a population information perspective, despite being necessary for consistency. As other fixed- T N -consistent estimators exist that do not require this assumption, it is worth investigating how else the CCE assumptions can be used in estimation. I use the quasi-long-differencing (QLD) transformation of Ahn, Lee, and Schmidt (2013) to explore the implications of this model and show that the additional ignored CCE pure factor moments are relevant for the estimation of the parameters of interest in the main equation.^[1]

Ahn, Lee, and Schmidt (2013) choose a particular normalization of the unobserved factors that induces a smaller set of estimable parameters. They include these parameters in their QLD transformation that can then asymptotically eliminate the space spanned by the factors. While they did not originally assume a pure factor structure in the covariates, I use their transformation to study the CCE model. I show that the reduced form pure factor model provides information for estimating the parameters of interest, which is ignored by the pooled CCE estimator, but can be integrated into more efficient joint GMM estimation using the Ahn, Lee, and Schmidt (2013) estimator. Further, the CCE estimator generally uses more factor proxies than necessary that can lead to inefficiency. I also demonstrate how the literature’s current understanding of the factor loadings causes problems for inference under model misspecification, which I correct for. Finally, the CCE estimator cannot accommodate more covariates than time periods, a highly restrictive condition in microeconometric settings. For example, in an intervention analysis with only pre-treatment, treatment, and post-treatment observations, classical CCE would require the treatment indicator to be the only regressor. My estimators will not require this restriction.

Another potential source of heterogeneity in linear models comes from the slope coefficients on the observed variables of interest. Pesaran (2006) proves fixed-T consistency of the mean group CCE estimator under random slopes but assumes they are independent of everything else in the model. Asymptotic normality requires T → ∞ and pooled CCE is studied under constant slopes. Recent work by Westerlund and Kaddoura (2022) and Brown, Schmidt, and Wooldridge (2023) extend this analysis to a fixed-T setting, but require strong assumptions on the individual slopes for consistency. I prove fixed-T consistency and asymptotic normality of my new pooled and mean group QLD estimators. I show that the first-stage estimation of the QLD parameters does not affect consistency, which mirrors the pooled OLS result of Wooldridge (2005), who assumes known factors. To the best of my knowledge, this paper is the first to consider arbitrary random slopes in the context of fixed-T panels with factor-driven endogeneity.

The rest of the paper is structured as follows: Section 2 discusses the model of interest. Section 3 provides the estimators under consideration and derives their theoretical properties. Section 4 provides simulation evidence for the finite-sample properties of the expanded GMM estimators, along with the new QLD estimators. Section 5 compares the various estimators by looking at the effect of education expenditure on standardized test performance using a school district-level data set from the state of Michigan. Section 6 contains concluding remarks.

2 Model

This section lays out the models considered in Westerlund, Petrova, and Norkute (2019) and Ahn, Lee, and Schmidt (2013), the fixed-T CCE and QLD approaches respectively. Throughout the paper, the equation of interest is

where y _i is a T × 1 vector of outcomes, X _i is T × K matrix of covariates, β _i is a K × 1 vector of random slope coefficients, F ₀ is a T × p ₀ matrix of factors common to all units in the population, γ _i is a p ₀ × 1 vector of factor loadings, and u _i is a T × 1 vector of idiosyncratic shocks.^[2] A ‘0’ subscript denotes the true or realized value of an unobserved parameter. We are interested in inference on the K × 1 vector β ₀ = E( β _i ). The factor structure F ₀ γ _i is treated as a collection of nuisance parameters. p ₀ is then unobserved because F ₀ and γ _i are unobserved. However, we can consistently test for p ₀, so it will be treated as known. Simulation evidence from this paper and others also suggests that overestimating p ₀ does not cause inconsistency in QLD estimation. p denotes the number of factors specified by the econometrician.

The random slope condition as stated in equation (2) is fairly general. It allows for the slopes to be written as β _i = β ₀ + b _i where E( b _i ) = 0 . Endogeneity occurs when b _i is correlated with other aspects of the model. Most fixed-T treatments of random slope models either exclude or simplify the factor structure as in a fixed effects analysis. Examples of fixed effects treatments include Juhl and Lugovskyy (2014), Campello, Galvao, and Juhl (2019), and Breitung and Salish (2021). Though Chudik and Pesaran (2015); Neal (2015); Norkutė et al. (2021); Pesaran (2006) allow for random slopes and arbitrary factors, they require T to grow to infinity and make strong exogeneity conditions that I avoid.^[3] Wooldridge (2005) considers this model but with known factors. Fixed-T studies with unknown factors included Westerlund and Kaddoura (2022) and Brown, Schmidt, and Wooldridge (2023). However, they both assumed exogeneity of b _i with respect to aspects of the covariate process. I introduce estimators that allow for varying degrees of correlation, including one that allows for arbitrary correlation between X _i and b _i . Finally, all results in this paper hold in the case of homogeneous slopes when β _i = β ₀ for all i.

I define p ₀ as the number of factors whose loadings correlate with X _i . This interpretation is similar to Ahn, Lee, and Schmidt (2013) and implicit to the CCE model as discussed in the following section. One justification of this interpretation is to write the full error as D ₀ ρ _i + ϵ _i where D ₀ is a possibly infinite dimensional matrix of common factors and ϵ _i is a vector of idiosyncratic errors. Then F ₀ γ _i is the set of variables from D ₀ ρ _i that are correlated with X _i and the rest are absorbed into the error. However, it is entirely likely that γ _i is correlated with the other loadings that are uncorrelated with X _i . For this reason, I allow the loadings and errors to correlate. The factors are treated as constant. I could alternatively treat them as random and independent of the cross-sectional data like in Westerlund, Petrova, and Norkute (2019).

3 Estimation

I now state this paper’s primary assumptions. The first assumption defines the model of interest. The second set specifies the pure factor structure in X _i similar to Westerlund, Petrova, and Norkute (2019).

Assumption 1 defines the relevant population model. I do not explicitly state the exogeneity conditions on the random slopes because different estimators will be shown to be consistent under different assumptions, as will be made clear in the relevant theorems. Assumption 2 specifies the pure factor assumption similar to Pesaran (2006) and Westerlund, Petrova, and Norkute (2019). Unlike these CCE analyses, I do not require independence between the errors in the main or reduced form equations. In fact, I only restrict E( u _i | V _i ) = 0 but place no assumptions on the conditional distribution D( V _i | u _i ). This assumption allows for heteroskedasticity conditional on both observables and unobservables in both sets of errors which, while common in the fixed-T GMM literature, is ruled out in the CCE approaches of Pesaran (2006) and Westerlund, Petrova, and Norkute (2019).

I assume the factor loadings are random and iid in the cross section. I could relax this assumption at the cost of notational complexity. Westerlund, Petrova, and Norkute (2019) show that more general sampling techniques are allowed in the asymptotic analysis. For example, I could replace Assumption 2(iv) with Assumption C of Westerlund, Petrova, and Norkute (2019). However, I do not assume ( γ _i , Γ _i ) is orthogonal to ( u _i , V _i ). While this assumption is reasonable when the factor structure captures all correlation between X _i and the full error Fγ _i + u _i , it can fail if the model is misspecified. I show in Section 3.2 that N -consistent estimation is possible even if 1 N ∑ i = 1 N V i ⊗ γ i does not converge to zero due to model misspecification. Further, because I assume random sampling, I define population moments in terms of a generic ‘i’ unit. For example, Assumption 2(iv) makes conditions on E( γ _i ), which is the population mean for every unit’s factor loadings.

As discussed earlier, I do not require T > K + 1, unlike the CCEP estimator. I directly use the moments E H 0 ′ Z i = 0 to remove the factors and only require K ≥ p ₀ + 1, a restriction also made by Pesaran (2006) and Westerlund, Petrova, and Norkute (2019). As long as there are enough time periods to cover all the unobserved effect, my procedure can allow for an arbitrary number of covariates. I also discuss in Section 3.2 how to include known factors like a heterogeneous intercept that decreases the number of relevant factors and makes the assumption even less restrictive.

4 Simulations

This section considers the finite-sample performance of the QLD estimators compared to the GMM and CCE estimators of Ahn, Lee, and Schmidt (2013) and Pesaran (2006) respectively. The main model is

as in Assumptions 1 and 2. There are two variables with slopes β ₀ = (1, 1)′ I do not include random slopes as they would only serve to increase the amount of noise in the model. I refer the reader to Campello, Galvao, and Juhl (2019) for simulation studies regarding the performance of pooled estimators when slopes are correlated with the variables of interest.

The two factors are generated as AR(1) random processes with initial value from a normal distribution with mean 1 and variance 1, having parameters 0.75 and −0.75 respectively. The factors are generated once then fixed over repeated replications. The loadings on X _i are drawn as

where Γ_1,1 and Γ_2,2 are the upper-left and bottom-right diagonal values of Γ _i . The errors u _i and V _ik (k = 1, 2) are independently drawn from a multivariate normal distribution with mean 0 _T×1 and variance C where C is the correlation matrix from an AR(1) process with parameter 0.75. Each simulation study includes 1,000 replications.

Table 1 compares the Ahn, Lee, and Schmidt (2013) estimator both with and without the additional moments E H 0 ′ Z i = 0 . Both estimators are computed as two-step estimators where the optimal weight matrix is calculated with a consistent first-step estimator. The first-step estimator uses an identity weight matrix. I report the results for each (N, T) pair for both sets of coefficients.

The GMM estimator based on the Ahn, Lee, and Schmidt (2013) residual E ( vec ( X i ) ⊗ H 0 ′ ( y i − X i β 0 ) ) only is GMM1, whereas the GMM estimator using the Ahn, Lee, and Schmidt (2013) residual and the additional moments E H 0 ′ Z i = 0 is GMM2. GMM1 uses TK(T − 2) moments while GMM2 uses an additional (T − 2)K. The GMM estimator using both sets of moments generally outperforms the original Ahn, Lee, and Schmidt (2013) estimator in terms of root mean square error, implying that the additional moments are practically relevant in finite samples.

Before turning to a comparison of the pooled QLD and CCE estimators, I first investigate the performance of QLDP when p ₀ is misspecified in estimation of θ ₀. p ₀ = 2 and I look at the performance of QLDP for estimation under p = 1, 2, 3.

Table 2 gives the results for the QLDP under the different specifications. My results track with previous simulation evidence provided by Ahn, Lee, and Schmidt (2013) and Breitung and Hansen (2021). Underestimating p ₀ leads to substantial bias that does not decrease with N. However, overestimating p ₀ leads to only slightly worse performance than correct specification. The bias is larger but decreases with N; in fact, even N = 300 gives reasonable bias for the p = 3 estimator. The p = 3 estimator also performs worse than the correctly specified estimator in terms of standard deviation, which is not surprising. Overall, I find evidence that overestimation of p ₀ does not lead to substantial bias in estimation, but underestimating p ₀ can.

I also consider hypothesis testing for different specifications of p. Using the same model but setting β ₀ = (0, 0)′, I construct the QLDP estimators under p = 1, p = 2, and p = 3, when the true value is p ₀ = 2. Table 3 includes the average rejection rate for the usual Wald statistics of the individual hypothesis tests H ₀: β ₁ = 0 and H ₀: β ₂ = 0 against the relevant two-sided alternative. I estimate the standard error via nonparametric bootstrap with I carry out the tests at the 5 % level, so the test is considered a rejection if the p-value associated with the Wald statistic (evaluated with a standard normal distribution) is greater than or equal to 0.975. We can see that correct specification and overestimation of p ₀ leads to reasonable rejection rates when the null hypothesis is true, especially as N increases for a given T. However, underestimating p ₀ gives wildly unrealistic rejection rates due to the bias caused by underestimating p ₀ as described by Table 2. These results further bolster the simulation evidence of Breitung and Hansen (2021) who study the classical Ahn, Lee, and Schmidt (2013) GMM estimator.

Table 4 looks at the QLDP estimator compared to the CCEP estimator where the QLD transformation is estimated under p = p ₀ = 2 when K = 2 (returning the original DGP with β ₀ = (1, 1)′). I omit the GMM estimators because they are outperformed by the just-identified QLDP in terms of root mean square error. Unsurprisingly, the QLDP bias is significantly lower than both GMM estimators. Its standard deviation is often significantly lower when N is smaller, but this trend starts to reverse compared to GMM2 for larger samples. Note that the CCEP is badly biased when T = 3 as K + 1 = 3. However, the QLDP is still consistent here. Further, the QLD estimators takes p ₀ as known while the CCE estimators “overestimates” p ₀ with the cross-sectional averages, of which there are K + 1 = p ₀ + 1. One might suspect this overestimation leads to inefficiency, which is born out by the SD of the simulations. The QLDP estimator consistently shows a 15%–25 % decline in standard deviation over the CCEP estimator. Further, the CCE identifying condition requires T > K + 1, which causes severe bias when violated. The QLDP estimator significantly outperforms the CCEP estimator in every setting provided.

Comparing Tables 1 and 4, the QLDP performs much better than either of the GMM estimators when N is small despite the fact that we know they are using valid instruments. That the QLDP has better finite-sample performance is most likely due to the fact that it uses a just identified system of moments. I also include simulations in Table 5 where T is larger. The number of parameters estimated by the QLDP grows linearly with T so we would expect poor performance relative to CCEP when T grows. Below, we see that CCEP generally outperforms QLDP in terms of RMSE. When T = 20, the QLDP performs exceptionally poorly. While we may gain efficiency using the QLD estimators when T is small, QLD should not be applied to cases where T is relatively close to N.

Finally, I investigate the performance of the mean group quasi-long-differencing (QLDMG) and mean group common correlated effects (CCEMG) estimators. The QLDMG estimator is given by equation (21) and the CCEMG estimator is identical to the QLDMG estimator but with M F ̂ in place of H ̂ H ̂ ′ . Consistency is proved in Pesaran (2006) but, like the pooled estimator, will eventually require a modern treatment that either controls for the asymptotic degeneracy in M F ̂ like Karabiyik, Reese, and Westerlund (2017) and Westerlund, Petrova, and Norkute (2019). Table 6 contains the results for the mean group estimators where the QLD transformation is estimated assuming p = p ₀ = 2. I start at T = 5 so that T − p ₀ > p ₀ and the CCEMG estimator is well-defined.

Despite T > 2K + 1 for each setting, the CCEMG estimator exhibits substantial bias when T = 6, though the QLDMG estimator appears unbiased. The QLDMG outperforms the CCEMG in terms of RMSE for each N and T besides N = 600 and T = 8. We would expect the CCEMG to perform well relative to the QLDMG as T grows due to the incidental parameter problem in the first-stage QLD estimation. However, even for moderately low values of N and large values of T, the QLDMG has optimistic properties.

I include additional simulations in the Appendix that consider weak factors and a failure of identification where the factor loadings all have mean zero. I compare the QLDP, CCEP, and GMM estimators together. All estimators perform reasonably well in the weak factor case and poorly when the loadings are mean zero. The GMM estimators outperform the linear estimators because they do not require the CCE moments for identifying β ₀.

5 Application

I evaluate the effect of education expenditures on student performance using standardized test pass rates as a proxy. The policy variable of interest is average real expenditure per pupil, as it represents the effect of additional expenditure on test scores. Starting in the 1994/1995 school year, the state of Michigan began awarding “foundation grants” that were based on the per-student spending of the school district in the previous year. The goal was to eventually bring schools up to a benchmark “basic foundation” amount that increased over time. The state started by awarding foundation grants to increase expenditure to a minimum $4,200 per student or an additional $250 per student, whichever was higher. By 2000, the minimum and benchmark amounts were equal at $5,700.

Michigan students undertake a battery of standardized tests in elementary, junior, and secondary school. Like Papke (2005) and Papke and Wooldridge (2008), I focus on the fourth-grade math test because it was consistently defined and measured over the sample. While the abrupt nature of the policy change may appear to present a natural experiment setting, it is difficult to compare Michigan’s standardized test pass rates to external states. Standardized tests are only standardized within a state: students from different states are tested on different curricula with different grading methodologies, meaning that passing grades cannot be compared between states. Because traditional causal inference methods like difference-in-differences or synthetic control are not applicable here (since they require commonly measured outcomes), I instead model the conditional mean of a district’s pass rate in terms of educational inputs. However, we may believe that unobserved variables will bias results. For example, school funding came primarily from local property taxes before the fulfillment of the new state policy. It is likely that local tax receipts are affected by macroeconomic shocks at differential rates, depending on district-level industrial makeup and exposure to foreign trade. Because these macroeconomic shocks are difficult to measure precisely, we instead model them using interactive fixed effects and remove them using the methodologies described in this paper.

I consider school district-level data in the state of Michigan over the time periods 1995–2001. There are N = 501 school districts observed for T = 7 school years over 1995–2001.^[8] I present summary statistics and descriptions for the variables of interest.

The outcome variable, math4, denotes the pass rate for fourth-grade students taking a standardized math test and stands as a measure of student achievement. Expenditures per pupil were averaged over the current year as well as the previous three, meaning average real expenditure per pupil in 1995 is an average of expenditure in 1992, 1993, 1994, and 1995. The equation of interest is

which is similar to Papke (2005) but with the allowance for interactive fixed effects. I collect lunch _it , log(enroll) _it , and log(avgrexp) _it and use the reduced form CCE equation from Assumption 2 to implement the pooled QLD estimator.^[9] This specification allows me to test for the number of factors. I also use the Ahn, Lee, and Schmidt (2013) GMM function to test for p ₀, with and without the CCE equations.

Table 7 provides the p-values for testing the hypothesis H ₀: p ₀ = p versus H ₁: p ₀ > p.

A rejection of the hypothesis suggests more factors than the tested value, and a failure to reject suggests the current value is correct. The titles ‘GMM1’, ‘GMM2’, and ‘RF2’ (for reduced form) refer to the respective objective function used to test the relevant hypothesis. I stress that testing for p ₀ comes from a long-established literature, briefly described in Ahn, Lee, and Schmidt (2013). The only new concept I introduce with respect to this specific specification test is using the reduced form moments E H 0 ′ Z i = 0 .

GMM1 is just the Ahn, Lee, and Schmidt (2013) objective function. GMM2 is the Ahn, Lee, and Schmidt (2013) objective function with the additional moments E H 0 ′ Z i = 0 . Finally, RF is just the reduced form moments E H 0 ′ Z i = 0 . GMM1 suggests that the correct number of factors is p ₀ = 2. GMM2 and RF both reject p ₀ = 2 at any reasonable confidence level, and GMM2 rejects p ₀ = 3, though it uses a much larger set of moments than the other two which may decrease power. It may also suffer from the same global identification problems discussed in Hayakawa (2016), which suggests the GMM1 test will perform better practically. I stop testing at p ₀ = 3 because RF is just identified at p ₀ = 4. Regardless of the tests, the moments E H 0 ′ Z i = 0 only allow me to estimate up to four factors. Even if p ₀ > 4, the QLDP nets more unobserved heterogeneity than TWFE.

I estimate the model’s parameters via TWFE, CCEP, QLDP, and the two GMM estimators. As T = 7 and K = 3, the CCEP estimator can accommodate X ̄ , y ̄ , and a heterogeneous intercept in F ̂ . Further, the pooled QLD estimator is computed with p = K + 1 = 4 after eliminating a heterogeneous intercept from X _i and y _i , unit-by-unit. As such, QLDP is a natural comparison to TWFE. Theorem 3.4 tells us that β ̂ Q L D P is invariant to common variables when p = K + 1 and simulation evidence in the previous section suggests that overestimating p ₀ is not particularly problematic from the perspective of bias or inference. Since it also eliminates a heterogeneous intercept, it should be consistent if TWFE is consistent.

I present results in Table 8 that show estimation after eliminating a heterogeneous intercept. For CCEP, this simply amounts to F ̂ = ( 1 , y ̄ , X ̄ ) . I manually remove a unit fixed effect for QLDP, GMM1, and GMM2. I also cross-sectionally demean the data for GMM1 and GMM2 to control for secular time effects. I set p = 2 for the GMM estimators because there is no algebraic benefit to overestimating p as with the QLDP estimator. Standard errors are in parentheses while p-values are in brackets. I provide the usual cluster-robust standard errors for TWFE. For the other estimators, the reported standard errors are generated via the nonparametric bootstrap, with 5,000 replications for CCE and QLDP. The GMM estimators are reported with 500 replications due to computational cost.

The QLDP estimator suggests substantial estimates for the effect of per student expenditures. A 10 % increase in the average expenditure per student is associated with an 8.3 percentage point increase in the math test pass rate, which is significant at the 5 % level. This estimate is more than twice as large as the TWFE estimate and more than three halves the CCEP estimate. These results suggest that TWFE is not adequately controlling for the heterogeneity present in the data set. Both the CCEP and QLDP estimates are statistically significant at the 5 % level. The TWFE standard errors are generally smaller than CCE and QLD because it removes less variation from the data. Both of the GMM estimators give a similar point estimate to the CCE estimator, but with smaller standard errors. Surprisingly, GMM2 has a larger standard error than GMM1.

I also considered estimation via the mean group QLD and CCE estimators. However, both parameter estimates and standard errors were unreasonable compared to the other estimators. In fact, the p-values were significantly larger than any other reported case and suggested a lack of precision. Recall that the mean group estimators require much stronger exogeneity and identifying conditions than the pooled estimators.

6 Conclusions

This paper considers fixed-T estimation of linear panel data models where the errors have a general unknown factor structure. I use the quasi-long-differencing transformation studied by Ahn, Lee, and Schmidt (2013) to eliminate the factor structure and provide moment conditions for estimation. I study the moments implied by assuming a common correlated effects model. Applying the QLD transformation to the independent variables improves efficiency of estimating the parameters of interest in the main equation, which is information that CCEP does not use. Current proofs of fixed-T asymptotic normality of the CCEP estimator assume loadings that are strictly exogenous with respect to the idiosyncratic errors in the independent variables. I show that the uncorrelated loadings assumptions implies the existence of an even larger number of moments which CCE neglects. I also provide robust standard errors in a more general setting than the CCE models in Pesaran (2006) and Westerlund, Petrova, and Norkute (2019).

I apply the moment-based perspective to a heterogeneous slopes model similar to the original Pesaran (2006) setting. I prove consistency and asymptotic normality of pooled and mean group estimators based on the QLD transformation and put no restrictions on the relationship between T and K, in contrast to CCE. These estimators are shown to outperform CCE estimators in finite samples even when N is small. The pooled QLD estimator also has the desirable property of invariance to common variables, like time trends and macroeconomic indicators, when the estimated number of factors equals the number of regressors. I reexamine estimation of educational expenditures on standardized test performance and find significantly larger effects of educational spending compared to fixed effects regression. These estimates are also reported up to reasonable precision, suggesting that applied researchers may not be adequately controlling for heterogeneity in their analyses.

One important direction for future work concerns the overestimation of p ₀. It is known that CCE is robust to K + 1 > p ₀. Moon and Weidner (2015) prove that principal components estimation is also robust to overestimating the number of factors, provided T is large. However, while there is ample simulation evidence suggesting the robustness of QLD to such a failure, a formal proof is lacking. It would also be useful to investigate the robustness of the QLDP estimators to failure of the reduced form equation in Assumption 2.