Quantile Difference in Differences with Time-Varying Qualification in Panel Data

2.1 Setup

Consider a DID panel model with three time periods given by t = 1, 2, 3 in which treatment is applied in t = 3 to a group of qualified individuals. In the standard framework, the qualification indicator Q is assumed to be time-constant. However, in practice, Q is often a time-varying variable such as income, wealth, age, number of children, or residential location. Therefore, the time-varying qualification status is denoted with time subscript t, Q _t . In the presence of time-varying qualification, individuals are divided into four groups: in-stayers (Q ₂ = 1, Q ₃ = 1), in-movers (Q ₂ = 0, Q ₃ = 1), out-movers (Q ₂ = 1, Q ₃ = 0), and out-stayers (Q ₂ = 0, Q ₃ = 0).

Let Y _it be the outcome of interest for individual i at time t and let X _it be covariates including time-constant ones C _i and time-varying ones W _it . The treatment indicator is given by D _it = Q _it × 1[t = 3]. X _it is realized first, then D _it at the start of period t, and Y _it at the end of period t. Both X _it and D _it can directly affect Y _it :

For each period t and individual i, the potential outcomes for the treated and untreated are given by Y i t 1 and Y i t 0 , respectively. Observed outcome Y _it is expressed as a function of the potential outcomes and the treatment variable:

This implies that Y i 1 = Y i 1 0 , Y i 2 = Y i 2 0 and Y i 3 = ( 1 − Q i 3 ) Y i 3 0 + Q i 3 Y i 3 1 .

The time-varying qualification can introduce a bias in estimating the treatment effect on the treated in a conventional DID setting. The estimate is identified under the common trends assumption, stating that the difference in untreated potential outcomes is independent of the treatment status:

To see how the presence of movers can introduce bias in the quantile DID estimator, we consider the following panel linear model without covariates as in Lee and Kim (2014) and Lee and Sawada (2020):

where V _it is a time-varying error term. Under the assumption that the error term V is independent of the treatment variable D. Under this model, and the assumption that β _q ≠ 0, the presence of movers (in-movers and out-movers) implies:

Therefore, the assumption Y 3 0 − Y 2 0 ⫫ D 3 , is violated, and there is a need for a new estimator accounting for movers in order to estimate quantile treatment effects. In the next section, we show how to account for this “movers effect” bias in a distributional DID framework.

3.1 Parameters of Interest

For two random variables U and V, let F _U (u) denote the distribution function of U, Supp(U) the support of U, and F _U|V (u) the conditional distribution of U given V. The τ-quantile of U with τ ∈ (0, 1) is defined as:

In the presence of time-varying qualification, in-stayers and in-movers are the only treatment recipients. Therefore, our parameters of interest are the quantile treatment effect for the in-stayers (QTIS) and the quantile treatment effect for the in-movers (QTIM) given by:

3.2 Identification

This subsection provides identification results for the QTIS. Results for the QTIM are similar and thus relegated to the Online Appendix C.

Before stating the identification assumptions, let us introduce additional notations. For any random variable U, let F U | i j = F U | Q 2 = i , Q 3 = j , Y s | i j = Supp ( Y s | Q 2 = i , Q 3 = j ) , Y s | i j ( d ) = Supp Y s d | Q 2 = i , Q 3 = j , Δ Y s | i j = Supp ( Δ Y s | Q 2 = i , Q 3 = j ) , and Δ Y v | i j ( d ) = Supp Δ Y v d | Q 2 = i , Q 3 = j for s ∈ {1, 2, 3}, v ∈ {2, 3}, and i, j, d ∈ {0, 1}. Finally, for all i, j ∈ {0, 1}, define p _i,j = Pr(Q ₂ = i, Q ₃ = j) and p _i,j (x) = Pr(Q ₂ = i, Q ₃ = j|X = x) for all x ∈ Supp(X). We now state our identifying assumptions.

Assumption 1 is an extension of the conditional common trend assumption. It postulates that conditional on covariates X (in this subsection, covariates are assumed to be time-invariant), the distribution of the change in untreated potential outcomes for the in-stayers (treatment group) is identical to that for the control group. Assumption 2 introduces a stability restriction on the copula function of their marginal distributions C Δ Y 3 0 , Y 2 0 | Q 2 = 1 , Q 3 = 1 ( ⋅ , ⋅ ) . In other words, it requires that the dependence between the marginal distribution F Δ Y 3 0 | 11 and F Y 2 0 | 11 is identical to the dependence between the distribution F Δ Y 2 0 | 11 and F Y 1 0 | 11 , implying that when the dependence structure in the past period is obtained, we can use it to describe the dependency in the current period. An interpretation of this assumption is that if the upper tail of the distribution experiences, for example, the highest increases in an outcome variable from the first to second periods, this should also hold from the second to third periods.

Motivation for using Assumption 1 under time-varying qualification is the following. It allows a case where unobservable covariates can be distributed differently over time within each group, in-stayers, in-movers, out-movers, and out-stayers, while a leading quantile method by Athey and Imbens (2006) assumes that the distribution of unobservable covariates is different across groups but does not change over time. Because, in our setting on time-varying qualification, in-movers and out-stayers are those that have changed their qualification status over time for some observable and/or unobservable reasons, their distributions on unobservable covariates are likely to change over time. In addition, our Assumption 1 is conditional on covariates rather than unconditional. This allows, for example, the impact of increasing food subsidies on expenditure share depending on a household’s characteristics, which will be discussed more in Section 4.

Motivation for using Assumption 2 is the same as Callaway and Li (2019): recovering the dependence between untreated potential outcome for treated units in initial period and the change in untreated potential outcome for treated units by using that in the previous period for obtaining point identification. However, utilizing Assumption 2 for in-stayers seems to be reasonable in our setting. If we rather assumed the CSA for all groups altogether, then including units that have changed their qualification status over time (i.e. in-movers and out-moves) would be likely to violate the assumption.

Assumption 3 ensures the continuous distribution of potential outcome variables and their differences. The first part of Assumption 4 states that there is a positive probability that a household is in-stayer. The second part postulates that there is a positive probability for a household with covariates x for any value of x, to be part of the out-stayers group.

Given these assumptions, we now state the main identification results of the paper. The proof can be found in the Online Appendix B1.

Theorem 1 identifies the counterfactual distribution of untreated outcomes for in-stayers. The intuition behind this result is simple. Assumption 1 allows us to identify the distribution of Δ Y 3 0 | Q 2 = 1 , Q 3 = 1 as the distribution of Δ Y 3 0 | Q 2 = 0 , Q 3 = 0 . Since the outcome variable of the untreated in t = 3 is equal to the sum of the change from period t = 2 to t = 3 plus the value in period t = 2, i.e. Δ Y 3 0 + Y 2 0 = Y 3 0 , the distribution of Y 3 0 | Q 2 = 1 , Q 3 = 1 can be identified by combining the copula stability and the remaining assumptions. Equation (3) can be interpreted as a weighted average of the distribution of the change in the outcome variable by out-stayers. More weight is put on households in the control group, whose covariates are similar to those in the treatment group, i.e. a higher p ₁₁(X)/p ₀₀(X).^[2]

The following example provides a situation where Assumptions 1 and 2 hold. Its proof is relegated to the Online Appendix B1.

The model imposes assumptions only on how untreated potential outcomes are generated as in Example 2 of Callaway and Li (2019), thus allowing individuals to select into treatment due to anticipated treated potential outcomes in an unrestricted way. The untreated potential outcomes also depend on the qualification status as in the model (M ₀) in Lee and Kim (2014). In this model, we show that Assumptions 1 and 2 hold.

It is possible in some cases that the Copula Stability Assumption (CSA) holds only conditional on covariates. This conditional assumption states that the copula between the difference in untreated potential outcomes and the initial level of untreated potential outcomes for the treatment group (in-stayers) remains the same over time after conditioning on covariates. We show in the next proposition that even under the conditional CSA, the quantile treatment effect for in-stayers is still identified.

Using the conditional CSA is advantageous because it allows the path of untreated potential outcomes to change with covariates. An example is when the return to some covariates, such as the return to family size, changes over time. However, with this assumption, nonparametric estimation of the parameters of interest would require estimating several conditional distribution functions, which can be computationally challenging. Therefore, our application in the next section uses Assumption 2 rather than Assumption 2.1. As explained in Section D5 of the Online Appendix D, the CSA is likely to hold without conditioning on covariates in our application.

4.1 Results

Figures 1 and 2 and Table 2 show our main results of this application. Figure 1 shows the QTIS using the food-at-home expenditure share as an outcome variable, and Figure 2 shows the QTIS using the food-at-home expenditure level. The average treatment effect on the treated (the horizontal dotted lines in the figures) is positive and statistically significant. For example, Figure 1 shows that the share of food expenditure at home increases by 2.27 percentage points on average after the increase in SNAP benefits. More importantly, it shows heterogeneous treatment effects over the distribution of household’s expenditure share of food at home, while their difference is statistically insignificant due to large standard errors. A SNAP household at the 20th percentile of the distribution has a coefficient of 0.71, with a standard error of 1.02. In contrast, at the other end of the distribution, a SNAP household at the 70th percentile has a coefficient of 4.89, with a standard error of 1.76, thus being statistically significant.

Figure 1:

QTT on the share of food expenditure at home: ATT 2.27 [s.e. 0.98].

The figure provides estimates of the QTIS with covariates on the effect of an increase in SNAP benefits on food expenditure share at home. The solid line denotes the estimate for each quantile measured on the horizontal axis. The black dotted line is a 95% confidence interval calculated by the bootstrap with 300 iterations. The red dashed horizontal line is the average treatment effect.

Figure 2:

QTT on food expenditure at home: ATT 83.32 [25.86].

The figure provides estimates of the QTIS with covariates on the effect of an increase in SNAP benefits on food expenditure level at home. The solid line denotes the estimate for each quantile measured on the horizontal axis. The black dotted line is a 95% confidence interval calculated by the bootstrap with 300 iterations. The red dashed horizontal line is the average treatment effect.

The magnitude of the estimates is also not negligible. As shown in Table D2 of the Online Appendix D, the average share of food expenditure at home is 21.59% among the SNAP households (combining infra-marginal and extra-marginal households). Using the average share, the estimated effect suggests that a SNAP household at the 70th percentile of the distribution increases its food expenditure share at home by 22.65%, while those at the 20th percentile increases its food expenditure share only by 3.29%.

Similar heterogeneous results are observed when we use the unconditional version of Assumption 1, as reported in Figures 3 and 4 and the final four columns of Table 2. For example, while a SNAP participant at the 20th percentile increases their food expenditure share at home by 0.47 percentage points (statistically insignificant), those at the 70th percentile raise their shares by 4.82 percentage points (statistically significant). The observed pattern that an increment in the food expenditure share is increasing in a percentile of the distribution is not solely due to our identification assumptions. It is also confirmed using another distributional DID method by Athey and Imbens (2006), which is reported in the Online Appendix E.

Figure 3:

QTT on the share of food expenditure at home (no covariates): ATT 2.32 [s.e. 0.97].

The figure provides estimates of the QTIS without covariates on the effect of an increase in SNAP benefits on food expenditure share at home. The solid line denotes the estimate for each quantile measured on the horizontal axis. The black dotted line is a 95% confidence interval calculated by the bootstrap with 300 iterations. The red dashed horizontal line is the average treatment effect.

Figure 4:

QTT on food expenditure at home (no covariates): ATT 55.46 [24.17].

The figure provides estimates of the QTIS without covariates on the effect of an increase in SNAP benefits on food expenditure level at home. The solid line denotes the estimate for each quantile measured on the horizontal axis. The black dotted line is a 95% confidence interval calculated by the bootstrap with 300 iterations. The red dashed horizontal line is the average treatment effect.

The ATT and QTT become smaller when ignoring the changes in qualification status than they would be when considering these changes. For example, the result with all types of households (in-stayers, in-movers, out-stayers, and out-movers) shows that a SNAP participant at the 70th percentile of the distribution has a coefficient of 4.54, and the average treatment effect is 2.02 (not reported). This is probably because new participants, in-movers, should be relatively wealthier and have lower food expenditure shares, thus resulting in a smaller treatment effect.

Finally, we provide supportive evidence for our identification assumptions: (1) the Distributional Difference in Differences (DDID) Assumption, (2) the CSA, (3) the Continuity Assumption, and (4) the Overlap Assumption.^[5] The Online Appendix D5 provides supportive evidence for each of them, suggesting that they are likely to hold.