Age–Period–Cohort Models and the Perpendicular Solution

1.1 The perpendicular solution in the literature

The solution that is perpendicular to the null vector appears, in various guises in the methodological literature, as a solution to APC identification problem. Kupper and Janis and colleagues (1980, 1983, 1985) showed that this solution is equivalent to the principle component regression solution used in the rank deficient by one situation (retaining as regressors the components with non-zero eigenvalues), and they demonstrated that this solution is equivalent to that generated by the Moore–Penrose generalized inverse. The perpendicular solution to the APC model is equivalent to the intrinsic estimator introduced by Fu and his collaborators (2000; Yang et al., 2004, 2008). Fu (2000) derived the intrinsic estimator using a ridge regression approach (as the “shrinkage penalty” diminishes, the ridge regression solution converges toward the perpendicular solution). Tu et al. (2013) added partial least squares to this list of perpendicular solutions and emphasized that this relationship holds when the maximum number of components is extracted for partial least squares and for principle component regression. Browning et al. (2013) demonstrated that the maximum entropy solution is essentially equivalent to the intrinsic estimator solution (the perpendicular solution). Though not specifically addressing APC models, Press et al. (1992) noted that the perpendicular solution is the singular value decomposition solution and produces a minimum norm solution. Various versions of the perpendicular solution have been used in dozens of recent studies during the past decade with dependent variables ranging from blood pressure (Tao et al., 2013) to hepatocellular carcinoma mortality (Tu et al., 2011) from marijuana use (Miech and Koester, 2012) to religious beliefs and activities (Schwadel, 2011). Then in a different context these methods end up being rediscovered again (Fukuda, 2014).

These solutions are equivalent. They produce the solution that is perpendicular to the null vector, which I label as “perpendicular solution.” Given the common use of the perpendicular solution in empirical studies and its multiple discoveries and rediscoveries, it is important to explicate the properties of this particular solution. What are its algebraic characteristics? What are its geometric characteristics? What are its statistical characteristics?

2.1 The model

The standard APC multiple classification model can be conceptualized as modeling an age–period table. Table 1 shows the typical setup using Frost’s (1939) tuberculosis data for males’ aged 0–9 to 60–69 for the periods 1880–1930. The intervals between periods are the same as the number of years in each age group: for example, age groups 0–9, 10–19,…, 60–69 are matched with the periods 1880, 1890,…, 1930. The cells contain the age–period-specific male tuberculosis mortality rates per 100,000 male residents (Frost did not supply counts) as the dependent variable. With this setup, the earliest birth cohort (1810–1819) is represented by the bottom left-hand cell (475 per 100,000), the next earliest birth cohort (1820–1829) is represented by the two cells on the downward diagonal starting with those 50–59 in 1880 and ending with those 60–69 in 1890. We proceed in this manner until the most recent cohort (1920–29) is represented by one observation: the one in the upper right-hand cell of the age–period table. The task is to model these cell values as dependent on the age, period, and cohort effects with age on the rows, periods on the columns, and cohorts on the diagonals.

A standard depiction of the APC multiple classification model is

where the observed value of the dependent variable for the ij th cell of the age–period table, μ is the intercept, ai is the i th age effect, pj is the j th period effect, ck is the k th cohort effect, and εij is the residual for the ij th cell of the age–period table. There are I ages, J periods, and K cohorts, where k=I−i+j. I use analysis of variance coding (effect coding), where the parameters in equation 1 are subject to the following constraints:

When analyzing rates, the natural log of the yij is used as the dependent variable.

Equation [1] can be rewritten in matrix notation as:

where X is the matrix of independent variables with the first column containing all ones for the intercept, the next I−1 columns contain the age-group effects except for the reference category for ages, the next J−1 columns contain the period effects except for the reference category for periods, and the final K−1=I+J−2 columns contain the cohort effects except for the reference category for cohorts. In the I by J age–period table there are 2I+J−3 columns in the X-matrix. The number of rows in the matrix is I×J, one for each cell in the age–period table. The vector b is a column vector of 2I+J−3 elements that correspond to the intercept, age, period, and cohort coefficients (for the non-reference categories). The y vector is an I×J column vector of the observed values for the outcome (dependent variable), and ε is an I×J column vector of residuals.

The age, period, and cohort effects in X are effect coded. Each row corresponds to a particular cell in the age–period table and each column in a row contains a 1 if that column corresponds to the age-group, period or cohort for the cell represented by that row; a row representing a cell that is in the reference category is coded with a minus 1 on the column variables for which it is the reference category. All other entries are coded as zero.

In the APC model, X is not of full column rank; the columns are not linearly independent. This means that there is a vector v that when premultiplied by X produces a zero vector with I×J elements. This vector is referred to as the null vector. It is unique up to multiplication by a scalar, so we may write Xsv=0, where s is a scalar and v is the null vector. ^[2] The null vector plays a major role in this paper. We first use it to derive an important feature of the solutions to the APC multiple classification model. That is, for the just identifying constrained APC solutions all of the best fitting solutions lie on a single line in multidimensional solution space (Kupper et al., 1983; O’Brien, 2011).

For ordinary least squares, a solution is a vector of parameter estimates that minimize the sum of the squared residuals ∑(yij−yˆij)2, where yij is the observed value of the dependent variable and yˆij are predicted (expected) values from the following linear equation that minimize the sum of squared residuals:

In the APC context bc1 represents a solution vector (under the constraint c1) that produces a least squares solution.

Given the rank deficiency of X, the X-matrix post-multiplied by the null vector equals the zero vector. Thus we may write:

where bc represents any of the infinite number of least squares solutions for eq. [4] when X is rank deficient by 1. This means that any solution based on bc1+sv produces the same best fitting solution. These solutions all lie on a line of solutions in multidimensional solution space. We represent this line of solutions as bc=bc1+sv, where bc1 is one of an infinite number of best fitting solutions, and bc represents any of the infinite number of constrained solution vectors that provide a best fitting solution (as s changes values, it produces the line of solutions). The just identifying constrained solutions are related to each other in that they are all on the line of solutions and differ from one another by sv.

This result generalizes to the solutions based on generalized linear models (for example, logistic regression or Poisson regression) where we can write the model as:

where g is a link function, u is the expected value of the outcome variable, and bc1 is a constrained solution to the model (under constraint c1). Here, bc1 is one of the constrained solutions that identifies the model under the constraint and thus provides the best fit for the dependent variable. In this situation, where X is rank deficient by 1, we can write eq. [6] as

The discussion below holds for generalized linear models as well as ordinary least squares.

When we identify eq. [3] using a single constraint such as age 1=age 2 or cohort 1=cohort 2 or using a more complex single constraint such as constraining the period coefficients to have a zero slope (Holford, 1983) or the constraint that produces the solution orthogonal to the null vector (Kupper et al., 1985), these solutions all lie on the line of solutions and fit the outcome variable equally well. We say that these solutions are identified under the particular constraint. Without a constraint they are not identified.

2.2 The null vector

Kupper et al. (1983) provide a convenient method for deriving the null vector when the age, period, and cohort variables are coded using effect coding for the multiple classification model. Based on their presentation, the null vector for the effect coded X-matrix is

The zero is the null vector element for the intercept. The elements of the null vector for the age, period, and cohort effects are within the first, second, and third square brackets in eq. [8]. I and J are the number of age groups and period, respectively: in the first square bracket i=1 to I−1, in the second square bracket j=1 to J−1, and in the third square bracket k=1 to I+J−2. This assumes the final categories are designated the reference categories.

As an example, for a 4×4 age–period matrix the null vector elements for age are −1.5, −0.5, and 0.5; for period they are 1.5, 0.5, and −0.5; and for cohorts the null vector elements are −3, −2, −1, 0, 1, and 2. When we place these elements into the null vector including the zero as the first element for the intercept, the resulting vector premultiplied by X (based on a 4×4 age–period matrix that is effect coded) produces a zero vector.

4.1 Minimum norm

As noted above, the perpendicular solution is a minimum norm solution and this can be considered a statistical property of this solution. The solution vector b that minimizes the distance: ||y−Xb|| is the least squares solution. In the rank deficient by one case, which we encounter in the APC model, this criterion of best fit does not yield a unique solution because any solution on the line of solutions minimizes this distance. Only one solution, however, minimizes ||y−Xb|| and ||b||: that is, b⊥. The same holds true for generalized linear models; except, of course, the fit criterion does not minimize the sum of the squared deviations, but b⊥ is the shortest of the solution vectors on the line of solutions that produces the best fitting model for the specific generalized linear model.

4.2 Smallest variance

Kupper et al. (1983) note that in the special case where the constraint equals the null vector c=v the estimate “deals with the exact linear dependency… by involving only the non-zero eigenvalues associated with the eigenvectors… in its formulation.” They note that since the other constrained estimators “do not deal directly with this multicollinearity problem, one can argue that [this solution] is to be preferred on variance grounds” (p. 2797). The perpendicular solution is perpendicular to the vector that is most highly related to the column vectors in X: that is, the vector that shows that the columns are linearly dependent. Kupper et al. (1983) realized that this would reduce the variance of the estimated coefficients just as principal components regression is designed to do. Although they note the potential of this solution to reduce the variance of the estimates, they emphasize that it could easily lead to more bias than some of the other constrained solutions: “so that the optimal method for choosing c should probably take into account both bias and variance (i.e., mean square error) considerations. Of course, when the squared multiple correlation coefficient R2 is fairly close to 1, a result which seems to occur not infrequently in practice, then bias becomes the main area of concern” (p. 2797). Yang et al. (2004) in appendix B of their paper prove that the perpendicular solution “has a variance smaller than that of any CGLIM [constrained generalized linear model] estimator – i.e., var(b)−var(B) is positive-definite for a nontrivial identifying constraint,” where B represents the perpendicular solution and b represents any of the other just identified constrained solutions.

Given that the constrained solutions all fit the data equally well (while often providing very disparate estimates of the age, period, and cohort effects); this minimum variance property is not a compelling reason for using the perpendicular constraint.

6.1 Provide an unbiased solution for the “generating parameters”

The perpendicular solution does not solve the identification problem in terms of producing unbiased estimates of the parameter values that generated the outcome data. There is probably near universal agreement on this point among statisticians, but it is far too easy for substantive researchers to think that the perpendicular solution somehow solves the identification problem. Given that the X-matrix is rank deficient by one, such a solution does not exist without the researcher imposing a constraint that selects one of the infinite numbers of best fitting solutions on the line of solutions. The perpendicular solution imposes a constraint that is more complex than the constraint imposed by most researchers, but it is a constraint. That constraint determines a particular solution on the line of solutions. Only one of the solutions on the line of solutions is an unbiased estimator of the parameters that generated the outcome data. This is the case even though in the published literature some researchers proceed as if the perpendicular solution should be considered an appropriate estimator of the substantive effects of ages, periods, and cohort (citations in this paper and Luo, 2013).

6.2 Provide age, period, and cohort effects as estimable functions

The perpendicular solution is not an estimable function of the individual age, period, and cohort effects in the classical sense of Searle (1971), Scheffé (1950), and Bose (1944). ^[7] This statement is easy to verify, since the perpendicular solution does not meet the necessary and sufficient condition for an estimable function of the individual age, period, and cohort coefficients derived by Searle (1971), which is equivalent to the condition stated in Scheffé (1950) and based upon Bose (1944). Searle (1971:185) states that a given function “q′b is estimable if and only if q′H=q′,” where q′ is a vector and H=GX′X. Here, G is any one of the generalized inverses that produces a solution on the line of solutions. ^[8] For example, the age 1 coefficient is not estimable. If age 1 corresponds to the second column in the X-matrix, then the q vector is (0, 1, 0, 0, 0,…,0)′ with 2I+J−3 elements and q′H≠q′. Using this necessary and sufficient condition, the age 1 coefficient for the APC model estimated from the perpendicular solution or any other solution is shown to be not estimable (O’Brien, 2014b).

A different criterion for estimable functions was introduced by Kupper et al. (1985): l′v=0, where l is a vector that produces a linear combination of the elements of v. If this linear combination is zero, then the same linear combination of any of the solution vectors is estimable; they produce the same value no matter which constrained solution is used (O’Brien, 2014b). This criterion (as it should) agrees with that of Searle (1971). Yang and colleagues (2004, 2008) have modified the Kupper et al. (1985) criterion by allowing l to be a matrix rather than a vector and 0 to be a zero vector. With these changes in hand, they assert that the perpendicular solution is an estimable function. It is true that each of the rows of their matrix times v is zero, but most of the rows are linear combinations of several elements and not a linear combination consisting of a single age, period, or cohort effect (O’Brien, 2014b). In addition, we can use Kupper et al.’s criterion to show that the linear combination for the age 1 coefficient is not estimable by setting l′=0,1,0,0,…,0 and multiplying it times the null vector. ^[9]Kupper et al. (1985) on page 830 of their paper, demonstrate using their method that not all of the age, period, and cohort coefficients are estimable. For the individual elements of the solution vector, only those with zeroes in the corresponding element of the null vector are estimable (O’Brien, 2014a, 2014b).

6.3 Provide the most representative solution?

There is a claim in the literature that the perpendicular solution is some sort of average or most representative solution in the rank deficient by one situation. Smith (2004) states: “There is also a sense in which the IE [intrinsic estimator] is an average of CGLIM estimates” (p. 116). Press, Teukolsky, Vettering, and Flannery (1992) note that in the rank deficient by one situation: “If we want to single out one particular member of the solution-set of vectors as representative, we might want to pick the one with the smallest length” (p. 62).

I have pondered this claim at length and wish that the authors in the two sources cited above had laid out a convincing rationale for their conclusions. I am convinced by the comments of two external reviewers of this article that the claim of representativeness is difficult to establish. The reviewers noted that an infinitely long line, like the line of solutions, has no middle. In what sense then can the perpendicular solutions somehow be representative of all the infinite number of potential solutions? The other reviewer asked: why should we care about whether the perpendicular solution is representative; or not, if it is not an unbiased estimate of the parameters that generated the dependent variable values?