The Impact of Forecast Errors on Fiscal Planning and Debt Accumulation

2.1 Data

For our analysis, we use the AKS medium run tax revenue forecasts for the entirety of German states. We obtain these data from press releases of the German Federal Ministry of Finance that were published after each AKS meeting. The AKS medium run forecasts comprise estimates for the current year and the following four years. While the AKS as an independent institution exists since 1955, the complete set of estimates for all five horizons is only available from 1972 onwards. Our sample therefore starts in 1976. In recent years, the AKS conducts two forecasting rounds each year, one in spring and one in autumn. We take the spring forecasts from each year because during the period from 1986 to 2011 there were no medium run projections in autumn. The spring forecasts are usually published in May. In the years before 1986, however, the release of the first forecast in each year varied between February (1972, 1978) and August (1973, 1975–1977). Nevertheless, our setup offers the best possible comparability of the forecasts over time and horizons. All other (ex post) fiscal data are obtained from the German Federal Statistical Office.^[7]

We focus on the entirety of German states for the simple reason that even though the AKS also divides its forecast of total tax revenues into the tax revenues of individual states, these calculations are destined exclusively for the state governments and not made publicly available.^[8] The subsequent own fiscal forecasts of the individual states are publicly available. However, since the states are not bound to the input of the AKS their forecasts are prone to political considerations (Bischoff and Gohout 2010) and therefore not useful for our analysis. Due to the reunification the overall number of German states increased from 11 to 16 in October 1990 while the forecasts made in the late ’80s for the early ’90s refer to the former 11 states only. For this reason, we stick to the entirety of 11 states until the year 1994. This is possible, since the AKS separately reported forecasts for the old 11 as well as the five new states in the first years after the reunification. From 1995 onwards we then include all 16 states.

Figure 1 shows the revenue forecast errors of the AKS at different horizons, expressed in relation to nominal GDP. The forecast errors are defined as the difference between the ex post tax revenues in a given period t and the real-time forecasts for the tax revenues in t made in period t − i. As expected, the forecast errors are usually smaller the shorter the forecast horizon. The mean absolute error decreases from around 0.8 for the four-year ahead forecast to nearly 0.1 for the current year forecast (Table 1). The mean error for the current year forecast and the one-year ahead forecast is positive but small, which means that at these horizons revenues are on average slightly underestimated. At longer horizons (two-, three- and four-year ahead forecast) the mean error is considerably negative, which means that on average revenues are overestimated. We will get back to the notable upward-bias of the medium run forecasts in Section 4.^[9] Chronologically, we observe periods in which tax revenues were overestimated especially around the years 1981–1985 and 1997–2005. By contrast, tax revenues in the late ’80s and early ’90s as well as in the more recent years from 2011 onwards were underestimated.

Figure 1:

Revenue forecast errors 1976–2017. Revenue forecast errors in relation to nominal GDP at different forecast horizons.

2.2 The Baseline Regression Model

We denote the tax revenue forecast made in year t − i for the year t, expressed in relation to nominal GDP Y _t , by r _t|t−i . Accordingly, f _t|t−i  = (r _t  − r _t|t−i ) denotes the corresponding forecast error. We are interested in the effect of the forecast error f _t|t−i at different forecast horizons t − i, i = 0,1, … ,4, on the final primary balance-GDP ratio p _t . The primary balance is defined as overall revenues, mostly consisting of tax revenues, less expenditures (excluding interest payments). Our baseline specification takes the form:

The vector x _t comprises additional explanatory variables. We consider an estimate of the output gap (g _t ) based on the Hodrick–Prescott filtered real GDP series and the debt–GDP ratio of the previous year (b _t−1).^[10] Both are typical explanatory variables in the literature on fiscal reaction functions (Golinelli and Momigliano 2009). The vector d _t comprises a set of dummy variables that have a value of one in the years 1992, 1993, and 1994, respectively. We include these dummy variables for the reason that from 1992 to 1994 the forecast errors still refer to the 11 West German states only as (medium run) forecast errors for the five new states are not available yet. GDP and budget balance data, however, refer to all 16 states of reunified Germany.

In addition to our baseline specification with the primary balance–GDP ratio p _t as dependent variable we also consider regressions with the expenditure-GDP ratio e _t as dependent variable (see Appendix A.2). Generally, a regression with expenditures on the left hand side is prone to simultaneous shifts in expenditures and revenues, for instance due to a change in the size of the government sector or federal structures. Such a regression is therefore more likely to be affected by omitted variables or structural instabilities than our baseline regression model with the primary budget balance on the left hand side.

When estimating (1) there might be problems of serial correlation and endogeneity. Serial correlation of the errors is a large issue, as signaled by a Durbin–Watson statistic of around 0.8. To address it, we follow Plödt and Reicher (2015) and model u _t as an autoregressive process. More specifically, u _t is assumed to follow an AR(2) process with persistence coefficients ρ ₁ and ρ ₂, i.e.

where ε _t is independent over time. The choice of AR(2) errors is motivated by the fact that we still find evidence for serial correlation when employing a specification with first-order AR errors only.^[11] A problem of endogeneity would arise when the budget balance reacts to the output gap and the output gap is affected in turn by fiscal policy. In our context, however, this simultaneity is likely to be minor since the individual states are only marginally engaged in active fiscal policy. We therefore refrain from an instrumental variable approach. In Section 3.2 we perform several sensitivity analyses with respect to our chosen baseline specification.

3.1 Results for Different Horizons

The estimation results for our baseline model are presented in Table 2 (specification A1). Our main finding is that not only the one-year ahead forecast errors but also the two-year and three-year ahead forecast errors have an economically and statistically significant effect on the primary balance-GDP ratio p _t . The strongest impact exhibit the forecast errors made in period t − 1 and t − 2. A revenue forecast error of 1 percentage point in each of these periods leads to change of almost 0.3 percentage points in the actual primary balance-GDP ratio.^[12] A revenue forecast error of 1 percentage point in period t − 3 leads to a change of around 0.1 percentage points. The four-year ahead forecast error f _t|t−4 also has a small effect on the budget balance though the estimated coefficient is not statistically significant at common significance levels. In contrast, the estimated coefficient for f _t|t−0 is nearly zero, which means that the spring forecast for the current year is almost irrelevant for the budgetary outcome.

We include the output gap as a measure for the business cycle to avoid overly persistent residuals and a possible omitted variable bias. The corresponding coefficient is estimated close to zero. The estimated debt coefficient is slightly positive but also not statistically distinguishable from zero. We thus find no strong evidence for consolidation policies of German states in response to past debt. The persistence coefficients are both sizeable and statistically significant, showing that serial correlation of the residuals is indeed an issue in the regression model given by Eq. (1).

3.2 Sensitivity Analyses

We perform several robustness checks to test the sensitivity of our baseline results to alternative model specifications. First, we model the residuals u _t as an AR(1) instead of an AR(2) process (specification A2). We find that the specification with AR(1) errors delivers results that are very close to our baseline specification. Second, we check the temporal stability of our results. To do so, we compare our baseline results with the results of a shorter pre-crisis sample ranging from 1976 to 2008 (A3). The choice of this sample is also motivated by the fact that the German debt brake was introduced on the federal level after the financial crisis and some of the Länder have also introduced a debt brake in the following years.^[13] We find that the estimated coefficients based on the shorter pre-crisis sample are broadly similar to the baseline estimates, indicating that the impact of the forecast errors has not changed significantly over time and that our baseline results are not noticeably driven by extraordinary events subsequent to the financial crisis. Third, we add both, the contemporaneous output gap as well as the output gap of the previous year as explanatory variables (A4). In this case, we find an even stronger impact of the two-year ahead forecast error f _t|t−2 and a smaller and statistically insignificant impact of f _t|t−1. The results of this specification are again only marginally affected when using AR(1) errors or a shorter pre-crisis sample (specification A5 and A6, respectively).

In Appendix A.1, we present a further set of robustness checks, in particular with regard to the handling of serial correlation, potential issues when dividing the dependent and the independent variables by a common denominator, and the use of a cyclically-adjusted primary balance variable. Overall, we conclude from all of these sensitivity analyses that our main findings are remarkably robust across alternative specifications and samples.

3.3 Cross-Horizon Mean Error and Asymmetric Reactions

So far, our baseline specification has included all the individual forecast errors f _t|t−i to investigate the importance of each forecast horizon t − i. In this subsection we consider instead the cross-horizon mean forecast error as explanatory variable. This allows us to quantify the impact of the average error across all forecast horizons concerning the tax revenues in a specific period t. Given our relatively small sample size, it also allows us to obtain a more parsimonious model. In particular, we calculate f t | t − i 04 = 1 5 ∑ i = 0 4 f t | t − i and estimate

The corresponding results are displayed in Table 3 (specification B1). The estimated coefficient for f t | t − i 04 equals around 0.8, with a value of one being within the one-standard error band. When considering the pre-crisis sample 1976–2008 (B2), we find an even higher point estimate of around 0.9. Moreover, adding both, the contemporaneous output gap as well as the output gap of the previous year as explanatory variables only marginally affects the results (B3).

These estimates show that the overwhelming part (around four-fifth) of the cross-horizon mean forecast error for a specific period ultimately translates into the final budget balance. They also suggest that revisions in revenue forecasts lead only to a minor degree to adjustments in the government’s expenditure plans. To empirically support this argument, we additionally regress the expenditure–GDP ratio on the cross-horizon mean forecast error (see Appendix A.2). We indeed find that the reaction of expenditures to the mean forecast error is very small and not statistically distinguishable from zero.^[14]

Next, we have a closer look at the budget impact of the average error over the medium run horizons t − i,i = 2,3,4 versus the short-run horizons t − i,i = 0,1. For this specification we calculate f t | t − i 24 = 1 3 ∑ i = 2 4 f t | t − i and f t | t − i 01 = 1 2 ∑ i = 0 1 f t | t − i and include both as explanatory variables in our model (specification B4). We find a strong effect of the medium run mean forecast error on the primary balance–GDP ratio, with an estimated coefficient for f t | t − i 24 of almost 0.5. Moreover, we find a slightly lower coefficient for f t | t − i 01 , which is not statistically significant at common levels. This result again demonstrates the importance of the medium run forecasts for the final budgetary outcome.

Finally, we explore potential asymmetric reactions of the primary balance–GDP ratio to forecast errors. To do so, we consider a model specification that allows for different reactions to positive and negative cross-horizon mean forecast errors. This specification takes the form:

Our estimation results do not point to notable asymmetric reactions (B5). The difference between the reaction to positive and negative forecast errors is very minor and not statistically significant. These results suggest that in the long term forecast errors contribute to an increase in debt levels, if forecasts are overall too optimistic. On the other hand, forecast errors that go in both directions may cancel each other out. Hence, the results of this specification underline the importance of unbiased forecasts.