Constrained Hamiltonian Monte Carlo in BEKK GARCH with Targeting

Introduction

Interest in modeling volatility dynamics of time-series data has been growing in many areas of empirical economics and finance. The management of financial asset portfolios involves estimation and forecasting of financial asset returns dynamics. Analysis of the dynamic evolution of variances and covariances of asset portfolios is used for a number of purposes, such as forecasting VaR thresholds to determine compliance with the Basel Accord.

In a recent article by Caporin and McAleer (2012), henceforth CM, the BEKK model (Engle and Kroner 1995) and the dynamic conditional correlation (DCC) model (Engle 2002) are singled out as the “two most widely used models of conditional covariances and correlations” in the class of multivariate GARCH models. CM provide a deep and insightful theoretical analysis of the similarities and differences between the BEKK and DCC. CM note that DCC appears to be preferred to BEKK empirically because of the perceived curse of dimensionality associated with the latter. In an important insight, CM argue that this is a misleading interpretation of the suitability of the two models to be used in practice, as the full unconstrained comparable model versions are both affected by the curse of dimensionality alike. CM argue that either model is able to do “virtually everything the other can do” and hence their usage should depend on the object of interest: BEKK is well suited for conditional covariances while DCC for conditional correlations.

For either model type, parameter parsimony can be achieved by imposition of parametric restrictions on the full model version (Ding and Engle 2001; Engle, Shephard, and Sheppard 2009; Bilio and Caporin 2009). However, doing so is traded off with model specification issues. For example, Burda and Maheu (2013) show that restricted versions of the BEKK, the scalar and diagonal models, are clearly dominated by the full BEKK version in terms of model fit. Indeed, such parameter restrictions generally operate on the parameters driving the model dynamics.

An alternative way of reducing dimensionality is via so-called targeting which imposes a structure on the model intercept based on sample information. Targeting yields model constraints in a structured way, so that its implied long-run solution of the covariance (or correlation) ensures its independence from any of the parameters driving the model dynamics. Hence, targeting differs from merely setting a subset of parameters to zero as in the scalar and diagonal models. Variance targeting estimation was originally proposed by Engle and Mezrich (1996) as a two-step estimation procedure, whereby the unconditional covariance matrix of the observed process is estimated in the first step, and the remaining parameters are estimated in the second step. However, Aielli (2011) shows that the advantage of targeting as a tool for controlling the curse of dimensionality does not hold for DCC models, as the sample correlation is an inconsistent estimator of the long-run correlation matrix. CM also argue that targeting can be rigorously applied only to BEKK and its use in DCC models is inappropriate.

At the same time, enforcing positive definiteness of the conditional covariance matrices in the BEKK with targeting implies a set of model constraints that are nonlinear in parameters and, according to CM, “extremely complicated, except for the scalar case” (p. 742). Furthermore, the model fit of the targeted BEKK relative to the full BEKK remains an open empirical question: does parameter dimensionality reduction via targeting trade off with a model fit that is substantially worse, as in the case of the scalar and diagonal BEKK? Perhaps due to these difficulties and unknowns the BEKK with targeting has not been used extensively in practice, despite its potential benefits.

In this article, we suggest an approach to estimation and inference of the BEKK model with targeting based on Constrained Hamiltonian Monte Carlo (CHMC) (Neal 2011), a recent statistical method that deals very effectively with complicated nonlinear model constraints in the context of relatively high-dimensional problems and computationally costly log-likelihoods. It is particularly useful in cases where it is difficult to accurately approximate the log-likelihood surface around the current parameter draw in real time necessary for obtaining sufficiently high acceptance probabilities in importance sampling (IS) methods, such as for recursive models in finance. Complicated constraints pose further challenge for IS-based methods. In such situations, one would typically resort to random walk (RW) style sampling that is fast to run and does not require the knowledge of the properties of the underlying log-likelihood. Jin and Maheu (2012) used the RW sampler for covariance targeting in a class of dynamic component models. However, RW mechanisms can lead to very slow exploration of the parameter space with high autocorrelations among draws which might require a prohibitively large size of the Markov chain to be obtained in implementation to achieve satisfactory mixing and convergence. CHMC combines the advantages of sampling that is relatively cheap with RW-like intensity but superior parameter space exploration.

CHMC falls within a general class of Markov chain Monte Carlo (MCMC) methods that can be used under both the Bayesian and the classical paradigm, applied to posterior densities or directly to model likelihoods without prior information (Chernozhukov and Hong 2003). To the best of our knowledge, the constrained version of Hamiltonian (or Hybrid) Monte Carlo (HMC) has not yet been used in the economics literature. We elaborate in detail on the CHMC procedure, its application to the BEKK with targeting, and its usefulness in applicability to a wider class of problems of similar nature. We also contrast CHMC performance with RW sampling.

Using CHMC, we perform a model comparison of the BEKK with targeting and the full BEKK, providing evidence that favors the BEKK with targeting in terms of marginal likelihood. Thus, we show that the BEKK with targeting presents an effective way of reducing parameter dimensionality without compromising the model fit, unlike the scalar or diagonal BEKK. The model comparison is conducted in the context of an application concerning a multivariate dynamic volatility analysis of a foreign exchange rate returns portfolio. We believe that the suggested estimation approach and comparison evidence, along with the provided computer code for implementation, will encourage wider use of the BEKK model with targeting as a valuable tool for analysts and practitioners.

The remainder of the article is organized as follows. Section 2 describes the BEKK model and the targeting constraints. Section 3 elaborates on CHMC. Section 4 introduces the application and reports on model comparison results. Section 5 concludes.

BEKK with targeting

The BEKK class of multivariate GARCH models was introduced by Engle and Kroner (1995). Their specification was sufficiently general to allow the inclusion of special factor structures (Bauwens, Laurent, and Rombouts 2006). Here, we focus on the BEKK with all orders set to one, which is the predominantly used version in applications (Silvennoinen and Teräsvirta 2009). Let be a vector of asset returns with and denote the information set as . Under the BEKK model, the returns follow

[1]

[2]

is a positive definite conditional covariance matrix of given information at time , C is a lower triangular matrix, and A and B are matrices. We maintain a Gaussian assumption and a zero intercept for simplicity. The total number of parameters in this model is , collected into the vector .

The BEKK model with targeting is defined as

[3]

where

There are two types of constraints on the model that need to be taken into account. The first type, covariance stationarity constraints, is generally simple to impose, as shown by Engle and Kroner (1995). The second type, positive definiteness of the conditional covariance matrices, is satisfied by construction in the full BEKK version, but becomes difficult to deal with in the targeted case. CM note that one way in which positive definiteness of can be guaranteed is by imposing positive definiteness on but note that such constraint is complicated. Indeed, Figure 1 shows the effect of this constraint on the parameter space on an example of a log-likelihood plot over the vector field of covariance parameters where for (left) and (right). In the white areas not covered by a contour plot, the constraint is violated. The QMLE argmax is on the parameter space boundary which complicates inference. Implementation of this constraint is relatively cheap, since only one constraint check is required per likelihood evaluation, but its consequences are unfavorable for further QMLE analysis. An alternative way of imposing positive definiteness of that we adopt here is to check that all its eigenvalues are positive for each t and take corrective action in the sampling procedure if the constraint is violated. The effect of this constraint on the parameter space is shown in Figure 2 which plots the log-likelihood around the mode over the same vector space as in Figure 1. In our application, we did not detect a parameter without a well-defined mode due to this constraint.

Figure 1:

Targeting constraints on the intercepts.

Figure 2:

Targeting constraints on .

Nonetheless, an occurrence of an argmax on or very close the parameter space boundary cannot be ruled out a priori in general. Furthermore, the log-likelihood surface can be highly asymmetric, as illustrated in Figure 3 on the case of a contour plot over the space of parameter vector for and for The asymmetry appears more severe in higher dimensions. Due to both of these features, the accuracy of QMLE asymptotic inference, based on a Fisher information matrix that is symmetric by construction, is questionable. MCMC-based procedures, including CHMC presented in the next section, provide an alternative estimation and inference approach for either constraint case, unfettered by the aforementioned likelihood irregularities.

Figure 3:

Likelihood Kernel skewness.

3 Constrained Hamiltonian Monte Carlo

The original HMC has its roots in the physics literature where it was introduced as a fast method for simulating molecular dynamics (Duane et al. 1987). Since then, it has become popular in a number of application areas including statistical physics (Akhmatskaya, Bou-Rabee, and Reich 2009; Gupta, Kilcup, and Sharpe 1988), computational chemistry (Tuckerman et al. 1993), or a generic tool for Bayesian statistical inference (Neal 1993, 2011; Ishwaran 1999; Liu 2001; Beskos et al. 2010). HMC and its constrained version, CHMC, apply to a general class of models, nesting the BEKK, that is parametrized by a Euclidean vector for which all information in the sample is contained in the model likelihood assumed known up to an integrating constant. Formally, this class of models can be characterized by a family of probability measures on a measurable space where is the Borel -algebra. The purpose of MCMC methods is to formulate a Markov chain on the parameter space for which, under certain conditions, is the invariant (also called “equilibrium” or “long-run”) distribution. The Markov chain of draws of can be used to construct simulation-based estimates of the required integrals, and functionals of that are expressed as integrals. These functionals include objects of interest for inference on such as quantiles of

The Markov chain sampling mechanism specifies a method for generating a sequence of random variables starting from an initial point in the form of conditional distributions for the draws Under relatively weak regularity conditions (Robert and Casella 2004), the average of the Markov chain converges to the expectation under the stationary distribution:

A Markov chain with this property is called ergodic. As a means of approximation, we rely on large but finite number of draws in which the analyst has the discretion to select in applications.

can be obtained from a given (economic) model and its corresponding likelihood . Typically, has a complicated form which precludes direct sampling in which case the Metropolis–Hastings (M–H) principle is usually employed for from see Chib and Greenberg (1995) for a detailed overview. Suppose we have a proposal-generating density where is a proposed state given the current state of the Markov chain. The M–H principle stipulates that be accepted as the next state with the acceptance probability

[4]

otherwise Then, the Markov chain satisfies the so-called detailed balance condition

which is sufficient for ergodicity. is the probability of the move , if the dynamics of the proposal-generating mechanism were to be reversed. The proposal-generating density can be chosen to be sampled easily, even though may be difficult or expensive to sample from. The popular Gibbs sampler arises as a special case when the M–H sampler is factored into conditional densities. The proposal draws from in eq. [4] are generated in one step.

In contrast, HMC uses a whole sequence of proposal steps whereby the last step in the sequence becomes the proposal draw. This facilitates efficient exploration of the parameter space with the resulting Markov chain. The proposal sequence is constructed using difference equations of the law of motion yielding high acceptance probability even for distant proposals. The parameter space is augmented with a set of independent auxiliary stochastic parameters that fulfill a supplementary role in the proposal algorithm, facilitating the directional guidance of the proposal mechanism. The proposal sequence takes the form starting from the current state and yielding a proposal The detailed balance is then satisfied using the acceptance probability

[5]

In CHMC, constraints are incorporated into the HMC proposal mechanism via “hard walls” representing a barrier against which the proposal sequence, simulating a particle movement, bounces off elastically. Constraints thus do not provide grounds for proposal rejection, eliminating any associated redundancies. Heuristically, the constraint is checked at each step of the proposal sequence, and if it is violated then the trajectory of the sequence is reflected off the hard wall posed by the constraint. This facilitates efficient exploration of the parameter space even in the presence of highly complex parameter constraints. We further synthesize the technical principles of CHMC in a generally accessible form in Appendix A. In the next section we apply CHMC to the task of model comparison of the full BEKK and the BEKK with targeting.

4 Application and model comparison

The model comparison is performed in the context of an empirical application with data on percent log-differences of foreign exchange spot rates for AUD/USD, GBP/USD, CAD/USD, and EUR/USD, from 2000/01/04 to 2011/12/30, total of T = 3,009 observations. We consider cases with two, three and four variables () with data used in the order presented. The associated parameter dimensionality for the full BEKK is 11, 24, and 42, and for the BEKK with targeting 8, 18, and 32, respectively. A time-series plot of the four series is shown in Figure 4, and summary statistics are provided in Table 1 in Appendix B. The sample mean for all series is close to 0, and skewness is small. The sample correlations indicate that all series tend to move together.

For identification, the diagonal elements of C and the first element of both A and B are restricted to be positive (Engle and Kroner 1995). All priors, other than the model identification conditions, are set to be diffused. In the implementation, we utilize the analytical expressions for the gradient of the BEKK log-likelihood from Hafner and Herwartz (2008), which in the case of the BEKK with targeting is subject to the targeting constraints [3]. Although numerical estimates of the gradients could also be used in forming the proposals, evaluation of analytical expressions increases the implementation speed. The initial in the GARCH recursion is set to the sample covariance. Starting from the modal parameter values, we collect a total of 50,000 posterior draws for inference, with 5,000 burnin section. The length of the proposal sequence was set to for the cases respectively, and the stepsize tuned to achieve acceptance rates close to 0.8.

Table 2 reports model comparison results of the full BEKK and BEKK with targeting in terms of marginal log-likelihood (Gelfend and Dey 1994; Geweke 2005). Since all parameters are integrated out, marginal likelihood does not explicitly depend on the dimensionality of the parameter space and hence is suitable for comparison of models with different dimensions. In all cases, , the evidence strongly favors the BEKK model with targeting over the full BEKK model, and this effect increases with dimensionality of variables The evolution of the mean of the conditional covariances over time is presented in Figure 5 for the full BEKK and in Figure 6 for the BEKK with targeting, in four variables. Both plots are virtually identical when examined closely, indicating a minimal loss of model prediction capability as a result of targeting.

Markov chain convergence diagnostics, obtained using the R package coda (Plummer et al. 2012), for both models and variable dimensions are reported in Table 3 confirming the validity of model inference and comparison. All chains have converged within the burnin section, as evidenced by the reported z-scores (mean, minimum, maximum, and standard deviation) of the Geweke (1992) convergence test standardized z-score and p-values of the Heidelberger and Welch (1983) stationary distribution test. The Raftery burnin control diagnostic (Raftery and Lewis 1996) confirms the sufficiency of the length of the burnin section. The CPU run time, also reported in Table 3, took in the order of hours and was virtually equivalent to the wall-clock run time. All chains were obtained using Fortran code with Intel compiler on a 2.8 GHz Unix machine.

The advantages of HMC-based procedures over RW sampling have been well documented (Neal 2011; Pakman and Paninski 2012). We illustrate the CHMC versus RW comparison in Figure 7, showing the trace plot of the Markov chain for the conditional variance parameter in the BEKK with targeting, with 4 variables and 32 parameters. The CHMC trace (left) mixes very well, exploring the tails of the likelihood kernel, while the RW trace (right) stays within a relatively narrow band with minimal tail exploration and without any signs of convergence. Overall, our results support the use of the BEKK with targeting, and CHMC provides a suitable method to sample effectively from its likelihood kernel with nonlinear constraints.

5 Conclusions

In this article, we suggest an effective approach for estimation and inference on the BEKK GARCH model with targeting. The approach is based on CHMC, a recent statistical technique devised to handle nonlinear constraints in the context of relatively costly and irregular likelihoods. Based on a model comparison with the unrestricted version of the BEKK, we present evidence favoring the BEKK with targeting in terms of marginal likelihood. Due to its potential and applicability to similar types of problems, we detail CHMC in a generally accessible form and provide computer code for its implementation. We also provide a comparison of the CHMC and RW sampling. We believe that the elaborated estimation approach and comparison evidence will encourage wider use of both the BEKK model with targeting and CHMC as valuable tools for analysts and practitioners.