On Defence Expenditure Reduction: Balancing Between Austerity and Security in Greece

Appendix A: The ANNs background

The theoretical concepts behind ANN are nowadays quite known in the research community. ANN are essentially directed graphs with nodes being formed in different layers. A node in an ANN is the basic computational element called neuron. Each neuron within a layer may be connected to other neurons in the same, previous or subsequent layer (depending on the type of the ANN) with weights which denote the strength or influence which the feeding node exercises on the receiving end. These connections are similar to the synapses observed between biological neurons in the human brain.

This paper uses the widely known feed-forward Multi-Layer Perceptron (MLP) ANN (McCulloch and Pitts 1943) as its basic prediction model, which organises its neurons in various layers Andreou and Zombanakis (2000). More specifically, the networks created and tested in this work consist of one hidden layer separated into three parallel sub-layers called slabs. Each slab is activated by a different function (Figure A1), something which gives birth to the so-called Multiply Activated MLP.

Figure A1

A multiply-activated MLP ANN logical architecture.

One of the difficulties one has to deal with is the selection of the most appropriate topology, that is, the number of nodes present in each slab of the hidden layer. In most of the cases a trial-and-error approach is followed based on forecasting accuracy. In this sense our approach develops different ANN structures trained using variations of learning schemes that were based on different activation functions listed in eq. (A1) to (A5) below. Then the best topologies were selected for further experimentation on the basis of specific error metrics.

A k-fold cross validation was performed during the learning process of the ANN, with k being equal to four. The training and validation datasets were formed as follows: Given a time series x={x(t): 1≤t≤N} we define two sets, the training set xtrain={x(t): 1≤t≤T}, and the test set xtest={x(t): T<t≤N}, where N is the length of the data series. The xtrain set is used to train the network until a certain level of convergence has been reached based on the error criterion. Next, the xtest set is employed to test for overfitting, that is, to check if the networks ability to learn was not the result of over-exposure to the training set and that the knowledge of the network was indeed learned and not memorised. The training process was repeated four times, each time splitting the data series in different parts (folds). Therefore, a different test set was inserted in each fold. This, in practise, enables rotation between samples participating in the training and testing sets and ensures that each data sample participates in exactly one test set in the whole experimentation process. We are thus able to address the problem of artificial performance which results from the participation of “suitable” samples for the model in an efficient way, thus enhancing its robustness.

In terms of a more detailed description of the learning process we may point out that each fold splitting produces unique sets of training and testing patterns formed as vectors of input values along with the actual output. During training the input patterns in the xtrain set are iteratively presented to the network aiming to map the underlying relationship, if any, between inputs and outputs to the best possible extent. The output of the network value is compared to the actual output for each pattern with the target being to minimise the distance between the two. This minimisation takes place through the modification of the weights of the connections between neurons. This iterative process is completed when one of the following stopping criteria is met: (a) a low predefined error value has been achieved, (b) training ceases to converge, (c) the maximum number of iterations has been reached. A successfully trained network is then tested against the independent set of vectors in the xtest set anticipating reasonably correct (similar) results against the test suite. The performance of the ANN is thus evaluated using a portion of the dataset which has never been seen by the network, once this set has not participated in the learning process at all (Azoff 1994). If the network was successfully trained to recognize the structure of the input series and their mapping to the actual output, by learning rather than memorizing it, then it must perform well when the testing set is presented. Otherwise, performance will be extremely poor on these “new” data values as a result of overfitting. Training of MLP is usually performed in a supervised manner using the error back-propagation algorithm (Rumelhart and McLelland 1986).

Appendix B: The long-run equilibrium: debt, investment and the GDP

The cointegration analysis aims at tracing the existence of common trends between two or more non-stationary variables, in order to derive a relationship that leads to at least one common stationary root, by solving the eigenvalue problem of the common variance-covariance matrix. In other words, the cointegration vector, which consists of the weights assigned to the variables of the system, results in stationary residuals, thus allowing the estimation of the short-run dynamics empowered by the inclusion of adjustments towards the long-run equilibrium. The Johansen’s cointegration analysis framework uses the trace and maximum eigenvalue tests in order to establish whether there exist any cointegartion relationships between the series, estimated in a maximum-likelihood context. The coefficient estimation framework of the Engle and Granger (1987) bivariate cointegration analysis, on the other hand, is based on least-squares estimations. In the present paper both these frameworks are employed.

Sticking to the specification employed in the ANN we may now supplement the analysis by the decomposition of the long-run equilibrium and the short-run dynamics between the defence expenditure on equipment (EDEF) and the GDP. To do so we form (B1) as follows:

reminding that GDP stands for the growth rate of the Greek Gross Domestic Product, DEBT is the stock of the total external debt of the country and INV is total investment spending, all expressed in logarithmic form. We have chosen to exclude the defence procurement EDEF from the long-run version of the function due to the disagreement reflected by contributions like Thompson (1974), Benoit (1978) and (Dunne, Smith, and Willenbockel 2005).⁹

Both explanatory variables are expected to be related to the GDP, because both investment activity and foreign borrowing aim at raising output pointing to a long-run direct relationship of the GDP growth of Greece with the country’s external debt and the total investment expenditure. As already indicated, these are expressed in terms of GDP shares while the estimation period ranges from 1971 to 2011.

The stationarity test suggests that the series have a unit root (Table B1) meaning that we shall need to resort to the first differences of these series to form the short-run relationships. We next proceed with examining the existence of cointegration effects among the series of debt, GDP and investment using the Johansen cointegration analysis, suitable for multivariate systems (Johansen 1988 and Johansen and Juselius 1990). Table B2 reports the trace and the maximum eigenvalue statistics and indicates a single stationary linear combination among the three variables at a 1% confidence interval.

The test for the composition of the cointegration vector (Johansen and Juselius 1992) reported in Table B3 indicates that only the GDP series is exogenous in the long-run structure thus reflecting causality patterns among the GDP and the other two variables and indicating that the GDP variable should be used for normalization of the cointegration vector.

An accurate estimation of the long-run coefficients requires the use of OLS regressions (Engle and Granger 1987). The stationary linear combination among the three variables we have established means that the OLS estimates will serve for the proper identification of the long-run relationship between GDP (as the normalization variable), debt and investment. The results are reported in Table B4 and mean that the long-run relationship can be shown as a cointegrating vector as follows:

According to the results of the unit root tests of Table B1, equation (B2) constitutes a stationary linear combination of the series of GDP, investment and debt and represents, therefore, a long-run equilibrium relationship. There exist, therefore, significant co-movements among the three series in the long run, indicating that the Greek GDP is significantly interlinked with debt and investment.

Appendix C: The short-run dynamics of the system

The vector error correction model (VECM) used consists of a VAR in which the GDP, the debt, the total investment flows and the defence expenditure appear in their first differences (the prefix D and the suffix S denoting first differences and GDP shares of the corresponding variables) together with an error correction term that captures adjustment dynamics towards the long-run cointegrating relationship specified by equation (B2). The VAR framework aims at treating co-linearity issues by capturing simultaneous effects among the underlying variables in the variance-covariance matrix. This isolates any co-variance effects that may influence the dynamics of the variables involved, thus enabling us to focus on the direction of the causality of the underlying relationships.

The lag structure used in the VECM, in order to capture the short-run dynamics of the system, are chosen by taking the system that minimizes the values of the Akaike and Bayesian information criteria (AIC and SIC, respectively). Next, after having chosen the lag structure of the system we estimate the coefficients of the VECM, that are used in order to derive lead-lag relationships among the three variables, but, also, the adjustment dynamics towards the long-run equilibrium. Finally, the impulse response functions are estimated based on a Choleski-decomposition taxonomy of the expected relationships following the results of the ANN.