Efficiency Analysis and Bayesian Neural Networks

1 Introduction

Machine learning techniques are essential tools for practitioners analyzing real-world data, particularly for revealing complex patterns that might otherwise be missed (Hauzenberger et al. 2025). These techniques are highly relevant across various sectors characterized by complex relationships, including, among others, agriculture, healthcare, manufacturing, and energy. This relevance is especially critical in the context of inefficiency measurement, as the precision of performance measures that guide firms, policymakers, and other stakeholders can be compromised by the limitations of conventional modeling approaches (Giannakas et al. 2003). Specifically, when applying the stochastic frontier analysis (SFA) technique introduced by Aigner et al. (1977) and Meeusen and van den Broeck (1977) to quantify inefficiency, researchers often rely on standard functional forms (i.e. Cobb-Douglas) to model the relationship between inputs and outputs. While standard functional forms are widely used for their simplicity and convenience, they may not adequately capture the highly non-linear relationships between inputs and outputs.

In this study, we examine the consequences of ignoring non-linearities in the production frontier when estimating time-invariant and time-varying inefficiencies. To do so, we (1) specify an SFA model that distinguishes between time-invariant and time-varying inefficiency components, (2) model the deterministic part of the frontier using a neural network to capture non-linearities in the input–output relationship, and (3) compare the results with those obtained from a conventional SFA model that assumes a linear Cobb–Douglas functional form for the deterministic part of the frontier.

Accounting for frontier non-linearities is particularly important in sectors where input–output relationships are complex and influenced by external conditions. In the energy sector, which serves as the empirical context of this study, the marginal productivity of inputs like labor, capital, and water can vary significantly depending on weather conditions, resource availability, and the regulatory framework. Such non-linear interactions may lead to sizable deviations from the predictions made by standard functional forms, highlighting the need for more flexible modeling approaches. Neural networks are powerful machine learning tools that have proven to be highly effective as function approximators, learning to map inputs to outputs by adjusting the network’s weights based on the available data (Pendharkar 2023). In particular, the network adjusts its parameters, enabling it to “learn” the underlying function that relates inputs to outputs, even when this function is complex or highly non-linear (Azadeh et al. 2010). This attribute has made neural networks attractive for studies such as those by Azadeh et al. (2010), Konstantakis et al. (2025), Liao et al. (2007), Pendharkar (2023), Wang (2003), and Xuan (2018), which have applied them in SFA to approximate the unknown functional form of the frontier. Additionally, neural networks have been employed to address uncertainty in the relationship between the inefficiency term and socio-economic characteristics, as highlighted in Tsionas et al. (2023).

To date, most of the studies in the field have not adequately addressed the panel structure of their data, particularly with regard to differentiating between time-varying and time-invariant inefficiency components, as suggested by Kumbhakar et al. (2014) and Tsionas and Kumbhakar (2014) in their generalized true random effects model. Additionally, the potential consequences of neglecting the presence of non-linearities between inputs and outputs within both time-invariant and time-varying inefficiency scores remain unexplored. In this study, we address these gaps by integrating a generalized true random effects SFA model with a Bayesian neural network to simultaneously capture both time-varying and time-invariant inefficiencies while accounting for potential non-linear relationships. We also estimate inefficiencies using the conventional Cobb-Douglas specification for the frontier function and compare these estimates with those derived from the proposed neural network model. To illustrate the empirical implications of our approach, we apply both models to a panel dataset from the energy sector. By comparing the results of the neural network-based model with those from the conventional Cobb–Douglas specification, we highlight the practical significance of adopting a more flexible functional form in frontier analysis.

The remainder of the paper is structured as follows. In the next section, we introduce the proposed generalized true random effects SFA model along with the specified neural network structure and estimation procedure. We also present the conventional SFA model based on a Cobb–Douglas functional form and outline the approach used to compare the two models. This is followed by a description of the dataset used in the analysis. The subsequent section presents the results obtained from the proposed neural network-based SFA model and compares them to those derived from the conventional Cobb–Douglas specification. The final section offers concluding remarks.

2 Models and Estimation

2.1 Neural Network SFA Model

The generalized true random effects SFA model for firm i at time t is written as:

(1) y t , i = μ t , i + a i + v t , i − η i − u t , i

where y _t,i is the log-transformed observed output and μ _t,i denotes the deterministic part of the production frontier. Regarding the random variables, a _i is the firm-specific random effect, capturing unobserved heterogeneity, and is assumed to follow a Normal distribution, a i ∼ N ( 0 , ω − 1 ) . The term v _t,i represents the noise component, accounting for random shocks, and is also assumed to be Normally distributed as v t , i ∼ N ( 0 , τ − 1 ) . The inefficiency components, η _i and u _t,i represent time-invariant (thereafter persistent) and time-varying (thereafter transient) inefficiencies, respectively, and are modeled with a Half-Normal distribution, where η i ∼ N + ( 0 , ψ − 1 ) and u t , i ∼ N + ( 0 , ϕ − 1 ) .^[1] The deterministic part μ _t,i is modeled using a neural network that captures the potentially complex, non-linear relationship between inputs and output:

(2) μ t , i = ∑ j = 1 H W 2 j ⋅ f ∑ k = 1 K W 1 k , j ⋅ X t , i , k + b 1 j + b 2

where X _t,i,k represents the k-th log-transformed input variable for firm i at time t, W1 _k,j are the weights from the input layer to the hidden layer of the neural network, W2 _j are the weights from the hidden layer to the output layer, b1 _j and b2 are the biases for the hidden and output layers, respectively, f(⋅) is the activation function applied to the weighted sums, and H is the number of hidden neurons.

The structure of a simple neural network designed for a scenario with three input features (X1, X2, and X3), a single hidden layer containing 2 neurons (H1 and H2), and one output (μ) is illustrated in Figure 1. The network begins with the 3 inputs, which represent the features of the dataset and serve as the starting point for neural computation. These inputs are fed into the hidden layer, where each is multiplied by corresponding weights associated with each connection to the hidden neurons. For each hidden neuron j, the neuron computes a weighted sum of its inputs plus a bias term, expressed as ∑ k = 1 K W 1 k , j ⋅ X t , i , k + b 1 j , which is then passed through an activation function f(⋅) to introduce non-linearity into the model. The outputs from the hidden neurons are then linearly combined to produce the final output μ, using another set of weights W2 _j and a bias term b2. This output represents the neural network’s prediction. Bias terms in both the hidden and output layers play a crucial role in shifting the activation functions, allowing the network to better fit the data by accounting for systematic deviations.

Figure 1:

Structure of the neural network.

Regarding the structure of the neural network in this study, the following decisions are made: 1) the hyperbolic tangent (tanh) function is chosen as the activation function f(⋅), due to its ability to introduce non-linearity into the model (Azadeh et al. 2007). The tanh function maps inputs to a range between −1 and 1, providing a zero-centered output that facilitates balanced training and improves convergence, 2) a single hidden layer is specified, based on the universal approximation theorem, which states that a neural network with one hidden layer can approximate any continuous function (Wang 2003), 3) the number of neurons in the hidden layer H is determined using the heuristic that suggests H should be approximately 2/3 of the size of the input layer plus the size of the output layer (Heaton 2008). This rule of thumb balances model complexity by reducing the risk of overfitting while maintaining sufficient model capacity. Given that the subsequent application includes 5 independent variables and 1 output variable, H is set equal to four.

3 Data

To empirically compare the neural network SFA model with the conventional SFA model, we use the dataset from Kerstens et al. (2019).^[2] The dataset, recorded monthly, includes the output and inputs of 16 Chilean hydroelectric power plants over the entire year of 1997, comprising a total of 192 observations. The output variable is the electricity generated (Q) in Gigawatt-hours (GWh), while the three input variables are: (a) the water volume (M) in thousands of cubic meters, (b) the labor quantity (L), measured by the total number of full-time employees, and (c) the value of capital (K) in real Chilean pesos. The raw observations for the output variable (Q) and the three input variables (M, L, and K) are illustrated analytically in Figures 2, 3, 4, and 5, respectively. In addition to these inputs, the model specification includes an intercept and a monthly time trend to capture temporal variation throughout the year.

Figure 2:

Electricity generated.

Figure 3:

Water volume.

Figure 4:

Labor quantity.

Figure 5:

Value of capital.

4 Results

The utilized sampling scheme involves a total of 700,000 iterations, with the first 300,000 discarded as a burn-in phase to allow the Markov Chains to stabilize. After burn-in, 400,000 iterations are used, but only every 10^th iteration is retained, resulting in 40,000 posterior samples. Thinning helps reduce autocorrelation between consecutive samples, ensuring more reliable parameter estimates. The parameter estimates from both the neural network SFA model and the conventional SFA model are presented in the Appendix in Tables A.1 and A.2, respectively. In addition to standard statistics (i.e. mean, standard deviation, and 95 % credible interval), the corresponding Geweke’s diagnostic and Monte Carlo standard error (MCSE) are also reported to evaluate the convergence and accuracy of the MCMC simulations. The Geweke’s diagnostic compares the means of the first and last portions of the Markov chain; when the chain has converged to its stationary distribution, these means should not differ systematically. Hence, values close to zero indicate satisfactory convergence. The MCSE quantifies the uncertainty associated with estimating the posterior mean from a finite number of draws and provides an indication of the effective sample size. Small MCSE values suggest that the posterior estimates are numerically stable and that sampling autocorrelation is minimal. Together, these diagnostics confirm that the MCMC chains have mixed well and that the reported parameter estimates are based on reliable and well-converged posterior samples.

The results from the neural network SFA model, presented in Table A.1, indicate that several weights W1, which determine how each input variable contributes to the activation of the hidden neurons, exhibit strong posterior signals with narrow credible intervals that exclude zero. In particular, weights associated with the labor and capital inputs contribute significantly to the non-linear structure of the estimated production frontier. For example, the parameter W1_3,1, linking labor to the first hidden neuron, has a posterior mean of 7.781 with a 95 % credible interval well above zero, indicating a strong positive contribution. The weights in the output layer (W2) determine how each hidden neuron contributes to the final prediction of the production frontier. Among them, W2₂ stands out with a posterior mean of 6.816 and a 95 % credible interval of [3.771, 11.689], suggesting that the second hidden neuron plays a substantial and statistically significant role in shaping the model’s output.

The bias terms b1₁ and b1₂ are large in magnitude and significant, highlighting their role in shifting the activation functions and improving model flexibility. Overall, the estimated network parameters confirm that the neural network SFA model effectively captures complex patterns between the input variables and the output, beyond what a linear specification can achieve. In contrast, the Cobb–Douglas SFA model presented in Table A.2 imposes constant elasticities, with each input affecting output in a fixed, proportional manner. Although all estimated coefficients in the Cobb–Douglas model are significant, the linear functional form limits its ability to represent more complex or varying relationships among inputs and output.

The relationships between each input variable and time with the expected output resulted from the neural network SFA model are illustrated in Figure 6. These relationships are shaped by the tanh activation function in the neural network’s hidden layer, resulting in S-shaped curves. Initially, increasing water quantity has little effect on the expected output, but beyond a certain threshold, a sharp increase occurs, suggesting that water quantity becomes a major factor driving production. For labor, lower values correspond to reduced output, and after a certain point, further increases in labor have minimal positive impact. Similarly, capital has little effect on the expected output when it is below a specific level, but once it crosses this threshold, it significantly boosts production, though additional increases in capital yield diminishing effects. Finally, the expected output increases rapidly in the early periods of time but stabilizes as time progresses. These non-linear relationships, including threshold effects, cannot be captured by standard linear functional forms like the Cobb-Douglas, as suggested by the constant positive elasticities shown in Table A.2 of the Appendix. This highlights the importance of using neural networks to uncover more complex input-output relationships.

Figure 6:

Expected output versus inputs and time (M, L, K and t).

Moreover, Figure 7 presents the density plots of persistent and transient inefficiencies for both the neural network SFA model and the conventional SFA model with a Cobb-Douglas functional form. In the left plot, which shows persistent inefficiency, the average for the neural network SFA model is slightly lower at 0.146 compared to 0.159 in the conventional SFA model. However, more substantial differences are observed in the right plot, which shows the transient inefficiency densities for the two models. Both distributions are right-skewed, but transient inefficiencies are significantly higher in the conventional SFA model. The average transient inefficiency is estimated at 0.235 for the neural network SFA model, while it is notably larger at 0.467 for the conventional SFA model. Additionally, the conventional model exhibits more outliers in the distribution of transient inefficiencies, suggesting a greater spread in inefficiency across units. The above differences in the inefficiency scores across the two models are empirically tested for significance using the procedure outlined in Subsection 2.3. Figure 8 presents histograms of the differences in the mean persistent and transient inefficiencies, highlighting the posterior mean differences along with their corresponding 95 % credible intervals. Focusing on Figure 8a (left panel), the difference in the mean persistent inefficiencies between the two models is relatively small, at 0.012. This is consistent with the previously reported averages: 0.159 for the conventional Cobb–Douglas model and 0.146 for the neural network SFA model. The 95 % credible interval for the difference, [−0.088, 0.114], includes zero, indicating that the difference is not significant. In contrast, Figure 8b (right panel) shows a substantial difference in the mean transient inefficiencies. The average transient inefficiency is 0.467 under the Cobb–Douglas model and 0.235 under the neural network SFA model, yielding a posterior mean difference of approximately 0.232. The corresponding 95 % credible interval, [0.093, 0.350], lies entirely above zero, providing strong evidence that the models yield significantly different estimates of transient inefficiency. This result suggests that the conventional Cobb–Douglas specification inflates the level of transient inefficiency, likely due to its inability to flexibly capture the underlying input–output relationship.

Figure 7:

Kernel densities of persistent (η _i ) and transient (u _t,i ) inefficiencies.

Figure 8:

Histograms of differences in mean inefficiencies.

Figure 9 shows the posterior distributions of the differences in the standard deviations of persistent and transient inefficiencies between the two models. These distributions offer insight into whether the variability in inefficiency is significantly affected by the choice of functional form. In Figure 9a (left panel), the mean difference in the standard deviation of persistent inefficiencies is relatively small at 0.011. The 95 % credible interval ranges from −0.045 to 0.068, which includes zero. This indicates that there is no significant difference in the dispersion of persistent inefficiency estimates across the two models. In contrast, Figure 9b (right panel) presents a strikingly different picture for transient inefficiencies. The posterior mean difference in standard deviations is 0.219, with a 95 % credible interval of [0.094, 0.314], which lies entirely above zero. This clearly indicates that the two models yield significantly different levels of variability in the transient inefficiency component. The greater dispersion observed under the Cobb–Douglas model suggests that it may inflate the heterogeneity in time-varying inefficiency compared to the more flexible neural network specification.

Figure 9:

Histograms of differences in the std. dev’s of inefficiencies.

Finally, Table 1 compares the marginal log likelihoods, prior probabilities, posterior probabilities, and DIC values for the neural network SFA and conventional SFA models. The neural network SFA model achieves a posterior probability of 1 and a lower DIC, indicating a clearly better fit to the data compared to the conventional specification. Combined with the results in Figures 8 and 9, this provides strong empirical evidence that the Cobb–Douglas model not only underperforms in terms of overall fit but also systematically inflates transient inefficiency. This highlights the importance of using more flexible functional forms, such as neural networks, when estimating production frontiers and inefficiency components.

5 Concluding Remarks

This study introduces a Bayesian neural network SFA model designed to capture non-linear relationships between inputs and outputs while estimating both persistent and transient inefficiencies. This is achieved by specifying a generalized true random effects model in which the deterministic part of the production frontier is modeled using a neural network. Rather than relying on pre-defined functional forms such as the Cobb–Douglas, the neural network learns the input–output relationship from the data by expressing the frontier as a weighted sum of non-linear transformations of the input variables. This flexible structure enables the model to approximate complex production relationships more accurately, thereby improving the reliability of the inefficiency estimates. For comparison, we also consider a generalized true random effects SFA model with the conventional linear Cobb–Douglas functional form. Both models are estimated within a Bayesian framework, and the comparison focuses on selected moments of the inefficiency distributions as well as overall model fit. The models are applied to panel data from Chilean hydroelectric power plants.

The empirical findings from the neural network SFA model reveal complex non-linear relationships between all inputs and output, patterns that are missed by the constant elasticities implied by the Cobb–Douglas specification. Both persistent and, more notably, transient inefficiency scores are inflated under the Cobb–Douglas SFA model compared to the neural network alternative. While the differences in the mean and standard deviation of persistent inefficiencies are not significant, the differences for transient inefficiencies are both substantial and significant. Moreover, model comparison based on Bayes factors and DIC strongly favors the neural network SFA model, indicating that it provides a better fit to the data.

These findings have important implications for practitioners aiming to assess and improve firm performance. The choice of functional form in stochastic frontier models can substantially influence the estimated levels of inefficiency, particularly transient inefficiency, which reflects short-term performance fluctuations. Relying on a restrictive specification like the Cobb-Douglas may lead to inflated inefficiency estimates, potentially resulting in misguided managerial responses. In contrast, the neural network SFA model, by accommodating non-linear input-output relationships, offers more reliable inefficiency estimates. Moreover, the superior model fit indicated by the Bayes factor and DIC comparison suggests that flexible, data-driven approaches better capture the underlying production technology. Practitioners are therefore encouraged to adopt more adaptable modeling frameworks, such as neural networks, when evaluating efficiency, especially in complex or data-rich environments.

From a policy perspective, the finding that conventional models inflate transient inefficiency has important implications. Transient inefficiency often reflects short-term performance variability that can result from temporary shocks, such as market volatility, or regulatory adjustments. If policymakers rely on estimates derived from restrictive functional forms, they may overstate short-term inefficiency and misinterpret temporary deviations as persistent performance gaps. This could lead to misguided interventions or resource allocations. By adopting more flexible approaches, such as the proposed neural network SFA, policymakers can obtain a clearer picture of firms’ true performance dynamics and design more targeted and proportionate policy measures.

Despite its promising results, this study has some limitations that should be acknowledged. The analysis is based on a specific neural network structure with a fixed activation function and number of hidden neurons, which may influence the model’s flexibility and performance. Moreover, the empirical application focuses on a single dataset from the energy sector, and thus the generalizability of the findings to other industries remains to be explored. These limitations also point toward meaningful directions for future research.

To further enhance the analysis, future research could explore two potential extensions. First, while the tanh activation function is widely used, alternative activation functions, such as the sigmoid and Rectified Linear Unit, could be tested to determine if they provide better model performance in capturing complex relationships. Second, within a Bayesian framework, the number of neurons in the neural network could be empirically determined using shrinkage priors, as suggested by Hauzenberger et al. (2025).