Efficiency Measurement in Healthcare: The Foundations, Variables, and Models – A Narrative Literature Review

1 Introduction

The last 30 years have witnessed considerable momentum in the number of studies published on the topic of healthcare efficiency. The theory of production and cost functions, following the seminal work of Farrell (1957), influences the current methods of efficiency evaluation. Many of the healthcare literature’s empirical methods revolve around estimating either technical or allocative efficiency or both (Worthington, 2004).

Researchers have extensively employed frontier-based efficiency techniques to measure healthcare units’ productivity and efficiency. Frontier techniques are divided into parametric and nonparametric methods. Both methods involve estimating a frontier against which the performance of healthcare providers is compared. A healthcare provider on the frontier is believed to be able to provide a given level of service using the least amount of inputs/minimum cost or the maximum level of services for a given level of inputs/cost (Hollingsworth & Peacock, 2008, p. 2). The degree of deviation from the efficient frontier provides an estimate of the level of inefficiency.

Data Envelopment Analysis (DEA) is a nonparametric methodology based on linear programming tools developed by Charnes et al. (1978) and is one of the commonly used frontier-based methodologies in health efficiency studies. The DEA frontier includes a series of linear segments connecting one efficient decision-making unit (DMU^[1]) to another. The frontier’s construction is based on “best-observed practice,” where inefficient DMUs are “enveloped” by the efficiency frontier. A notable feature of the traditional DEA, and often considered a drawback, is that all deviations from the frontier are attributed to inefficiency. However, this issue has been addressed by the bootstrap methods of Simar and Wilson (1998), which provide a mechanism to distinguish between inefficiency and statistical noise, thereby refining the accuracy of the efficiency estimates derived from DEA.

One of the earliest applications of efficiency measurement techniques was undertaken by Nunamaker (1983), who used the DEA to estimate the technical efficiency of 16 hospitals in the state of Wisconsin, USA. Soon after, Borden (1988) and Sherman (1984) also employed the DEA methodology to compute the technical efficiency scores of hospitals in the USA.

A major limitation of using the DEA comes from the fact that it makes an unverifiable and strong assumption of no measurement error or random variation in output (Newhouse, 1994). In particular, interpretation of DEA-based results may be problematic, as frontiers may be affected by stochastic variance, measurement error, or unobserved heterogeneity of data (Hollingsworth & Peacock, 2008, p. 37).

In the healthcare sector, there are occasions where the healthcare unit’s capacity to deliver services is affected by factors outside the healthcare provider’s control. For example, the sudden onset of a pandemic in a region, medical equipment may suddenly break down,^[2] or there may be errors in the measurement of the level of resources used. Since the DEA fails to account for random shocks, it can introduce bias to the efficiency scores (Jacobs et al., 2006, p. 153). Nevertheless, the DEA and its variants are still the most widely used tool in healthcare studies, possibly due to its ease of use and versatility (Jacobs et al., 2006, p. 13).

In the last 20 years, various studies have been put forward that can be used in conjunction with the DEA to deal with efficiency scores’ sensitivity. One of the most popular among these techniques is the application of the bootstrap methodology introduced by Simar and Wilson (1998, 2007). The bootstrap methodology, to some extent, has addressed the issue of the sensitivity of efficiency scores to the sampling variation and has provided the statistical properties of the nonparametric estimators. Some of the recent studies in healthcare literature using bootstrap methodology with the DEA include work by Alonso et al. (2015), Andrews (2020a), Andrews (2020b), Chowdhury and Zelenyuk (2016), and Jiang and Andrews (2020).

Stochastic Frontier Analysis (SFA) is, on the other hand, a parametric approach developed independently by Aigner et al. (1977) and Meeusen and Van den Broeck (1977). SFA differs from DEA in its assumption that discrepancies between actual and optimal organizational performance are due to inefficiencies and random shocks.

In order to incorporate the concept of stochastic shocks and inefficiency in SFA, the error term is defined as the sum of two components – a one-sided, non-negative term that represents inefficiency and the other component, which represents random or stochastic fluctuations. In addition to the distributional assumption, the production function specification is also required in SFA. On the other hand, DEA requires no specification of the production functions or distributional assumptions where the efficiency frontier is constructed purely based on observed data (Jacobs et al., 2006, p. 90; Nedelea & Fannin, 2013).

Even though there are challenges associated with SFA’s assumptions and specifications, its ability to separate random fluctuations beyond a hospital’s control has made it very popular. Furthermore, SFA allows researchers to estimate the relationships between outputs, inputs, and costs. Further, SFA allows researchers to separate healthcare provider-specific effects (heterogeneity) and time-specific effects when longitudinal data are available. Hence, applying SFA to longitudinal data also allows for a more robust estimate of parameters.

While the majority of studies in healthcare efficiency literature use classical inferences to estimate the model parameters, Koop et al. (1997) employed Bayesian inference to estimate the model parameters and cost efficiency by using longitudinal data on 382 non-teaching U.S. hospitals. More recently, Chen et al. (2016) used Bayesian SFA to estimate hospital cost efficiency in 31 provinces in China.

Although SFA’s implementation is more demanding in terms of modelling and interpretive skills (Jacobs et al., 2006, p. 13), it has been gaining more prominence in healthcare productivity and efficiency studies (Worthington, 2004). Some healthcare studies that employ SFA include work by Al-Amin et al. (2016), Chen et al. (2016), Colombi et al. (2017), and Jiang and Andrews (2020). Some more studies include the following:

Although SFA’s implementation is more demanding in terms of modelling and interpretive skills (Jacobs et al., 2006), it has been gaining prominence in healthcare productivity and efficiency studies (Worthington, 2004). Some healthcare studies employing SFA include the works of Al-Amin et al. (2016), Chen et al. (2016), Colombi et al. (2017), and Jiang and Andrews (2020). Recent additions to this body of literature are the analysis of health facility efficiency for non-communicable diseases by Bala et al. (2023), an examination of hospital cost efficiency in US acute care by Linde (2023), a dynamic analysis of cost efficiency in New Zealand healthcare providers by Andrews and Emvalomatis (2023), and an evaluation of the temporal–spatial evolution of Healthcare Services Efficiency in 31 Chinese provinces by Ye and Tao (2023).

Since the introduction of SFA in efficiency literature, several approaches have been put forward that employ it in the context of longitudinal data in various other sectors. Early research into longitudinal SFA focussed on estimating time-invariant (persistent) or long-run efficiency (Battese & Coelli, 1988; Kumbhakar, 1987; Pitt & Lee, 1981; Schmidt & Sickles, 1984), time-varying (transient) or short-run efficiency (Battese & Coelli, 1992, 1995; Cornwell et al., 1990; Kumbhakar, 1990). Other studies, such as those by Kumbhakar and Heshmati (1995) and Kumbhakar and Hjalmarsson (1995), estimated persistent efficiency and transient efficiency. On the other hand, studies by Greene (2005a,b), Kumbhakar and Wang (2005), and Wang and Ho (2010) estimated transient efficiency while accounting for heterogeneity at the cost of ignoring persistent inefficiency.

Currently, two main approaches exist in the efficiency literature that incorporates the idea of persistence in inefficiency. The difference between these two approaches is due to the specific treatment of the adjustment cost hypothesis in efficiency analysis. Studies such as those by Colombi et al. (2014), Filippini and Greene (2016), Filippini and Hunt (2015), Kumbhakar and Heshmati (1995), Kumbhakar and Hjalmarsson (1995), Kumbhakar et al. (2014), and Tsionas and Kumbhakar (2014) employ SFA without incorporating the adjustment cost theory. Instead, they divide total inefficiency into short-run (transient) and long-run inefficiency (persistent inefficiency). In healthcare literature, studies incorporating the persistent nature of inefficiency are scarce. So far, only one study by Colombi et al. (2017) has used SFA to evaluate the transient and persistent efficiency of 133 Italian hospitals.

An important point to note about these studies is that the authors differentiate between transient (short-run) and persistent (long-run) inefficiency by specifying a time-varying and one-sided time-invariant skewed error term, respectively. In such specifications, both short-run and long-run inefficiency terms necessitate one-sided distributional assumptions that only take positive values, such as half-normal, exponential, and gamma distributions. While these terms often employ the same type of distributional specification, they are considered independent of each other. Also, the short-run inefficiency estimates in these models are assumed to be independent between different time periods.

The motivation behind the existence of long-run inefficiency in these models fundamentally rests on the idea that there are long-run factors that give time-invariant characteristics to persistent inefficiency. Examples of such factors include obsolete production equipment and technology, substandard buildings, substandard transport systems, the continuous lack of workforce development leading to underexploited technologies, and other management rigidities associated with administrative practices.

In other words, long-run inefficiency stems from structural issues that constrain efficient methods due to operational rigidities over a longer time horizon. These operational rigidities are theorized to be related to physical capacity, infrastructural problems, recurring managerial incompetence, and modern technology availability. Though the motivation behind persistence in inefficiency makes economic sense, none of these studies incorporates dependencies in inefficiency through time.

Another strand of efficiency literature provides a more comprehensive and economically intuitive way of combining the idea of short- and long-run inefficiencies in SFA through a dynamic process. This approach explicitly highlights the existence of the adjustment costs of quasi-fixed inputs to be the primary reason for persistence in inefficiency over time. As a result, the organization would prefer to remain partly inefficient in the short run due to high adjustment costs and instead seek to achieve its targeted long-run efficiency level (i.e. steady-state efficiency level) in the long run. Furthermore, these models are dynamic, allowing the short-run inefficiency between periods to be dependent.

Ahn and Sickles (2000) pioneered the dynamic stochastic frontier approach by specifying an autoregressive process to accommodate persistence in inefficiency due to adjustment costs. The dynamic models they specified use generalized non-linear methods of moments (GMM) to estimate the parameters. However, a study by Bun and Windmeijer (2010) highlights that in dynamic longitudinal data models estimated via GMM methods, weak instruments – variables that inadequately correlate with endogenous predictors – can become particularly problematic near the unit root boundary, where variables exhibit stochastic trends without reverting to a long-term mean. This issue can lead to a skewed variance ratio of errors, a measure comparing the variability due to inefficiency versus other random effects, straying from the ideal value of unity. This divergence affects the model’s accuracy, especially in differentiating between short-run and long-run inefficiencies, emphasizing the need for careful instrument selection in econometric analyses to ensure reliable results.

Desli et al. (2003) put forth a version of the dynamic stochastic frontier model estimated using maximum likelihood methods (ML), assuming that the healthcare provider-specific intercept is autoregressive, with a set of covariates that influence a healthcare provider’s production frontier over time. However, as Khalaf and Saunders (2016) highlighted, this specification is prone to incidental parameter bias due to the correlation between unobserved heterogeneity and efficiency-specific covariates in the latent equation.

Using the Bayesian approach, Tsionas (2006) presented a dynamic model where an autoregressive process was applied to a transformed efficiency that can take any value on the real line, and thus, a standard autoregressive process can be imposed on it. Similarly, using the Bayesian approach, Emvalomatis (2012) used the inverse of the logistic function of technical efficiency as a transformation in the autoregressive process. Building on Tsionas (2006), many other Bayesian dynamic model versions have been presented in studies such as those by Emvalomatis et al. (2011), Galán et al. (2015), Lambarraa et al. (2015), and Skevas et al. (2018).

These dynamic models are motivated by the adjustment cost hypothesis, where the short-run efficiency is derived based on the organization’s performance relative to the production possibility frontier. In contrast, the long-run efficiency corresponds to the long-run equilibrium value of efficiency specified by the autoregressive process. Hence, the dynamic model is more flexible, as it accommodates the relationship between transient efficiency between different periods. However, no such relationship is found in the models where transient efficiency is assumed to be a one-sided time-varying error component. Further, in the non-dynamic model, short-run efficiency is obtained from a system that is always assumed to be in equilibrium. The assumption of constant equilibrium is unrealistic in the presence of adjustment costs and other rigidities arising from the sector’s regulatory framework.

To the best of current knowledge, there are no studies that have incorporated the idea of dynamic models in assessing healthcare providers’ efficiency or productivity performances. Jacobs et al. (2006, pp. 174–176) argue that in the short run, healthcare providers might only be able to perform relative to the various constraints imposed by infrastructure and available inputs (e.g. quality of clinical equipment and technology). Therefore, short-run efficiency levels should only be assessed based on the configuration of the inputs that a healthcare provider has available. On the other hand, healthcare providers may reconfigure their resources to bring about efficiency improvements in the long run. This implies that the healthcare production process should be modelled through a dynamic link between its present and past performances Jacobs et al. (2006, pp. 177–178).

Specifying a dynamic link basically makes it possible for the current output of a healthcare provider to depend on the ease with which the inputs can be reconfigured or through which technology may be adopted in the presence of adjustment costs. Given the prevalence of public finance as a fundamental source of healthcare services in the majority of developed countries and the existence of a highly regulated operating environment (Jacobs et al., 2006, p. 3), it is surprising how little attention is paid to this dynamic link in healthcare studies.

A possible reason for this might be the complexity associated with the Estimation of dynamic longitudinal models and the small data sizes that are prevalent in healthcare efficiency studies (Jacobs et al., 2006, pp. 37–38). Nevertheless, Colombi et al. (2017) and Hollingsworth and Street (2006) argue that identifying the nature and the form of inefficiency in healthcare systems is critical for formulating appropriate policy measures. For example, if a high degree of persistence in inefficiency exists, especially among public healthcare providers, then unless there is a reconfiguration of the current organizational structure or/and a significant change of government policy towards the overall system, efforts to improve efficiency will not yield expected outcomes.

A selective list of healthcare efficiency studies from the early 1990s to 2020 that have used frontier-based approaches is presented in Table A1. While not a complete list, it provides a fair representation of the methodology used in the last three decades. A comprehensive list of frontier-based healthcare efficiency studies can be found at Hollingsworth and Peacock (2008, pp. 102–117) and Worthington (2004).

It is also worth noting that, based on the studies listed in Table A1, 36 of the 40 studies that applied DEA, with or without bootstrapping, 17 of them exclusively used longitudinal data with no control for unobserved or unit-specific heterogeneity. This is likely to result in a substantial bias in the measure of efficiency. Additionally, of the 13 studies that used SFA on longitudinal data, only Barros et al. (2013), Chen et al. (2016), Colombi et al. (2017), and Koop et al. (1997) controlled for unobserved heterogeneity.

2 Previous Contribution to TFP Studies in Healthcare

When longitudinal data are available, it is insightful to investigate the changes in productivity over time and to decompose it into its components to investigate the relative contributions. In the healthcare sector, such a study will help analyse the effect of targeted policies on the provision of health services and ultimately determine the impact of various initiatives on the population’s health outcomes. For example, suppose the intention is to examine the productivity of a group of healthcare providers. In that case, one could determine whether productivity change for a specific healthcare provider is driven by improvement in the provider’s relative efficiency, scale improvement, or technological progress.

In practice, TFP can either be estimated by using index number methods or econometric techniques. Examples of index number methods include the Malmquist productivity index, the Hicks–Moorsteen productivity index, the Törnqvist productivity index, and the Fisher productivity index (Jacobs et al., 2006, p. 129). In the econometric approach, regression analysis and the SFA are often used to estimate a production or cost function with distributional assumptions to estimate the TFP change and its components.

A selective list of healthcare literature studies focusing on TFP change and its components is provided in Table A2. Among others, this list summarizes the methodology, variables, and results of several healthcare studies related to the assessment of TFP and components. While the studies in Table A2 decompose the TFP changes in efficiency, scale, and technological components, they tend to concentrate on the relative contribution of efficiency and technological change to changes in the TFP.

The list of studies in Table A2 shows the Malmquist productivity index (Malmquist, 1953) to be the most common approach to healthcare productivity. Malmquist’s productivity index was introduced into the literature through the seminal study by Caves et al. (1982), which adopted Malmquist’s approach to constructing quantity indices as distance function ratios.

Färe et al. (1992) undertook one of the earliest applications of the Malmquist productivity index in healthcare. The study analysed the productivity changes of a group of pharmacies in Sweden and concluded that most of the improvements in the TFP were due to technological progress. The healthcare studies that followed the studies by Linna (1998); Maniadakis et al. (1999); Ng (2011); Tambour (1997)) and that used the Malmquist productivity index also found positive technological progress.

However, following the study by Färe et al. (1992), another application of the Malmquist productivity index was undertaken by Burgess and Wilson (1995). Their study found that technological decline dominated the effect of technical efficiency on TFP for a group of 137 hospitals in the USA. Other studies, such as those by Giuffrida (1999), González and Gascón (2004), and Jiménez et al. (2003), have also either reported technological regress or no significant impact of technological change on the TFP.

As for the effect of efficiency changes on TFP, studies by Dismuke and Sena (1999), Gannon (2008), Linna (1998), and Maniadakis and Thanassoulis (2000) found that both efficiency and technological progress contributed to the increase in the TFP. On the other hand, Giuffrida (1999), who assessed the TFP growth of 90 English family health service authorities over the period 1991–1995, found that only technical and scale efficiency contributed to TFP growth while there was no noticeable technological progress. Further, the study highlighted that improvements in the TFP were minimal and expressed a limited scope of productivity growth in the healthcare sector.

Two Malmquist index studies have incorporated the quality of healthcare outputs in the assessment of the TFP. The earliest study was by Färe et al. (1995), which showed that the incorporation of quality significantly affects the measure of TFP change. More recently, Karmann and Roesel (2017) found that quality improvements contributed more growth towards TFP than just output volumes for a sample of German hospital data.

While Malmquist indices are frequently utilized in healthcare studies and demand data on inputs and outputs to be consistently measured over time (Jacobs et al., 2006, p. 137), their application in healthcare is challenging due to frequent policy shifts and variable data collection practices. It is important to note that, although the computation of Malmquist indices does not require the assumption of constant returns to scale (CRS), they are often calculated under this assumption. This common practice, albeit not a necessity, is primarily to facilitate the evaluation of productivity changes inclusive of scale effects. This approach, while not always ideal, allows for a simplified analysis of scale efficiency alongside efficiency and technical changes (Coelli et al., 2005, p. 293).

However, very few healthcare studies use econometric approaches to assess improvements in productivity and its components. Morikawa (2010) used a fixed-effects panel-data model to estimate the effects of increasing hospital size on TFP, based on data from 239 Japanese medical facilities. The study found that TFP increases more than 10% when the size of a hospital doubles. Using time-series regression analysis, Blank and Eggink (2014) used Dutch hospital data for the period 1972–2010 to analyse productivity improvements. Their studies looked into the effects of regulations on changes in the TFP and found that hospital competition reform failed to improve productivity among hospitals. Both of these studies excluded inefficiency in the cost/production functions. Concerning the econometric approach, only one study by Dismuke and Sena (1999) used the SFA to decompose the TFP change for a group of hospitals in Portugal. Their result found technical progress for most of the hospitals.

One of the advantages of using SFA is that it allows for the control of unobserved time-invariant heterogeneity when computing costs and input elasticities. Further, it offers an opportunity to decompose productivity changes into parts that have a straightforward economic interpretation. Despite numerous advantages, currently, SFA is not widely used in the healthcare sector to assess changes in the TFP and its components.

Another noteworthy point is that all the studies in Table A2, except for Linna (1998), have used the primal approach to estimate the TFP changes and their components. No study has undertaken a TFP analysis under both the primal and dual approaches. In TFP analysis, the primal and dual approaches offer distinct methodologies for estimating productivity growth. The primal approach, or output-based method, focuses on quantifying the output growth relative to the growth of inputs, directly measuring the changes in the quantities of inputs and outputs. It essentially examines the production function to derive productivity changes.

The dual approach, in contrast, is often characterized by the use of total cost as the dependent variable, with the prices of inputs and outputs serving as independent variables. This input-based or price-based method derives TFP growth by analysing how input and output prices impact the cost structure. It looks at the cost function, focusing on the relationship between costs and prices to deduce productivity changes.

The duality theory, as outlined by Jorgenson and Griliches (1967), suggests that these two approaches should theoretically yield consistent results since they are both grounded in the same economic behaviour but viewed from different perspectives. However, practical differences can arise in their outcomes due to factors like measurement errors, model misspecification, or data inconsistencies (Kee, 2004). For instance, the primal method might be skewed by inaccurate measurements of physical quantities, whereas the dual method could be influenced by price fluctuations or market changes that affect costs and the pricing of inputs and outputs. Hence, although the primal and dual methods are theoretically aligned, their empirical implementation can yield divergent conclusions, underscoring the need for meticulous data scrutiny and precise model specification in TFP research. Thus, applying both methods can provide a comprehensive view and help identify the reasons for variations in TFP estimations.

3 Variables used in Healthcare Efficiency and Productivity Studies

4 Summary

The literature review examines various methodologies applied in healthcare efficiency and productivity studies, revealing a variety of analytical approaches yet punctuated by significant gaps. While Data Envelopment Analysis (DEA) and Stochastic Frontier Analysis (SFA) have been prominent tools in this research domain, their utilization with longitudinal data, which is critical for understanding performance over time, remains limited. This gap suggests a potential underutilization of available data and methodologies to capture the dynamic nature of healthcare efficiency.

Moreover, the literature lacks studies that concurrently apply primal and dual approaches to measure Total Factor Productivity (TFP) changes. The primal approach, which focuses on output relative to input growth, is commonly contrasted with the dual approach that examines the impact of input prices and output on cost structures. The simultaneous application of both methods could offer a comprehensive analysis, reconciling the quantity-based and price-based perspectives of productivity growth. It could also inform discrepancies arising from external influences, including policy shifts and economic conditions, that singularly employed methodologies might overlook.

The review also highlights the variability in the application of inputs, outputs, and price variables in healthcare productivity and efficiency studies. This variability is not merely a methodological preference but often a response to the challenges of data acquisition and the particular objectives of research. The sector’s complexity necessitates carefully selecting these variables, where data availability and consistency constrain choices. This limitation impacts not only the precision of efficiency assessments but also the validity of subsequent policy recommendations.

A significant literature gap is identified in the control (or lack thereof) for cross-sectional unobserved heterogeneity in longitudinal studies. This methodological oversight can yield efficiency measurements fraught with bias, a concern highlighted by Greene (2005a) and others. The nuanced understanding of healthcare provision, influenced by a myriad of patient, provider, and systemic factors, calls for an analytical framework that accounts for these unobserved elements.

In the limited instances where studies like that of Colombi et al. (2017) have differentiated between short-run and long-run inefficiency, there remains an absence of analyses that model efficiency dynamically, acknowledging the inter-temporal dependencies of inefficiencies. Such dynamic modelling is crucial for capturing the true nature of efficiency evolution within healthcare providers, where past performances can cast long shadows on future productivity.

The review identifies that the majority of healthcare efficiency studies have traditionally focused on technical efficiency, often using inpatient admissions or discharges as proxies for output. However, there is an emerging recognition of the need to evaluate cost and allocative efficiency, which requires detailed information on input prices for labour and capital. The latter is particularly challenging in the healthcare sector, where capital is not just a static measure of infrastructure but a dynamic resource that evolves with technological advancements and strategic investments.

The complexity of healthcare services, which combine labour, capital, and materials to produce outcomes that impact health status, is mirrored in the complexity of measuring these very inputs and outcomes. Labour inputs, from medical to non-medical staff, contribute vitally to service delivery, yet their measurement varies from headcounts to full-time equivalents (FTEs), with the latter providing a more nuanced representation of labour utilization.

Capital, as the second crucial input, has typically been measured by proxies such as the number of beds or floor space, but these measures fail to capture the nuances of capital service flows. The “capital charge” model is an alternative that reflects the economic cost of capital, yet its adoption is not widespread in healthcare studies. Such measures are necessary to reflect the actual use and cost of capital more accurately in healthcare settings.

In conclusion, this literature review not only maps out the existing methodologies and applications in healthcare efficiency studies but also casts a spotlight on the avenues for methodological improvement. The identified gaps underscore the need for a more dynamic, nuanced approach to efficiency and productivity analysis that can accommodate the sector’s complexity. By addressing these gaps, future research can offer more detailed insights into the temporal dynamics of healthcare efficiency, ultimately informing policy decisions that could lead to better health outcomes and more efficient use of resources.