Volume 6, Issue 1 -

In a recent BMJ article, the authors conducted a meta-analysis to compare estimated treatment effects from randomized trials with those derived from observational studies based on routinely collected data (RCD). They calculated a pooled relative odds ratio (ROR) of 1.31 (95 % confidence interval [CI]: 1.03–1.65) and concluded that RCD studies systematically over-estimated protective effects. However, their meta-analysis inverted results for some clinical questions to force all estimates from RCD to be below 1. We evaluated the statistical properties of this pooled ROR, and found that the selective inversion rule employed in the original meta-analysis can positively bias the estimate of the ROR. We then repeated the random effects meta-analysis using a different inversion rule and found an estimated ROR of 0.98 (0.78–1.23), indicating the ROR is highly dependent on the direction of comparisons. As an alternative to the ROR, we calculated the observed proportion of clinical questions where the RCD and trial CIs overlap, as well as the expected proportion assuming no systematic difference between the studies. Out of 16 clinical questions, 50 % CIs overlapped for 8 (50 %; 25 to 75 %) compared with an expected overlap of 60 % assuming no systematic difference between RCD studies and trials. Thus, there was little evidence of a systematic difference in effect estimates between RCD and RCTs. Estimates of pooled RORs across distinct clinical questions are generally not interpretable and may be misleading.

Compartmental model diagrams have been used for nearly a century to depict causal relationships in infectious disease epidemiology. Causal directed acyclic graphs (DAGs) have been used more broadly in epidemiology since the 1990s to guide analyses of a variety of public health problems. Using an example from chronic disease epidemiology, the effect of type 2 diabetes on dementia incidence, we illustrate how compartmental model diagrams can represent the same concepts as causal DAGs, including causation, mediation, confounding, and collider bias. We show how to use compartmental model diagrams to explicitly depict interaction and feedback cycles. While DAGs imply a set of conditional independencies, they do not define conditional distributions parametrically. Compartmental model diagrams parametrically (or semiparametrically) describe state changes based on known biological processes or mechanisms. Compartmental model diagrams are part of a long-term tradition of causal thinking in epidemiology and can parametrically express the same concepts as DAGs, as well as explicitly depict feedback cycles and interactions. As causal inference efforts in epidemiology increasingly draw on simulations and quantitative sensitivity analyses, compartmental model diagrams may be of use to a wider audience. Recognizing simple links between these two common approaches to representing causal processes may facilitate communication between researchers from different traditions.

Routinely collected administrative and clinical data are increasingly being utilized for comparing quality of care outcomes between hospitals. This problem can be considered in a causal inference framework, as such comparisons have to be adjusted for hospital-specific patient case-mix, which can be done using either an outcome or assignment model. It is often of interest to compare the performance of hospitals against the average level of care in the health care system, using indirectly standardized mortality ratios, calculated as a ratio of observed to expected quality outcome. A doubly robust estimator makes use of both outcome and assignment models in the case-mix adjustment, requiring only one of these to be correctly specified for valid inferences. Doubly robust estimators have been proposed for direct standardization in the quality comparison context, and for standardized risk differences and ratios in the exposed population, but as far as we know, not for indirect standardization. We present the causal estimand in indirect standardization in terms of potential outcome variables, propose a doubly robust estimator for this, and study its properties. We also consider the use of a modified assignment model in the presence of small hospitals.

A unified theory is developed for attributable proportion (AP) and population attributable fraction (PAF) of joint effects, marginal effects or interaction among factors. We use a novel normalization with a range between –1 and 1 that gives the traditional definitions of AP or PAF when positive, but is different when they are negative. We also allow for an arbitrary number of factors, both those of primary interest and confounders, and quantify interaction as departure from a given model, such as a multiplicative, additive odds or disjunctive one. In particular, this makes it possible to compare different types of threeway or higher order interactions. Effect parameters are estimated on a linear or logit scale in order to find point estimates and confidence intervals for the various versions of AP and PAF, for prospective or retrospective studies. We investigate the accuracy of three confidence intervals; two of which use the delta method and a third bootstrapped interval. It is found that the delta method with logit type transformations, and the bootstrap, perform well for a wide range of models. The methodology is also applied to a multiple sclerosis (MS) data set, with smoking and two genetic variables as risk factors.