Volume 13, Issue s2 - Special Issue: Causal Inference; Issue Editors: Hélène Karcher and Fan Li

Context Modern causal inference methods – although core to epidemiological reasoning – may be difficult to master and less intuitive than Hill’s classical considerations. We developed a ‘How-Questions’ (HQ) framework to integrate Hill's classical considerations with modern causal inference methods in observational studies. Methods First, we extracted the main causal considerations from contemporary philosophy of science: characteristics of empirical associations, universality, depth, and degree of corroboration of a theory. From these, we developed a HQ framework based on six domains formulated as questions: (1) how valid? , (2) how time-ordered? , (3) how big? , (4) how shaped? , (5) how replicable? , and (6) how explainable? Then, we qualitatively checked whether Hill's classical considerations and key selected modern causal inference methods were compatible with the HQ framework. Lastly, as a proof-of-concept, we applied the HQ framework to two observational studies of current topics in epidemiology. Findings Both Hill’s considerations and key selected modern causal inference methods were compatible with the six domains of the HQ framework. (1) The how-valid domain is addressed by considering the same internal validity issues in Hill’s and modern methods, namely confounding, selection and measurement biases; modern methods use more formalized techniques, including quantitative bias analyses/sensitivity analyses (QBA/SA). (2) The how-time-ordered domain is addressed by considering reverse causation in Hill’s; modern methods may use G methods within the context of longitudinal data analyses and time-varying exposures. (3) The how-big domain is addressed by strength of association in Hill’s; modern methods first consider estimands and may use QBA/SA to assess robustness of effect estimates. (4) The how-shaped domain is represented by biological gradient in Hill’s; modern methods may use generalized propensity scores to estimate dose-response functions. (5) The how-replicable domain is addressed in Hill’s by consistency of study findings with existing evidence; modern methods may use triangulation of different study designs and consider generalizability and transportability concepts. (6) The how-explainable domain is addressed by biological plausibility in Hill’s and by mediation/interaction analyses in modern methods. The application of the HQ framework to two observational studies provides a proof-of-concept and suggests its potential usefulness to integrate Hill’s considerations with modern causal inference methods. Perspective We found that the six dimensions of the HQ framework integrated Hill’s classical considerations with modern causal inference methods for observational studies. Apart from its potential pedagogical value, the HQ framework may provide a holistic view for the causal assessment of observational studies in epidemiology.

Objectives Methods of causal inference are used to estimate treatment effectiveness for non-randomized study designs. The propensity score (i.e., the probability that a subject receives the study treatment conditioned on a set of variables related to treatment and/or outcome) is often used with matching or sample weighting techniques to, ideally, eliminate bias in the estimates of treatment effect due to treatment decisions. If multiple treatments are available, the propensity score is a function of the adjustment set and the set of possible treatments. This paper develops a compound representation that separates the treatment decision into a binary decision: treat or don't treat, and a potential treatment decision: choose the treatment that would be given if the subject is treated. Methods The compound representation was derived from Robin's definition of the propensity score, and a second proof is derived from importance sampling. A simulation study illustrates the use of the method. Results Multiple treatment stabilized marginal structural weights were calculated with this approach, and the method was applied to an observational study to evaluate the effectiveness of different neutralizing monoclonal antibodies to treat infection with various severe acute respiratory syndrome coronavirus 2 variants. Conclusions The method can greatly simplify the computation of multiple treatment propensity scores and reduce bias in comparison with improperly used logistic regression.

Objectives Highly flexible nonparametric estimators have gained popularity in causal inference and epidemiology. Popular examples of such estimators include targeted maximum likelihood estimators (TMLE) and double machine learning (DML). TMLE is often argued or suggested to be better than DML estimators and several other estimators in small to moderate samples – even if they share the same large-sample properties – because TMLE is a plug-in estimator and respects the known bounds on the parameter, while other estimators might fall outside the known bounds and yield absurd estimates. However, this argument is not a rigorously proven result and may fail in certain cases. Methods In a carefully chosen simulation setting, I compare the performance of several versions of TMLE and DML estimators of the average treatment effect among treated in small to moderate samples. Results In this simulation setting, DML estimators outperforms some versions of TMLE in small samples. TMLE fluctuations are unstable, and hence empirically checking the magnitude of the TMLE fluctuation might alert cases where TMLE might perform poorly. Conclusions As a plug-in estimator, TMLE is not guaranteed to outperform non-plug-in counterparts such as DML estimators in small samples. Checking the fluctuation magnitude might be a useful diagnosis for TMLE. More rigorous theoretical justification is needed to understand and compare the finite-sample performance of these highly flexible estimators in general.

Objectives In analysis of randomized controlled trials (RCTs) with patient-reported outcome measures (PROMs), Item Response Theory (IRT) models that allow for heterogeneity in the treatment effect at the item level merit consideration. These models for “item-level heterogeneous treatment effects” (IL-HTE) can provide more accurate statistical inference, allow researchers to better generalize their results, and resolve critical identification problems in the estimation of interaction effects. In this study, we extend the IL-HTE model to polytomous data and apply the model to determine how the effect of selective serotonin reuptake inhibitors (SSRIs) on depression varies across the items on a depression rating scale. Methods We first conduct a Monte Carlo simulation study to assess the performance of the polytomous IL-HTE model under a range of conditions. We then apply the IL-HTE model to item-level data from 24 RCTs measuring the effect of SSRIs on depression using the 17-item Hamilton Depression Rating Scale (HDRS-17) and estimate heterogeneity by subscale (HDRS-6). Results Our simulation results show that ignoring IL-HTE can yield standard errors that are as much as 50 % too small and create significant bias in treatment by covariate interaction effects when item-specific treatment effects are correlated with item location, and that the application of the IL-HTE model resolves these issues. Our empirical application shows that while the average effect of SSRIs on depression is beneficial (i.e., negative) and statistically significant, there is substantial IL-HTE, with estimates of the standard deviation of item-level effects nearly as large as the average effect. We show that this substantial IL-HTE is driven primarily by systematically larger effects on the HDRS-6 subscale items. Conclusions The IL-HTE model has the potential to provide new insights for the inference, generalizability, and identification of treatment effects in clinical trials using PROMs.

Objectives The Number Needed to Treat (NNT) is an efficacy index defined as the average number of patients needed to treat to attain one additional treatment benefit. In observational studies, specifically in epidemiology, the adequacy of the populationwise NNT is questionable since the exposed group characteristics may substantially differ from the unexposed. To address this issue, groupwise efficacy indices were defined: the Exposure Impact Number (EIN) for the exposed group and the Number Needed to be Exposed (NNE) for the unexposed. Each defined index answers a unique research question since it targets a unique sub-population. In observational studies, the group allocation is typically affected by confounders that might be unmeasured. The available estimation methods that rely either on randomization or the sufficiency of the measured covariates for confounding control result in statistically inconsistent estimators of the true EIN, NNE, and NNT. This study presents a theoretical framework for statistically consistent point and interval estimation of the NNE, EIN and NNE in observational studies with unmeasured confounders. Methods Using Rubin’s potential outcomes framework, this study explicitly defines the NNT and its derived indices, EIN and NNE, as causal measures. Then, we use instrumental variables to introduce a novel method to estimate the three aforementioned indices in observational studies where the omission of unmeasured confounders cannot be ruled out. To illustrate the novel methods, we present two analytical examples – double logit and double probit models. Next, a corresponding simulation study and a real-world data example are presented. Results This study provides an explicit causal formulation of the EIN, NNE, and NNT indices and a comprehensive theoretical framework for their point and interval estimation using the G-estimators in observational studies with unmeasured confounders. The analytical proofs and the corresponding simulation study illustrate the improved performance of the new estimation method compared to the available methods in terms of consistency and the confidence intervals empirical coverage rates. Conclusions In observational studies, traditional estimation methods to estimate the EIN, NNE, or NNT result in statistically inconsistent estimators. We introduce a novel estimation method that overcomes this pitfall. The novel method produces consistent estimators and reliable CIs for the true EIN, NNE, and NNT. Such a method may facilitate more accurate clinical decision-making and the development of efficient public health policies.

Objectives Determining the causal relationship between exposure and outcome is the goal of many observational studies. However, the selection of subjects into the study population, either voluntary or involuntary, may result in estimates that suffer from selection bias. To assess the robustness of the estimates as well as the magnitude of the bias, bounds for the bias can be calculated. Previous bounds for selection bias often require the specification of unknown relative risks, which might be difficult to provide. Here, alternative bounds based on observed data and unknown outcome probabilities are proposed. These unknown probabilities may be easier to specify than unknown relative risks. Methods I derive alternative bounds from the definitions of the causal estimands using the potential outcomes framework, under specific assumptions. The bounds are expressed using observed data and unobserved outcome probabilities. The bounds are compared to previously reported bounds in a simulation study. Furthermore, a study of perinatal risk factors for type 1 diabetes is provided as a motivating example. Results I show that the proposed bounds are often informative when the exposure and outcome are sufficiently common, especially for the risk difference in the total population. It is also noted that the proposed bounds can be uninformative when the exposure and outcome are rare. Furthermore, it is noted that previously proposed assumption-free bounds are special cases of the new bounds when the sensitivity parameters are set to their most conservative values. Conclusions Depending on the data generating process and causal estimand of interest, the proposed bounds can be tighter or wider than the reference bounds. Importantly, in cases with sufficiently common outcome and exposure, the proposed bounds are often informative, especially for the risk difference in the total population. It is also noted that, in some cases, the new bounds can be wider than the reference bounds. However, the proposed bounds based on unobserved probabilities may in some cases be easier to specify than the reference bounds based on unknown relative risks.