Instrumental variable regression via kernel maximum moment loss
-
Rui Zhang
Abstract
We investigate a simple objective for nonlinear instrumental variable (IV) regression based on a kernelized conditional moment restriction known as a maximum moment restriction (MMR). The MMR objective is formulated by maximizing the interaction between the residual and the instruments belonging to a unit ball in a reproducing kernel Hilbert space. First, it allows us to simplify the IV regression as an empirical risk minimization problem, where the risk function depends on the reproducing kernel on the instrument and can be estimated by a U-statistic or V-statistic. Second, on the basis this simplification, we are able to provide consistency and asymptotic normality results in both parametric and nonparametric settings. Finally, we provide easy-to-use IV regression algorithms with an efficient hyperparameter selection procedure. We demonstrate the effectiveness of our algorithms using experiments on both synthetic and real-world data.
1 Introduction
Instrumental variables (IVs) have become standard tools for economists, epidemiologists, and social scientists to uncover causal relationships from observational data [1,2]. Randomization of treatments or policies has been perceived as the gold standard for such tasks, but it is generally prohibitive in many real-world scenarios due to time constraints or ethical concerns. When treatment assignment is not randomized, it is generally impossible to discern between the causal effect of treatments and spurious correlations that are induced by unobserved factors. Instead, IVs enable the investigators to incorporate natural variation through an IV that is associated with the treatments, but not with the outcome variable, other than through its effect on the treatments. In economics, for instance, the season of birth was used as an IV to study the return from schooling, which measures causal effect of education on labor market earning [3]. In genetic epidemiology, the idea to use genetic variants as IVs, known as Mendelian randomization, has also gained increasing popularity [4,5].
There are numerous variations of methods for dealing with IV. Classical methods for IV regression often rely on a linearity assumption in which a two-stage least squares (2SLS) is the most popular technique [6]. The generalized method of moments (GMM) of ref. [7], which imposes the orthogonality restrictions, can also be used for the linear IV regression. For nonlinear regression, numerous methodologies have emerged in the field of nonparametric IV [8–11] and recent machine learning [12–18]. These estimators can be categorized into two general approaches. The former generalizes a two-stage optimization procedure [12,16], whereas the latter requires solving a minimax optimization problem [13,14,17,19,20]. While the equivalence between 2SLS and GMM is known in the linear setting, the connection between two-stage procedure and minimax optimization in the nonlinear setting was established recently, see, e.g., refs [15], [17, Appendix F], and [21, Sec 3.3]. For statistical inference, numerous approaches have been developed using asymptotic distributions of each point of uniform bands of the target functions by the sieve method [22–26].
In this article, we propose a novel nonlinear IV regression framework named Maximum Moment Restriction-IV (MMR-IV), which utilizes the reproducing kernel Hilbert spaces (RKHSs) and an associated positive definite kernel from the machine learning domain.
The MMR-IV possesses mainly two important features: (i) It reformulates the original conditional moment restriction (CMR) into a single-step empirical risk minimization (ERM) problem, by using the MMR framework of ref. [27] with the kernel and RKHSes. (ii) Based on the ERM problem, it reveals a closed form of the solution by U/V-statistics techniques [28]. Based on this framework, we propose two practical algorithms based on kernel functions and neural networks (NNs), which we call MMR-IV (RKHS) and MMR-IV (NN), respectively.
Our MMR-IV has the following advantages. First, the derived closed-form expression is computationally simple and stable. Further, owing to the generic structure of ERM, we can substitute NNs for kernel functions, which is suitable for highly nonlinear modeling. Second, MMR-IV can identify the optimal solution, i.e., the minimizer of the risk function is guaranteed to be unique, by using the positive definite property of the kernel function. Third, we can identify a distribution that the solution follows in the large sample limit, i.e., the asymptotic distribution. The identified asymptotic distribution allows us to analyze uncertainties of algorithms and estimators, such as statistical tests and confidence intervals. This result is applicable to the case where the model is both parametric and nonparametric, and especially in the nonparametric case, we utilize the functional differentiation technique [29] to derive the results. Moreover, on the basis of the asymptotic distribution, we discuss how the choice of kernel functions affects the efficiency of MMR-IV. Fourth, we provide a hyperparameter selection scheme, by developing a novel interpretation of MMR-IV from a Gaussian process (GP) perspective. This interpretation is obtained by mapping our ERM form to a likelihood of GPs. Since hyperparameter selection for IV regression is a nontrivial problem in both minimax and two-stage formulation, our framework sheds light on efficient methods to solve this problem. In the experimental section, we demonstrate the superiority of the proposed framework by validating its accuracy and effectiveness and show its usefulness in real data analysis.
Our contributions can be summarized as follows:
We study the simple ERM problem converted from the minimax optimization, and then propose the MMR-IV algorithms to minimize the risk based on kernel functions and NNs. This algorithm avoids the difficulties of minimax problems and is computationally simple and stable.
Our method can identify unique solutions by selecting positive definite kernel functions. Our analysis also reveals how the asymptotic efficiency varies with the choice of kernel functions.
We derive the asymptotic normal distribution of the MMR-IV estimators in both parametric and nonparametric settings. This result is derived from the ERM form and allows for statistical inference.
We develop the efficient framework to select hyperparameters of MMR-IV. This approach is based on the interpretation of MMR-IV with kernel functions from a GP perspective.
We compare our methods to a wide range of baselines on different settings and show that MMR-IV has competitive performance.
The rest of this article is organized as follows. Section 2 introduces the IV regression problem and the MMR framework. Our methods are presented in Section 3, followed by the hyperparameter selection in Section 4. Next, we provide the consistency and asymptotic normality results in Section 5. We then provide a thorough discussion on related works in Section 6. Finally, the experimental results are presented in Section 7. All proofs of the main results can be found in the appendix.
2 Preliminaries
2.1 IV regression
Following standard setting in the literature [12–16], let
where we assume that
Instrumental variable (IV). To address this problem, we assume access to an instrumental variable
A causal graph depicting an IV
Conditional moment restriction (CMR). Together with the structural assumption in (1), these conditions imply that
That is, there exists an infinite number of moment conditions, each of which is indexed by the function
2.2 Maximum moment restriction (MMR)
Throughout this article, we assume that
Instead of considering all measurable functions
The benefits of this formulation are twofold. First, it is computationally intractable to learn
Lemma 1
([27], Theorem 3.3) Assume that
We assume throughout that the reproducing kernel
Assumption 1
The kernel
Popular kernel functions that satisfy Assumption 1 are the Gaussian RBF kernel and Laplacian kernel
where
where
Next, we further assume the identification for the minimizer of
Assumption 2
Consider the function space
A sufficient condition for identification follows from the completeness property of
3 Our method
We propose to learn
It is instructive to note that population risk
Proposition 1
Assume that the kernel k is ISPD. Then for every real-valued measurable function f,
Proposition 1 holds as long as the kernel
Proposition 2
If
3.1 Empirical risk minimization
The previous results pave the way for an ERM framework [43] to be used in our work. That is, given an i.i.d. sample
and
Both forms of empirical risk can be used as a basis for a consistent estimation of
where
On the basis of
where
Kernelized GMM. We may view the objective (5) from the GMM perspective [32]. The assumption that the instruments
Hence, our objective
3.1.1 Theoretical perspective: Identification and efficiency
Before introducing practical algorithms, we discuss the theoretical aspects of our framework, specifically, the identifiability and efficiency issues by changing the norm constraint in the formulation. The key difference between our risk
Note that the test function
Identification: An advantage of our design (3) with RKHSes is that it uniquely defines the function that minimizes the risk. Since RKHSes equips a positive definite kernel, and we further assume the kernel is ISPD in Assumption 1, the risk
Efficiency of IV: Another important issue is the efficiency of the estimator, i.e., the variance of the asymptotic distribution of estimators. For the linear regression problem with finite-dimensional parameters, we can define the optimal IV, i.e., the estimator with the IV that minimizes the asymptotic variance [8]. In contrast, our regression model in this setup is nonlinear, infinite-dimensional, and also based on U-statistics, hence it is not trivial to discuss optimality.
To address the latter issue, we provide the variance of the asymptotic distribution of our estimator as a way to measure efficiency. In Proposition 5 in Section 5, we derive asymptotic Gaussian approximations of the estimator, both pointwise and uniformly, that reveal their variance. The result in Proposition 5 is summarized as follows. We first define coefficients
where
compared to the case without kernel functions. The case with
3.2 Practical MMR-IV algorithms
A workflow of our algorithm based on
| Algorithm 1 MMR-IV |
|---|
|
Input: Dataset
|
|
Output: The estimate of
|
| 1: Compute the kernel matrix
|
| 2: Define the residual
|
| 3:
|
| 4: return
|
Deep NNs. In the parametric setting, the function class
| Algorithm 2 MMR-IV (NN) |
|---|
|
Input: Dataset
|
|
Output: predictive value at
|
| 1: Compute
|
| 2:
|
| 3: return
|
Kernel machines. In a nonparametric setting, the function class
where
Nyström approximation. The MMR-IV (RKHS) algorithm is computationally costly for large datasets as it requires a matrix inversion. To improve the scalability, we resort to Nyström approximation [48] to accelerate the matrix inversion in
First, we randomly select a subset of
We will refer to this algorithm as MMR-IV (Nyström); Algorithm 3. The runtime complexity of this algorithm is
| Algorithm 3 MMR-IV (Nyström) |
|---|
|
Input: Dataset
|
|
Output: predictive value at
|
| 1:
|
| 2:
|
| 3:
|
| 4:
|
| 5: return
|
4 Hyperparameter selection
We discuss the selection of hyperparameters such as the regularization coefficient
A summary of proposed algorithms. Following the KMMR framework, two methods – MMR-IV (NN) and MMR-IV (RKHS) – are proposed, where
4.1 Leave-M-Out Cross Validation (LMOCV)
We utilize the LMOCV approach for the hyperparameter selection, which is described as follows. Let
| 1: |
|
| 2: | Split the data into
|
| 3: |
for each
|
| 4: |
|
| 5: |
|
| 6: |
|
| 7: |
|
| 8: |
|
| 9: | end for |
| 10: | Output
|
One disadvantage of this approach is that it is computationally expensive because the empirical risks are optimized many times. To alleviate this problem, we study an analytical form of the score function
4.2 GP interpretation
Our interest is to obtain an analytical empirical risk for MMR-IV (Nyström) with the V-statistic objective (7). Toward the goal, we relate our ERM form to a stochastic model with a GP, inspired by ref. [51]. Different from the counterpart in the ordinary kernel regression, we need to apply additional analysis to deal with the additional weight matrix
We build the connection between the V-statistic risk and the posterior distribution of a GP model in this subsection, after which we will show that the MMR-IV (Nyström) estimator is equivalent to the Maximum-A-Posteriori estimator based on the GP. The relationship is inspired by the similarity of our objective function to that of GLS.
Prior and likelihood. Let us consider a GP prior over a space of functions
Here,
Hence, the likelihood on both
This likelihood function is regarded as a generalized version of a loss function of generalized least squares with heteroskedastic noise [52]. That is, if we set
This likelihood assumes correlated errors with the normal distribution. It is closely related to the standard GP regression that has a similar Gaussian likelihood, but the covariance matrix is computed on
Maximum
The MAP estimate of
We can see that the MAP estimator returns the same result as that of the MMR-IV (RKHS) estimator given that
We summarize the result in Proposition 3. Such a GP interpretation is used for an elegant derivation of the analytical CV error in the following subsection.
Proposition 3
Given
4.3 Analytical form of error
Now, we derive the analytical error of LMOCV via Bayes’ rules based on the posterior GP. We split the whole dataset
which implies that
where
where
5 Theoretical result: Consistency and asymptotic normality
We provide the consistency and asymptotic normality of
5.1 Consistency
We first show the consistency of
Proposition 4
(Consistency of
If
5.2 Asymptotic normality with finite-dimension case
We analyze the asymptotic normality of the estimator
We first consider the
Theorem 1
(Asymptotic normality of
holds, where
5.2.1 Connection to efficiency
The variance of the asymptotic distribution varies with the choice of kernel. To see this, we consider the constants
Their positiveness and existence are guaranteed by Assumption 1. By Theorem 4.51 in ref. [37],
where
Proposition 5
Suppose that Assumption
1
holds. Define
This result evaluates the increase in the asymptotic variance due to the choice of the kernel by the upper bound. Strictly speaking, as the eigenvalues
Intuitively, this result represents the difficulty of choosing the kernel function as the optimal IV in nonlinear IV regression. Unlike the case of linear regression [30], the nonlinear IV regression does not provide information such as which eigenspaces of data have a large variance. Thus, the anisotropy of the eigenvalues of kernels may worsen the efficiency in the worst case. However, efficiency can be improved if the anisotropy of the eigenvalues fits the data. If such properties could be known, one can develop an optimal kernel selection procedure based on them.
5.3 Asymptotic normality of infinite-dimension case
We show the asymptotic normality of an infinite-dimensional estimator
Assumption 3
(Low-entropy condition) There exists
for every
Although the covering number condition
Theorem 2
(Asymptotic normality of
with
Note that
We provide the detailed derivation in the appendix. A specific form of the operator
This nonparametric asymptotic normality is a generic result that leads to finite-dimensional asymptotic normality. That is, for every finite set
where
Remark 1
(On asymptotic normality) Theorem 2 and
6 Related work
Several extensions of 2SLS and GMM exist for the nonlinear IV problem. In the two-stage approach, the function
The IV regression has also recently received attention in the machine learning community. Hartford et al. [12] proposed to solve the integral equation by first estimating
Our work follows in spirit many GMM-based approaches for IV regression, namely, [13,14,27]. We adopt the MMR framework of ref. [27], which only considers a conditional moment testing problem, to the parameter estimation problem of IV regression, and hence, our approaches and both theoretical and experimental analyses are different from theirs. In fact, this framework was initially inspired by refs [13,14], which instead parametrize the instruments by deep NNs. By combining the GMM framework with RKHS functions, the objective function can be evaluated in a closed form as shown in our work. As a result, our IV estimate can be obtained by minimizing the empirical risk, as opposed to an adversarial optimization [13,14]. It is important to note that recently these adversarial approaches have been improved similarly with RKHSs by refs [18,50], while there is a major difference. First, ref. [50] extends the work of ref. [13] to a single-stage algorithm similar to our MMR-IV (RKHS) [50, Section 4]. Although both works employ RKHSs in the minimax frameworks, ref. [50] incorporate a Tikhonov regularization on
We also discuss connections to existing works in the areas outside of the IV setting in Section B in Appendix.
7 Experimental results
We present the experimental results in a wide range of settings for IV estimation. Following refs [13,14], we consider both low- and high-dimensional scenarios. In the experiments, we compare our algorithms to the following baseline algorithms:
DirectNN: A standard least square regression on
2SLS: A vanilla 2SLS on raw
Poly2SLS: The 2SLS that is performed on polynomial features of
DeepIV [12]: A nonlinear extension of 2SLS using deep NNs. We use the implementation available at https://github.com/microsoft/EconML.
KernelIV [16]: A generalization of 2SLS by modeling relations among
AGMM [13]: This algorithm models
DeepGMM [14]: This algorithm is a variant of AGMM with optimal inverse-covariance weighting matrix. The publicly available implementation at https://github.com/CausalML/DeepGMM is used, and all of the aforementioned baselines are provided in the package except KernelIV.
AGMM-K [50]: This algorithm extends AGMM by modeling
DualIV [15]: This algorithm solves the CMR via a saddle-point reformulation and obtains a minimax problem. By using an RKHS for the inner maximization, an objective similar to ours is obtained, while the central kernel matrix is evaluated on
7.1 Low-dimensional scenarios
Following [14], we employ the following data generation process:
where
7.1.1 Experimental settings
For the experiments on low-dimensional data, we consider both small-sample (
Hyperparameters of NNs in Algorithm 2, including the learning rate and the regularization parameter, are chosen by a two-fold CV for fair comparisons with the baselines. As per the dimensions of
Hyperparameters of NNs used in the experiments
| Scenario | Model
|
Learning rates |
|
|---|---|---|---|
| Low dimensional | FCNN(100,100) | (
|
(
|
| MNISTZ | FCNN(100,100) | (
|
(
|
| MNISTX | CNN | (
|
(
|
| MNISTXZ | CNN | (
|
(
|
| Mendelian | FCNN(100,100) | (
|
(
|
In contrast to our NN-based method, the RKHS-based method in Algorithm 3 has the analytic form of CV error. For the kernel function on instruments, we employ the sum of Gaussian kernels
and the Gaussian kernel for
7.1.2 Experiment results
Tables 2 and 3 report the results for the large-sample and small-sample regimes respectively. First, under the influence of confounders, DirectNN performs worst as it does not use instruments. Second, MMR-IVs perform reasonably well in both small-sample and large-sample regimes. For the linear function, 2SLS and Poly2SLS tend to outperform other algorithms as the linearity assumption is satisfied in this case. For nonlinear functions, some NN-based methods show competitive performance in certain cases. Notably, GMM+NN has unstable performance as the function
The mean square error (MSE)
| Algorithm | True function
|
|||
|---|---|---|---|---|
| abs | linear | sin | step | |
| DirectNN | 0.116
|
0.035
|
0.189
|
0.199
|
| 2SLS | 0.522
|
0.000
|
0.254
|
0.050
|
| Poly2SLS | 0.083
|
0.000
|
0.133
|
0.039
|
| GMM+NN | 0.318
|
0.044
|
0.694
|
0.500
|
| AGMM | 0.600
|
0.025
|
0.274
|
0.047
|
| DeepIV | 0.247
|
0.056
|
0.165
|
0.038
|
| DeepGMM | 0.027
|
0.005
|
0.160
|
0.025
|
| KernelIV |
0.019
|
0.009
|
0.046
|
0.026
|
| AGMM-K | 181
|
2.34
|
19.4
|
4.13
|
| DualIV | 0.344
|
0.034
|
0.379
|
0.345
|
| SieveIV | 0.170
|
0.279
|
0.021
|
6.89
|
| MMR-IV NN |
0.011
|
0.005
|
0.153
|
0.040
|
| MMR-IV Nys |
0.011
|
0.001
|
0.006
|
0.020
|
The MSE
| Algorithm | True function
|
|||
|---|---|---|---|---|
| abs | linear | sin | step | |
| DirectNN | 0.143
|
0.046
|
0.404
|
0.253
|
| 2SLS | 0.564
|
0.003
|
0.304
|
0.076
|
| Poly2SLS | 0.125
|
0.003
|
0.164
|
0.077
|
| GMM+NN | 0.792
|
0.203
|
1.56
|
0.550
|
| AGMM | 0.031
|
0.011
|
0.330
|
0.080
|
| DeepIV | 0.204
|
0.047
|
0.197
|
0.039
|
| DeepGMM |
0.022
|
0.032
|
0.143
|
0.039
|
| KernelIV | 0.063
|
0.024
|
0.086
|
0.055
|
| AGMM-K | 12.3
|
1.32
|
1.57
|
1.71
|
| DualIV | 0.202
|
0.103
|
0.251
|
0.362
|
| SieveIV | 0.466
|
0.452
|
0.363
|
9.07
|
| MMR-IV (NN) |
0.019
|
0.004
|
0.292
|
0.075
|
| MMR-IV (RKHS) | 0.030
|
0.011
|
0.075
|
0.057
|
We also record the runtimes of all methods on the large-sample regime and report them in Figure 3. Compared to the NN-based methods, i.e., AGMM, DeepIV, DeepGMM, our MMR-IV (NN) is the most computationally efficient method, which is clearly a result of a simpler objective. By using a minimax optimization between two NNs, AGMM is the least efficient method. DeepGMM and DeepIV are more efficient than AGMM, but are less efficient than MMR-IV (NN). Finally, all four RKHS-based methods, namely, KernelIV, AGMM-K, DualIV, and MMR-IV (RKHS), have similar computational time. All four methods are observed to scale poorly on large datasets. In contrast, SieveIV is the most computationally efficient method in this experiment.
Runtime in the large-sample regime (
7.2 High-dimensional structured scenarios
In a high-dimensional setting, we employ the same data-generating process as in Section 7.1. We consider only the absolute function for
7.2.1 Experimental settings
We sample
7.2.2 Experiment results
We report the results in Table 4. Like [14], we observe that the DeepIV code returns NaN when
The MSE
| Algorithm | Setting | ||
|---|---|---|---|
|
|
|
|
|
| DirectNN | 0.134
|
0.229
|
0.196
|
| 2SLS | 0.563
|
|
|
| Ridge2SLS | 0.567
|
0.431
|
0.705
|
| GMM+NN | 0.121
|
0.235
|
0.240
|
| AGMM |
0.017
|
0.732
|
0.529
|
| DeepIV | 0.114
|
n/a | n/a |
| DeepGMM | 0.038
|
0.315
|
0.333
|
| AGMM-K+NN | 0.021
|
1.05
|
0.327
|
| MMR-IV (NN) | 0.024
|
0.124
|
0.130
|
| MMR-IV (Nys) |
0.015
|
0.442
|
0.425
|
7.3 Mendelian randomization
We demonstrate our method in the setting of Mendelian randomization, which relies on genetic variants that satisfy the IV assumptions. The “exposure”
where
In Mendelian randomization, it is known that genetic variants may act as weak IVs [5,68,69], so this experiment aims to evaluate the sensitivity of different methods to the number of instruments (
Figure 4 depicts the experimental results. Overall, 2SLS performs well on all settings due to the linearity assumption, except particular sensitivity to the number of (weak) instruments, which is a well-known property of 2SLS [1]. Although imposing no such assumption, MMR-IVs perform competitively and even more stably, since the information of instruments is effectively captured by the kernel
The MSE of different methods on Mendelian randomization experiments as we vary the numbers of instruments (left), the strength of confounders to exposures
7.4 Vitamin D data
Finally, we apply our algorithm to the Vitamin D data [70, Section 5.1]. The data were collected from a 10-year study on 2,571 individuals aged 40 to 71 and 4 variables are employed: age (at baseline), filaggrin (a binary indicator of filaggrin mutations), VitD (Vitamin D level at baseline), and death (a binary indicator of death during the study). The goal is to evaluate the potential effect of VitD on death. We follow [70] by controlling age in the analyses, using filaggrin as an instrument, and then by applying the MMR-IV (Nyström) algorithm. Sjolander and Martinussen [70] modeled the effect of VitD on death by a generalized linear model (GLM) and found that the effect is insignificant by 2SLS (
![Figure 5
Estimated effect of vitamin D level on mortality rate, controlled by age. All plots depict normalized contours of
f
′
(
X
,
C
)
f^{\prime} \left(X,C)
defined in Section 7.4.1, where blue represents low mortality rate and yellow the opposite. We can divide each plot roughly into the left (young age) and right (old age) parts. While the right parts reflect similar information (i.e., lower vitamin D levels and old age lead to a higher mortality rate), the left parts are different. (a) A high level of vitamin D at a young age can result in a high mortality rate, which is counterintuitive. A plausible explanation is that it is caused by some unobserved confounders between vitamin D levels and mortality rates. (b) On the other hand, this spurious effect disappears when the filaggrin mutation is used as an instrument, i.e., a low vitamin D level at a young age has only a slight effect on death, but a more adverse effect at an old age [71]. This comparison demonstrates the benefit of an IV. (c) and (d) correspond to the results obtained by using [70]’ GLM, from which we can draw similar conclusions. It is noteworthy that MMR-IV allows more flexible nonlinearity for a causal effect. (a) Kernel ridge regression (IV: none). (b) MMRIV (IV: Filaggrin mutation). (c) GLM (IV: none). (d) 2SLS + GLM (IV: Filaggrin mutation).](/document/doi/10.1515/jci-2022-0073/asset/graphic/j_jci-2022-0073_fig_005.jpg)
Estimated effect of vitamin D level on mortality rate, controlled by age. All plots depict normalized contours of
7.4.1 Experimental settings on Vitamin D data
We normalize each variable to have a zero mean and unit variance to reduce the influence of different scales. We consider two cases: (i) without instruments and (ii) with instruments. By without instrument, we mean that the
Distribution of Vitamin D data. Data points are plotted in the middle, the solid curve and histogram on the right describe the kernel density estimation and histogram of Vitamin D, and those on the top are for age.
In this experiment, we use age as a control variable by considering a structural equation model, which is similar to the model (1) except for the presence of the controlled (exogenous) variable
where
relevance:
exclusion restriction:
unconfounded instrument(s):
for every measurable function
8 Conclusion and discussion
Learning causal relations when hidden confounders are present is a cornerstone of reliable decision-making. IV regression is a standard tool to tackle this task, but currently faces challenges in nonlinear settings. This work presents a simple kernel-based framework that overcomes some of these challenges. We employ RKHS theory in the reformulation of CMR as a MMR based on which we can approach the problem from the ERM perspective. As we demonstrate, this framework not only facilitates theoretical analysis but also results in easy-to-use algorithms that perform well in practice compared to existing methods. This article also shows a way of combining the elegant theoretical approach of kernel methods with the practical merits of deep NNs. Despite these advantages, the optimal choice of the kernel
The efficiency issue, i.e., the volume of the error by our estimator, is one of the major unsolved problems. Importantly, since the efficiency heavily depends on the choice of kernel function
In the classical IV problem with parametric models, the optimal choice of IV is based on the variance of the asymptotic distribution of estimators [30]. However, our setup corresponds to a nonparametric regression problem, which requires a more complicated approach. A recent work [72] tackles this problem, but some arbitrariness remains. Another challenge is recent works on a variational method of moments [73]. This work provides the optimal weighted objective and the closed-form expression of the estimator, which can achieve the semiparametric efficiency bound for solving the CMR. This approach would be highly relevant to the discussion of efficiency in our setting.
Acknowledgments
We thank Yuchen Zhu, Vasilis Syrgkanis, and Heiner Kremer for fruitful discussions. We are also indebted to anonymous reviewers for their constructive feedback on the initial draft of this manuscript.
-
Funding information: Rui Zhang was sponsored by PhD scholarship from Data61 and by the Empirical Inference Department of Max Planck Institute. Masaaki Imaizumi was supported by JSPS KAKENHI (Grant Number 18K18114) and JST Presto (Grant Number JPMJPR1852). Bernhard Schölkopf is a member of the Machine Learning Cluster of Excellence, EXC number 2064/1 – Project number 390727645. This work was partly supported by the German Federal Ministry of Education and Research (BMBF): Tübingen AI Center, FKZ: 01IS18039B.
-
Author contributions: All authors have accepted responsibility for the entire content of this article and approved its submission. The study and manuscript have received significant contributions from all authors whose contributions are reflected in the list of authors.
-
Conflict of interest: The authors have a conflict of interest with the Australian National University, Data61 of Commonwealth Scientific and Industrial Research Organisation (CSIRO), Australia, Max Planck Institute (MPI), CISPA–Helmholtz Center for Information Security, and the University of Tokyo.
-
Ethical approval: The conducted research is not related to either human or animal use.
-
Data availability statement: The datasets generated during and analyzed during the current study are available at https://github.com/RuiZhang2016/MMRIV, where the implementation of the approaches is also provided.
Appendix A Detailed proofs
This section contains detailed proofs of the results that are missing in the main article. Most of the proofs on consistency and asymptotic normality take advantages of the useful resource by ref. [53]. Readers are referred to it for more detailed discussions on assumptions. Note that our proofs are based on the normed
A.1 Proof of Lemma 1
Proof
Since
where we used the reproducing property of
as required.□
A.2 Proof of Proposition 1
Proof
First, the law of iterated expectation implies that
By Lemma 1, we know that
where we define
A.3 Proof of Proposition 2
Proof
Given
By Lemma 1, we know that
The equality (a) is obtained by considering
A.4 Uniform convergence of risk functionals
The results presented in this section are used to prove the consistency of
Lemma A1
(Uniform consistency of
Proof
First, let
The third inequality follows from the Cauchy-Schwarz inequality. Since
Hence, our assertion follows from [53, Lemma 8.5].□
Lemma A2
(Uniform consistency of
Proof
First, let
The third inequality follows from the Cauchy-Schwarz inequality. Since
Furthermore, we show that
A.5 Indefiniteness of weight matrix
W
U
Theorem A1
If Assumption
1
holds,
Proof
By definition, we have
where
A.6 Consistency of
f
ˆ
V
with convex
Ω
(
f
)
Theorem A2
(Consistency of
Proof
Given
Recall that
A.7 Proof of Proposition 4
Proof
From the conditions of Lemma A1, we know that
Consequently, we assert the conclusion by [53, Theorem 2.1].□
A.8 Consistency of
f
ˆ
U
Theorem A3
(Consistency of
Proof
By the conditions of Lemma A2, we know that
Consequently, we assert the conclusion by [53, Theorem 2.1].□
A.9 Asymptotic normality of
θ
ˆ
U
In this section, we consider the regularized U-statistic risk
We will assume that
A.10 Asymptotic normality of
∇
θ
R
^
U
,
λ
(
f
θ
∗
)
We first show the asymptotic normality of
Lemma A3
Suppose that
Proof
The proof follows from [28, Sections 5.5.1 and 5.5.2] and we need to show that (i)
To prove (ii), we first note that
As the aforementioned equation holds for any
where we note that
This concludes the proof.□
A.11 Uniform consistency of
∇
θ
2
R
^
U
,
λ
(
f
θ
)
Next, we consider the second derivative
Lemma A4
Suppose that
Proof
The proof is similar to that of Lemma A2 and both applies extended results of [53, Lemma 2.4]. As
We first show
We then show that (ii)
Therefore, we obtain
This concludes the proof.□
Theorem A4
(Asymptotic normality of
Proof
The proof follows by [53, Theorem 3.1], and we need to show that (i)
The proof of (i) is very similar to Theorem A3 except that we consider finite dimensional parameter space instead of functional space. For a neat proof, we would like to omit the detailed proof here. We can first show the uniform consistency
Furthermore, from the conditions that
If for all
Finally by Lemma A4, we know that
A.12 Proof of Theorem 1
We restate the notations
Lemma A5
Suppose that conditions of Lemma
A3
hold. Then
Proof
As
Lemma A6
Suppose that
Proof
We apply [53, Lemma 8.5] for this proof and need to show (i)
We first see that
and
Thus, we assert the conclusion by [53, Lemma 8.5].□
Proof of Theorem 1
The proof is the same as that of Theorem A4 except that
A.13 Asymptotic normality in the infinite-dimension case
We first state the asymptotic normality theorem for
Theorem A5
Suppose Assumption
1
holds, l is a bounded kernel,
An exact covariance of
To prove the theorem, we provide some notation. Let
where
where
Further, we define a functional associated with measure. We consider functional spaces
and also let
Uniform central limit theorem: We first achieve the uniform convergence. Note that a measure
Theorem A6
(Theorem 4.4 in ref. [76]) Suppose
holds, for all
Here,
with
To apply the theorem, we have to show that
Lemma A7
For any
Proof of Lemma A7
We simply obtain the following:
as required.□
Lemma A8
Suppose the assumptions of Theorem 2 hold. Then, the followings hold:
For any
Proof of Lemma A8
For preparation, fix
For the first condition, as shown in equation (2.1.7) in ref. [55], it is sufficient to show that
for arbitrary
with constants
For the second condition, we have the similar strategy. For any
with a constant
since
From Theorem A6 and Lemma A8, we rewrite the central limit theorem utilizing terms of functionals. Note that
Learning map and functional delta method: We consider a learning map
Obviously, we have
We consider a derivative of
First, we define a partial derivative of the map
To this end, we consider the following derivative
Here,
and
Lemma A9
If the assumptions in Theorem
2
hold, then
Proof of Lemma A9
We consider a sequence of functions
The convergence follows the definition of
Here, we consider its RKHS-type formulation of
Obviously,
This equivalence follows Theorem 7.4.1 and Lemma 8.7.1 in ref. [77].
Next, we apply the implicit function theorem to obtain an explicit formula of the derivative of
which follows (b) in Lemma A.2 in ref. [29]. Its basic properties are provided in the following result.
Lemma A10
If Assumption
1
and the assumptions in Theorem
2
hold, then
Proof of Lemma A10
By (b) in Lemma A.2 in ref. [29],
For the injectivity, we fix nonzero
The last equality follows the property of
For the compactness, we follow Lemma A.5 in ref. [29] and obtain that operators
We define the Gateau derivative of
where
Proposition A1
Suppose the assumptions in Theorem
2
hold. For
Proof of Proposition A1
This proof has the following two steps, (i) define a proxy operator
(i) Define
For simple derivatives, Lemma A.2 in ref. [29] provides
and
(ii) Apply the implicit function theorem: By its definition and the optimal conditions, we have
Also, we obtain
Then, for each
and it satisfies
Also, we have
Then, we obtain the statement.□
Now, we are ready to prove Theorems 2 and A5.
Proof of Theorem A5
As a preparation, we mention that
Then, we apply the functional delta method. As shown in Theorem A6 and Lemma A8, we have
Hence, we obtain
since
in
Proof of Theorem 2
This proof is completed by substituting the Central Limit Theorem part (Theorem A6) of the proof of Theorem A5. From Section 3 in ref. [78], the V- and U-processes have the same limit distribution asymptotically, so the same result holds.□
A.14 Proof of Proposition 5
Proof of Proposition 5
For short, we write
We start by investigating
where
On the basis of these facts, we obtain the following form
where
By using the result, we study
Let
Finally, we study
By using the statement, we study the Frobenisu norm
Landau’s Big O notation in the last equality picks up the terms depends only
B Related studies beyond IV regression setting
Beyond the IV regression, there are numerous prior studies that are related to ours, especially in policy evaluation, reinforcement learning, and causal inference. First, the idea of “kernel loss” was proposed in ref. [79] to estimate the value function in reinforcement learning. Similar ideas have been used to estimate the importance ratio of two state or state-action distributions in refs [80,81] and to estimate the average policy effect (APE) and policy learning in refs [82,83]. Second, in the area of causal inference, ref. [84] employed a similar technique to estimate an average treatment effect. Despite the methodological similarity to our work, these works did not consider the IV regression setting and the analytical cross validation error. We will first introduce the objective functions of the aforementioned works for a better understanding of the connection and then highlight the differences, challenges, and novelties of our work.
Although estimating different subjects
where
Compared with our population and empirical risks (4) and (5), the kernel loss (A2) has a similar quadratic form. The difference is that the kernel loss has variables involved in both the kernel function and the residual, whereas the kernel function in our objective depends on the instrument
The difference in losses also simplifies the estimation in the related work and requires us to perform new analyses. Specifically, the consistency of the estimators in the aforementioned related work holds under mild conditions. That is, in the estimation of the value function, minimizing the population objective function to zero guarantees a solution for the Bellman equation, which is consistent to the value function according to the unique solution property of the Bellman equation [79]; for the importance ratio and the APE estimation, the consistency holds under mild conditions of data distributions [80,82]. In contrast, the estimator obtained by minimizing the MMR risk (4) to zero is hardly consistent to the true
References
[1] Angrist JD, Pischke JS. Mostly harmless econometrics: an empiricistas companion. Princeton and Oxford: Princeton University Press; 2008. 10.2307/j.ctvcm4j72Search in Google Scholar
[2] Klungel O, Jamal Uddin M, de Boer A, Belitser S, Groenwold R, Roes K. Instrumental variable analysis in epidemiologic studies: an overview of the estimation methods. Pharm Anal Acta. 2015;6(353):2. 10.4172/2153-2435.1000353Search in Google Scholar
[3] Card D. The causal effect of education on earnings. In: Ashenfelter O, Card D, editors. Handbook of labor economics. vol. 3 of Handbook of labor economics. Elsevier; 1999. p. 1801–63. 10.1016/S1573-4463(99)03011-4Search in Google Scholar
[4] Burgess S, Small DS, Thompson SG. A review of instrumental variable estimators for Mendelian randomization. Statist Meth Med Res. 2017;26(5):2333–55. 10.1177/0962280215597579Search in Google Scholar PubMed PubMed Central
[5] Burgess S, Foley CN, Allara E, Staley JR, Howson JMM. A robust and efficient method for Mendelian randomization with hundreds of genetic variants. Nature Commun. 2020;11(1):376. 10.1038/s41467-019-14156-4Search in Google Scholar PubMed PubMed Central
[6] Angrist JD, Imbens GW, Rubin DB. Identification of causal effects using instrumental variables. J Amer Stat Assoc. 1996;91(434):444–55. 10.1080/01621459.1996.10476902Search in Google Scholar
[7] Hansen LP. Large sample properties of generalized method of moments estimators. Econometrica. 1982;50(4):1029–54. 10.2307/1912775Search in Google Scholar
[8] Newey WK, Powell JL. Instrumental variable estimation of nonparametric models. Econometrica. 2003;71(5):1565–78. 10.1111/1468-0262.00459Search in Google Scholar
[9] Hall P, Horowitz JL. Nonparametric methods for inference in the presence of instrumental variables. Ann Stat. 2005 12;33(6):2904–2929. 10.1214/009053605000000714Search in Google Scholar
[10] Blundell R, Chen X, Kristensen D. Semi-nonparametric IV estimation of shape-invariant Engel curves. Econometrica. 2007;75(6):1613–69. 10.1111/j.1468-0262.2007.00808.xSearch in Google Scholar
[11] Horowitz JL. Applied nonparametric instrumental variables estimation. Econometrica. 2011;79(2):347–94. 10.3982/ECTA8662Search in Google Scholar
[12] Hartford J, Lewis G, Leyton-Brown K, Taddy M. Deep IV: a flexible approach for counterfactual prediction. In: Proceedings of the 34th ICML. vol. 70. PMLR; 2017. p. 1414–23. Search in Google Scholar
[13] Lewis G, Syrgkanis V. Adversarial generalized method of moments; ArXiv Preprint: 1803.07164; 2018. Search in Google Scholar
[14] Bennett A, Kallus N, Schnabel T. Deep generalized method of moments for instrumental variable analysis. In: NeurIPS 32. Curran Associates, Inc., 2019. p. 3564–74. Search in Google Scholar
[15] Muandet K, Mehrjou A, Lee SK, Raj A. Dual instrumental variable regression. In: NeurIPS 33. Curran Associates, Inc.; 2020. Search in Google Scholar
[16] Singh R, Sahani M, Gretton A. Kernel instrumental variable regression. In: NeurIPS 32. Curran Associates, Inc.; 2019. p. 4593–605. Search in Google Scholar
[17] Liao L, Chen Y, Yang Z, Dai B, Wang Z, Kolar M. Provably efficient neural estimation of structural equation model: an adversarial approach. In: NeurIPS 33. Curran Associates, Inc.; 2020. Search in Google Scholar
[18] Bennett A, Kallus N. The variational method of moments; ArXiv Preprint: 2012.09422; 2020. Search in Google Scholar
[19] Daskalakis C, Panageas I. The limit points of (optimistic) gradient descent in min-max optimization. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems; Curran Associates, Inc. 2018. p. 9256–66. Search in Google Scholar
[20] Bailey JP, Gidel G, Piliouras G. Finite regret and cycles with fixed step-size via alternating gradient descent-ascent. In: Conference on Learning Theory. PMLR; 2020. p. 391–407. Search in Google Scholar
[21] Mastouri A, Zhu Y, Gultchin L, Korba A, Silva R, Kusner MJ, et al. Proximal causal learning with kernels: two-stage estimation and moment restriction. In: Proceedings of the 38th International Conference on Machine Learning. vol. 139 of Proceedings of Machine Learning Research. PMLR; 2021. p. 7512–23. Search in Google Scholar
[22] Horowitz JL, Lee S. Uniform confidence bands for functions estimated nonparametrically with instrumental variables. J Econom. 2012;168(2):175–88. 10.1016/j.jeconom.2011.12.001Search in Google Scholar
[23] Chen X, Christensen TM. Optimal uniform convergence rates and asymptotic normality for series estimators under weak dependence and weak conditions. J Econom. 2015;188(2):447–65. 10.1016/j.jeconom.2015.03.010Search in Google Scholar
[24] Chen X, Christensen TM. Optimal sup-norm rates and uniform inference on nonlinear functionals of nonparametric IV regression. Quant Econom. 2018;9(1):39–84. Search in Google Scholar
[25] Babii A. Honest confidence sets in nonparametric IV regression and other ill-posed models. Econometric Theory. 2020;36(4):658–706. 10.1017/S0266466619000380Search in Google Scholar
[26] Chen X, Christensen T, Kankanala S. Adaptive estimation and uniform confidence bands for nonparametric iv. 2021. arXiv: http://arXiv.org/abs/arXiv:210711869. Search in Google Scholar
[27] Muandet K, Jitkrittum W, Kübler J. Kernel conditional moment test via maximum moment restriction. In: Proceedings of the 36th Conference on UAI. vol. 124 of Proceedings of Machine Learning Research. PMLR; 2020. p. 41–50. Search in Google Scholar
[28] Serfling R. Approximation theorems of mathematical statistics. New York, NY: John Wiley & Sons; 1980. 10.1002/9780470316481Search in Google Scholar
[29] Hable R. Asymptotic normality of support vector machine variants and other regularized kernel methods. J Multivariate Anal. 2012;106:92–117. 10.1016/j.jmva.2011.11.004Search in Google Scholar
[30] Newey W. Efficient estimation of models with conditional moment restrictions. In: Handbook of statistics. vol. 11. Elsevier; 1993. p. 419–54. 10.1016/S0169-7161(05)80051-3Search in Google Scholar
[31] Donald SG, Newey WK. Choosing the number of instruments. Econometrica. 2001;69(5):1161–91. 10.1111/1468-0262.00238Search in Google Scholar
[32] Hall AR. Generalized method of moments. Advanced texts in econometrics. Oxford: Oxford University Press; 2005. Search in Google Scholar
[33] Aronszajn N. Theory of reproducing kernels. Trans Am Math Soc. 1950;68(3):337–404. 10.1090/S0002-9947-1950-0051437-7Search in Google Scholar
[34] Schölkopf B, Smola A. Learning with Kernels: support vector machines, regularization, optimization, and beyond. Cambridge, MA, USA: MIT Press; 2002. 10.7551/mitpress/4175.001.0001Search in Google Scholar
[35] Berlinet A, Thomas-Agnan C. Reproducing Kernel Hilbert spaces in probability and statistics. Amsterdam: Kluwer Academic Publishers; 2004. 10.1007/978-1-4419-9096-9Search in Google Scholar
[36] Muandet K, Fukumizu K, Sriperumbudur B, Schölkopf B. Kernel mean embedding of distributions: a review and beyond. Foundations Trends Machine Learn. 2017;10(1–2):1–141. 10.1561/2200000060Search in Google Scholar
[37] Steinwart I, Christmann A. Support vector machines. New York, NY: Springer Science & Business Media; 2008. Search in Google Scholar
[38] Steinwart I. On the influence of the kernel on the consistency of support vector machines. JMLR 2002 Mar;2:67–93. Search in Google Scholar
[39] Fukumizu K, Bach FR, Jordan MI. Dimensionality reduction for supervised learning with reproducing kernel Hilbert spaces. JMLR. 2004 Dec;5:73–99. 10.21236/ADA446572Search in Google Scholar
[40] Sriperumbudur BK, Fukumizu K, Lanckriet GRG. Universality, characteristic kernels and RKHS embedding of measures. JMLR. 2011 Jul;12:2389–410. Search in Google Scholar
[41] Simon-Gabriel CJ, Schölkopf B. Kernel distribution embeddings: universal kernels, characteristic kernels and kernel metrics on distributions. JMLR. 2018;19(44):1–29. Search in Google Scholar
[42] D’Haultfoeuille X. On the completeness condition in nonparametric instrumental problems. Econom Theory. 2011;27(3):460–71. 10.1017/S0266466610000368Search in Google Scholar
[43] Vapnik VN. Statistical learning theory. Hoboken, NJ, USA: Wiley-Interscience; 1998. Search in Google Scholar
[44] Kariya T, Kurata H. Generalized least squares. Hoboken, NJ, USA: John Wiley & Sons; 2004. 10.1002/0470866993Search in Google Scholar
[45] Van der Vaart A. Asymptotic statistics. Cambridge, UK: Cambridge University Press; 2000. Search in Google Scholar
[46] Carrasco M, Florens JP, Renault E. Linear inverse problems in structural econometrics estimation based on spectral decomposition and regularization. In: Heckman JJ, Leamer EE, editors. Handbook of econometrics. vol. 6B. 1st ed. Elsevier; 2007. 10.1016/S1573-4412(07)06077-1Search in Google Scholar
[47] Schölkopf B, Herbrich R, Smola AJ. A generalized representer theorem. In: International Conference on Computational Learning Theory. Springer; 2001. p. 416–26. Search in Google Scholar
[48] Williams CKI, Seeger M. Using the Nyström method to speed up kernel machines. In: NeurIPS 13. Cambridge, MA: MIT Press; 2001. p. 682–8. Search in Google Scholar
[49] Flannery BP, Press WH, Teukolsky SA, Vetterling W. Numerical recipes in C. vol. 24. New York: Press Syndicate of the University of Cambridge; 1992. p. 78. Search in Google Scholar
[50] Dikkala N, Lewis G, Mackey L, Syrgkanis V. Minimax estimation of conditional moment models. CoRR. 2020;abs/2006.07201. Search in Google Scholar
[51] Vehtari A, Mononen T, Tolvanen V, Sivula T, Winther O. Bayesian leave-one-out cross-validation approximations for Gaussian latent variable models. JMLR. 2016;17(1):3581–618. Search in Google Scholar
[52] Greene WH. Econometric analysis. Delhi: Pearson Education India; 2003. Search in Google Scholar
[53] Newey WK, McFadden D. Chapter 36 Large sample estimation and hypothesis testing. In: Handbook of econometrics. vol. 4. Elsevier; 1994. p. 2111–245. 10.1016/S1573-4412(05)80005-4Search in Google Scholar
[54] Wainwright MJ. High-dimensional statistics: a non-asymptotic viewpoint. vol. 48. Cambridge, UK: Cambridge University Press; 2019. 10.1017/9781108627771Search in Google Scholar
[55] Van der Vaart A, Wellner J. Weak convergence and empirical processes. New York: Springer; 1996. 10.1007/978-1-4757-2545-2Search in Google Scholar
[56] Zhou DX. The covering number in learning theory. J Complexity. 2002;18(3):739–67. 10.1006/jcom.2002.0635Search in Google Scholar
[57] Steinwart I, Hush DR, Scovel C. Optimal rates for regularized least squares regression. COLT; 2009. p. 79–93. Search in Google Scholar
[58] Caponnetto A, De Vito E. Optimal rates for the regularized least-squares algorithm. Foundation Comput Math. 2007;7(3):331–68. 10.1007/s10208-006-0196-8Search in Google Scholar
[59] Chen X, Pouzo D. Estimation of nonparametric conditional moment models with possibly nonsmooth generalized residuals. Econometrica. 2012;80(1):277–321. 10.3982/ECTA7888Search in Google Scholar
[60] Chen X, Christensen TM. Optimal sup-norm rates and uniform inference on nonlinear functionals of nonparametric IV regression. Quant Econ. 2018;9(1):39–84. 10.3982/QE722Search in Google Scholar
[61] Darolles S, Fan Y, Florens JP, Renault E. Nonparametric instrumental regression. Econometrica. 2011;79(5):1541–65. 10.3982/ECTA6539Search in Google Scholar
[62] Song L, Huang J, Smola A, Fukumizu K. Hilbert space embeddings of conditional distributions with applications to dynamical systems. In: Proceedings of the 26th ICML (ICML); 2009. p. 961–8. 10.1145/1553374.1553497Search in Google Scholar
[63] Song L, Fukumizu K, Gretton A. Kernel embeddings of conditional distributions: a unified kernel framework for nonparametric inference in graphical models. IEEE Signal Processing Magazine. 2013;30(4):98–111. 10.1109/MSP.2013.2252713Search in Google Scholar
[64] Schölkopf B, Herbrich R, Smola AJ. A generalized representer theorem. In: Helmbold D, Williamson B, editors. Computational learning theory. Berlin, Heidelberg: Springer Berlin Heidelberg; 2001. p. 416–26. 10.1007/3-540-44581-1_27Search in Google Scholar
[65] Ivanov VK, Vasin VV, Tanana VP. Theory of linear ill-posed problems and its applications. Inverse and ill-posed problems series. Boston: De Gruyter; 2002. 10.1515/9783110944822Search in Google Scholar
[66] LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE. 1998;86(11):2278–324. 10.1109/5.726791Search in Google Scholar
[67] Hartwig FP, Davey Smith G, Bowden J. Robust inference in summary data Mendelian randomization via the zero modal pleiotropy assumption. Int J Eepidemiol. 2017;46(6):1985–98. 10.1093/ije/dyx102Search in Google Scholar PubMed PubMed Central
[68] Kuang Z, Sala F, Sohoni N, Wu S, Córdova-Palomera A, Dunnmon J, et al. Ivy: instrumental variable synthesis for causal inference. In: Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics. vol. 108 of Proceedings of Machine Learning Research. PMLR; 2020. p. 398–410. Search in Google Scholar
[69] Hartford JS, Veitch V, Sridhar D, Leyton-Brown K. Valid causal inference with (some) invalid instruments. CoRR. 2020;abs/2006.11386. Search in Google Scholar
[70] Sjolander A, Martinussen T. Instrumental variable estimation with the R package ivtools. Epidemiol Meth. 2019;8(1):20180024. 10.1515/em-2018-0024Search in Google Scholar
[71] Meehan M, Penckofer S. The role of vitamin D in the aging adult. J Aging Gerontol. 2014;2(2):60. 10.12974/2309-6128.2014.02.02.1Search in Google Scholar PubMed PubMed Central
[72] Zhang R, Muandet K, Schölkopf B, Imaizumi M. Instrument space selection for kernel maximum moment restriction. 2021. arXiv: http://arXiv.org/abs/arXiv:210603340. Search in Google Scholar
[73] Bennett A, Kallus N. The variational method of moments. 2020. arXiv: http://arXiv.org/abs/arXiv:201209422. Search in Google Scholar
[74] Hoeffding W. Probability inequalities for sums of bounded random variables. J Am Stat Assoc. 1963;58(301):13–30. 10.1080/01621459.1963.10500830Search in Google Scholar
[75] Mann HB, Wald A. On stochastic limit and order relationships. Ann Math Stat. 1943 Sep;14 (3):217–26. 10.1214/aoms/1177731415Search in Google Scholar
[76] Arcones MA, Gine E. Limit theorems for U-processes. Ann Probab. 1993;21(3):1494–542. 10.1214/aop/1176989128Search in Google Scholar
[77] Luenberger DG. Optimization by vector space methods. Hoboken, NJ: John Wiley & Sons; 1997. Search in Google Scholar
[78] Akritas MG. Empirical processes associated with V-statistics and a class of estimators under random censoring. The Annals of Statistics. 1986;14(2):619–37. 10.1214/aos/1176349942Search in Google Scholar
[79] Feng Y, Li L, Liu Q. A kernel loss for solving the bellman equation. NeurIPS. Curran Associates, Inc. 2019;32. Search in Google Scholar
[80] Liu Q, Li L, Tang Z, Zhou D. Breaking the curse of horizon: infinite-horizon off-policy estimation. In: NeurIPS. vol. 31. Curran Associates, Inc.; 2018. Search in Google Scholar
[81] Uehara M, Huang J, Jiang N. Minimax weight and q-function learning for off-policy evaluation. In: ICML. PMLR; 2020. p. 9659–68. Search in Google Scholar
[82] Kallus N. Balanced policy evaluation and learning. NeurIPS. Curran Associates, Inc. 2018;31. Search in Google Scholar
[83] Kallus N. Generalized optimal matching methods for causal inference. JMLR. 2020;21(62):1–54. Search in Google Scholar
[84] Wong RK, Chan KCG. Kernel-based covariate functional balancing for observational studies. Biometrika. 2018;105(1):199–213. 10.1093/biomet/asx069Search in Google Scholar PubMed PubMed Central
© 2023 the author(s), published by De Gruyter
This work is licensed under the Creative Commons Attribution 4.0 International License.
Articles in the same Issue
- Research Articles
- Adaptive normalization for IPW estimation
- Matched design for marginal causal effect on restricted mean survival time in observational studies
- Robust inference for matching under rolling enrollment
- Attributable fraction and related measures: Conceptual relations in the counterfactual framework
- Causality and independence in perfectly adapted dynamical systems
- Sensitivity analysis for causal decomposition analysis: Assessing robustness toward omitted variable bias
- Instrumental variable regression via kernel maximum moment loss
- Randomization-based, Bayesian inference of causal effects
- On the pitfalls of Gaussian likelihood scoring for causal discovery
- Double machine learning and automated confounder selection: A cautionary tale
- Randomized graph cluster randomization
- Efficient and flexible mediation analysis with time-varying mediators, treatments, and confounders
- Minimally capturing heterogeneous complier effect of endogenous treatment for any outcome variable
- Quantitative probing: Validating causal models with quantitative domain knowledge
- On the dimensional indeterminacy of one-wave factor analysis under causal effects
- Heterogeneous interventional effects with multiple mediators: Semiparametric and nonparametric approaches
- Exploiting neighborhood interference with low-order interactions under unit randomized design
- Robust variance estimation and inference for causal effect estimation
- Bounding the probabilities of benefit and harm through sensitivity parameters and proxies
- Potential outcome and decision theoretic foundations for statistical causality
- 2D score-based estimation of heterogeneous treatment effects
- Identification of in-sample positivity violations using regression trees: The PoRT algorithm
- Model-based regression adjustment with model-free covariates for network interference
- All models are wrong, but which are useful? Comparing parametric and nonparametric estimation of causal effects in finite samples
- Confidence in causal inference under structure uncertainty in linear causal models with equal variances
- Special Issue on Integration of observational studies with randomized trials - Part II
- Personalized decision making – A conceptual introduction
- Precise unbiased estimation in randomized experiments using auxiliary observational data
- Conditional average treatment effect estimation with marginally constrained models
- Testing for treatment effect twice using internal and external controls in clinical trials
Articles in the same Issue
- Research Articles
- Adaptive normalization for IPW estimation
- Matched design for marginal causal effect on restricted mean survival time in observational studies
- Robust inference for matching under rolling enrollment
- Attributable fraction and related measures: Conceptual relations in the counterfactual framework
- Causality and independence in perfectly adapted dynamical systems
- Sensitivity analysis for causal decomposition analysis: Assessing robustness toward omitted variable bias
- Instrumental variable regression via kernel maximum moment loss
- Randomization-based, Bayesian inference of causal effects
- On the pitfalls of Gaussian likelihood scoring for causal discovery
- Double machine learning and automated confounder selection: A cautionary tale
- Randomized graph cluster randomization
- Efficient and flexible mediation analysis with time-varying mediators, treatments, and confounders
- Minimally capturing heterogeneous complier effect of endogenous treatment for any outcome variable
- Quantitative probing: Validating causal models with quantitative domain knowledge
- On the dimensional indeterminacy of one-wave factor analysis under causal effects
- Heterogeneous interventional effects with multiple mediators: Semiparametric and nonparametric approaches
- Exploiting neighborhood interference with low-order interactions under unit randomized design
- Robust variance estimation and inference for causal effect estimation
- Bounding the probabilities of benefit and harm through sensitivity parameters and proxies
- Potential outcome and decision theoretic foundations for statistical causality
- 2D score-based estimation of heterogeneous treatment effects
- Identification of in-sample positivity violations using regression trees: The PoRT algorithm
- Model-based regression adjustment with model-free covariates for network interference
- All models are wrong, but which are useful? Comparing parametric and nonparametric estimation of causal effects in finite samples
- Confidence in causal inference under structure uncertainty in linear causal models with equal variances
- Special Issue on Integration of observational studies with randomized trials - Part II
- Personalized decision making – A conceptual introduction
- Precise unbiased estimation in randomized experiments using auxiliary observational data
- Conditional average treatment effect estimation with marginally constrained models
- Testing for treatment effect twice using internal and external controls in clinical trials
