General regularization in covariate shift adaptation
-
Duc Hoan Nguyen
Abstract
Sample reweighting is one of the most widely used methods for correcting the error of least squares learning algorithms in reproducing kernel Hilbert spaces (RKHS), which is caused by future data distributions that are different from the training data distribution. In practical situations, the sample weights are determined by values of the estimated Radon-Nikodým derivative of the future data distribution with regard to the training data distribution. In this chapter, we review known error bounds for reweighted regression in RKHS and obtain, by combination, novel results. We show, under weak smoothness conditions, that the amount of samples needed to achieve the same order of accuracy, as in standard supervised learning without differences in data distributions, is smaller than proven by state-of-the-art analyses.
Abstract
Sample reweighting is one of the most widely used methods for correcting the error of least squares learning algorithms in reproducing kernel Hilbert spaces (RKHS), which is caused by future data distributions that are different from the training data distribution. In practical situations, the sample weights are determined by values of the estimated Radon-Nikodým derivative of the future data distribution with regard to the training data distribution. In this chapter, we review known error bounds for reweighted regression in RKHS and obtain, by combination, novel results. We show, under weak smoothness conditions, that the amount of samples needed to achieve the same order of accuracy, as in standard supervised learning without differences in data distributions, is smaller than proven by state-of-the-art analyses.
Chapters in this book
- Frontmatter I
- Preface V
- Contents VII
-
Part I: Mathematical aspects of data-driven methods in inverse problems
- On optimal regularization parameters via bilevel learning 1
- Learned regularization for inverse problems 39
- Inverse problems with learned forward operators 73
- Unsupervised approaches based on optimal transport and convex analysis for inverse problems in imaging 107
- Learned reconstruction methods for inverse problems: sample error estimates 163
- Statistical inverse learning problems with random observations 201
- General regularization in covariate shift adaptation 245
-
Part II: Applications of data-driven methods in inverse problems
- Analysis of generalized iteratively regularized Landweber iterations driven by data 273
- Integration of model- and learning-based methods in image restoration 303
- Dynamic computerized tomography using inexact models and motion estimation 331
- Deep Bayesian inversion 359
- Utilizing uncertainty quantification variational autoencoders in inverse problems with applications in photoacoustic tomography 413
- Electrical impedance tomography: a fair comparative study on deep learning and analytic-based approaches 437
- Classification with neural networks with quadratic decision functions 471
- Index 495
Chapters in this book
- Frontmatter I
- Preface V
- Contents VII
-
Part I: Mathematical aspects of data-driven methods in inverse problems
- On optimal regularization parameters via bilevel learning 1
- Learned regularization for inverse problems 39
- Inverse problems with learned forward operators 73
- Unsupervised approaches based on optimal transport and convex analysis for inverse problems in imaging 107
- Learned reconstruction methods for inverse problems: sample error estimates 163
- Statistical inverse learning problems with random observations 201
- General regularization in covariate shift adaptation 245
-
Part II: Applications of data-driven methods in inverse problems
- Analysis of generalized iteratively regularized Landweber iterations driven by data 273
- Integration of model- and learning-based methods in image restoration 303
- Dynamic computerized tomography using inexact models and motion estimation 331
- Deep Bayesian inversion 359
- Utilizing uncertainty quantification variational autoencoders in inverse problems with applications in photoacoustic tomography 413
- Electrical impedance tomography: a fair comparative study on deep learning and analytic-based approaches 437
- Classification with neural networks with quadratic decision functions 471
- Index 495
