Home Penalty-based smoothness conditions in convex variational regularization
Article
Licensed
Unlicensed Requires Authentication

Penalty-based smoothness conditions in convex variational regularization

  • Bernd Hofmann ORCID logo EMAIL logo , Stefan Kindermann and Peter Mathé
Published/Copyright: March 8, 2019
Journal of Inverse and Ill-posed Problems
From the journal Journal of Inverse and Ill-posed Problems Volume 27 Issue 2

Abstract

The authors study Tikhonov regularization of linear ill-posed problems with a general convex penalty defined on a Banach space. It is well known that the error analysis requires smoothness assumptions. Here such assumptions are given in form of inequalities involving only the family of noise-free minimizers along the regularization parameter and the (unknown) penalty-minimizing solution. These inequalities control, respectively, the defect of the penalty, or likewise, the defect of the whole Tikhonov functional. The main results provide error bounds for a Bregman distance, which split into two summands: the first smoothness-dependent term does not depend on the noise level, whereas the second term includes the noise level. This resembles the situation of standard quadratic Tikhonov regularization in Hilbert spaces. It is shown that variational inequalities, as these were studied recently, imply the validity of the assumptions made here. Several examples highlight the results in specific applications.

Keywords: Variational regularization; convex penalty; error estimates; convergence rates; smoothness conditions; index functions; linear ill-posed problems
MSC 2010: 47A52; 65J20

Funding source: Austrian Science Fund

Award Identifier / Grant number: P 30157-N31

Funding source: Deutsche Forschungsgemeinschaft

Award Identifier / Grant number: HO 1454/12-1

Funding statement: The research of the first author was supported by Deutsche Forschungsgemeinschaft (DFG-grant HO 1454/12-1). The research of the second author was supported by the Austrian Science Fund (FWF) project P 30157-N31.

A Proofs

Let us define the noisy and exact residuals, and the noise term as

pαδ:=A⁢xαδ-yδ,pα:=A⁢xα-y=A⁢(xα-x†),Δ:=yδ-y.

Notice that all quantities pα, pαδ as well as Δ belong to the Hilbert space H. The subsequent analysis will be based on the optimality conditions (recall our convention on the choice of ξαδ∈∂⁡J⁢(xαδ) and ξα∈∂⁡J⁢(xα))

(A.1)〈A⁢xαδ-yδ,A⁢w〉+α⁢〈ξαδ,w〉=0 for all ⁢w∈X,
(A.2)〈A⁢xα-A⁢x†,A⁢w〉+α⁢〈ξα,w〉=0 for all ⁢w∈X.

In particular, the optimality conditions lead to the following formulas, by

  1. subtracting (A.1) from (A.2) using w=xα-x†,

  2. using (A.1) with w=xα-xαδ,

  3. using (A.2) with w=xα-x†,

respectively:

(A.3)〈ξα-ξαδ,x†-xα〉=-1α⁢〈pαδ-pα,pα〉,
(A.4)-〈ξαδ,xα-xαδ〉=1α〈pαδ,pα-pαδ-Δ)〉,
(A.5)-〈ξα,x†-xα〉=-1α⁢∥pα∥2.

The following bounds will be the key for proving Theorem 2.8.

Lemma A.1.

Under Assumption 2.3 we have

(A.6)Bξα⁢(xα;x†)≤Ψ⁢(α)-12⁢α⁢∥pα∥2,
Bξαδ⁢(xαδ;xα)≤δ22⁢α-12⁢α⁢∥pα∥2+1α⁢〈pαδ,pα〉.

Proof.

Using the optimality condition (A.5), we find that

Bξα⁢(xα;x†)+12⁢α⁢∥pα∥2=J⁢(x†)-J⁢(xα)-〈ξα,x†-xα〉+12⁢α⁢∥pα∥2
=J⁢(x†)-J⁢(xα)-12⁢α⁢∥pα∥2
=1α⁢(Tα⁢(x†;y)-Tα⁢(xα;y))≤Ψ⁢(α),

which proves the first assertion.

For proving the second assertion we use the definition of Bξαδ⁢(xαδ;xα) and (A.4) to find

(A.7)Bξαδ⁢(xαδ;xα)=J⁢(xα)-J⁢(xαδ)+1α⁢〈pαδ,pα-pαδ-Δ〉.

The minimizing property of xα also yields

J⁢(xα)-J⁢(xαδ)≤12⁢α⁢[∥A⁢xαδ-y∥2-∥A⁢xα-y∥2]=12⁢α⁢[∥pαδ+Δ∥2-∥pα∥2].

We rewrite

〈pαδ,pα-pαδ-Δ〉=-∥pαδ∥2+〈pαδ,pα-Δ〉.

Using this and plugging the above estimate into (A.7) gives

Bξαδ⁢(xαδ;xα)≤12⁢α⁢[∥pαδ+Δ∥2-∥pα∥2]-1α⁢∥pαδ∥2+1α⁢〈pαδ,pα-Δ〉
=12⁢α⁢∥Δ∥2-12⁢α⁢∥pα∥2-12⁢α⁢∥pαδ∥2+1α⁢〈pαδ,pα〉
≤δ22⁢α-12⁢α⁢∥pα∥2+1α⁢〈pαδ,pα〉,

completing the proof of the second assertion and of the lemma. ∎

We are now in a position to give detailed proofs of the results in Section 2.

Proof of Proposition 2.4.

From (A.6) we know that, for all α>0,

∥A⁢xα-y∥22⁢α=∥pα∥22⁢α≤Ψ⁢(α).

Therefore, Assumption 2.3 implies that

J⁢(x†)-J⁢(xα)=J⁢(x†)-J⁢(xα)-∥A⁢xα-y∥22⁢α+∥pα∥22⁢α≤Ψ⁢(α)+∥pα∥22⁢α≤2⁢Ψ⁢(α),

which completes the proof. ∎

Proof of Proposition 2.6.

First, if infx∈X⁡J⁢(x)=J⁢(x†), then

α⁢J⁢(x†)=Tα⁢(x†;A⁢x†)≥Tα⁢(xα;A⁢x†)≥α⁢J⁢(xα)≥α⁢J⁢(x†),

and for all α>0 we have J⁢(xα)=J⁢(x†). This allows us to prove the assertion in the first (singular) case. Otherwise, assume to the contrary that there is an index function Ψ, and for a decreasing sequence of regularization parameters (αk)k with limk→∞⁡αk=0 the limit condition

limk→∞⁡Ψ⁢(αk)αk=0

holds. Consequently, we find from (2.5) that limk→∞⁡1αk⁢∥A⁢xαk-y∥=0, with y=A⁢x†.

Due to the optimality condition (A.2) for xα, we have that

A*⁢(A⁢xαk-y)+αk⁢ξαk=0 for some ⁢ξαk∈∂⁡J⁢(xαk)⊂X*.

This yields

ξαk=-1αk⁢A*⁢(A⁢xαk-y).

Since A*:H→X* is a bounded linear operator, we get

∥ξαk∥X*≤1αk⁢∥A*∥ℒ⁢(H,X*)⁢∥A⁢xαk-y∥→0 as ⁢k→∞.

Since ξαk∈∂⁡J⁢(xαk) and after taking the limit, we find for all z∈X that

J⁢(z)≥lim supk→∞⁡{J⁢(xαk)+〈ξαk,z-xαk〉}=lim supk→∞⁡J⁢(xαk)=J⁢(x†),

where we used Assumption 2.2 and limk→∞⁡Ψ⁢(αk)=0. Thus, we conclude that infx∈X⁡J⁢(x)=J⁢(x†). This contradicts the assumption, and hence the function Ψ cannot decrease to zero super-linearly as α→0. ∎

Proof of Theorem 2.8.

Here we recall the three-point identity (see, e.g., [29]). For u,v,w∈X and ξ∈∂⁡J⁢(w), η∈∂⁡J⁢(v), we have that

Bξ⁢(w;u)=Bη⁢(v;u)+Bξ⁢(w;v)+〈η-ξ,u-v〉,

and this specifies with u:=x†, v:=xα, and w:=xαδ to

(A.8)Bξαδ⁢(xαδ;x†)=Bξα⁢(xα;x†)+Bξαδ⁢(xαδ;xα)+〈ξα-ξαδ,x†-xα〉.

Inserting (A.3) into (A.8) gives

Bξαδ⁢(xαδ;x†)=Bξα⁢(xα;x†)+Bξαδ⁢(xαδ;xα)-1α⁢〈pαδ-pα,pα〉.

An application of the bounds in Lemma A.1 provides us with the estimate

Bξαδ⁢(xαδ;x†)≤Ψ⁢(α)+δ22⁢α-1α⁢∥pα∥2+1α⁢〈pαδ,pα〉-1α⁢〈pαδ-pα,pα〉=Ψ⁢(α)+δ22⁢α,

and the proof is complete. ∎

B Some convex analysis for index functions

We shall provide some additional details for convex index functions. First, it is well known that for a convex index function f we have that 0<s≤t yields f⁢(s)/s≤f⁢(t)/t. Indeed, we let 0<θ:=st≤1 and obtain that

f⁢(s)=f⁢(θ⁢t+(1-θ)⁢0)≤θ⁢f⁢(t)+(1-θ)⁢f⁢(0)=st⁢f⁢(t),

which allows us to prove the assertion. This implies that the limit g:=limt→0⁡f⁢(t)/t≥0 exists. If g>0, then f is linear near zero, and this case is not interesting in this study. Otherwise, we assume that g=0. In this interesting (sub-linear) case the following result is relevant.

Lemma B.1.

Suppose that f is a convex index function. The following assertions are equivalent.

  1. The quotient f⁢(t)/t, t>0, is a strictly increasing index function.

  2. There is a strictly increasing index function φ , and the companion Θ⁢(t):=t⁢φ⁢(t), t>0, such that the representation f⁢(t)=Θ2⁢((φ2)-1⁢(t)), t>0, is valid.

Proof.

Clearly, if f has a representation as in (ii), then we find, letting φ2⁢(s)=t, that

f⁢(t)t=Θ2⁢(s)φ2⁢(s)=s↘0,

as s→0.

For the other implication we observe that by assumption we can (implicitly) define the strictly increasing index function φ by

φ⁢(f⁢(t)t):=t,t>0.

This yields that

f⁢(t)=t⁢(φ2)-1⁢(t)=Θ2⁢((φ2)-1⁢(t)),t>0,

which completes the proof. ∎

As an interesting consequence we mention the following result for the Fenchel conjugate function f∗ to the convex (index) function f.

Corollary B.2.

Suppose that f is a convex index function such that the quotient f⁢(t)/t, t>0, is a strictly increasing index function. Then the Fenchel conjugate function f∗ is an index function, and there is a strictly increasing index function φ such that

f∗⁢(t)t≤φ2⁢(t),t>0.

Proof.

First, by Lemma B.1 there is a strictly increasing index function φ such that f⁢(t)=Θ2⁢((φ2)-1⁢(t)), t>0. Now we use the “poor man’s Young Inequality” of the form

φ2⁢(x)⁢y≤φ2⁢(x)⁢x+φ2⁢(y)⁢y,x,y>0,

which is easily seen by considering the cases x<y and y≤x separately. This in turn, by letting s:=φ2⁢(x) and t:=y, implies

s⁢t≤Θ2⁢((φ2)-1⁢(s))+Θ2⁢(t),s,t>0.

For the Fenchel conjugate f∗ this yields

f∗⁢(t):=sups>0⁡{s⁢t-f⁢(s)}≤Θ2⁢(t).

From this bound we conclude that f∗ will be an index function for which the quotient f∗⁢(t)/t has the desired bound. ∎

Acknowledgements

We thank Peter Elbau (University of Vienna) and Jens Flemming (TU Chemnitz) for suggesting to us essential ingredients for the proofs of Propositions 2.6 and 3.2, respectively.

References

[1] V. Albani, P. Elbau, M. V. de Hoop and O. Scherzer, Optimal convergence rates results for linear inverse problems in Hilbert spaces, Numer. Funct. Anal. Optim. 37 (2016), no. 5, 521–540. 10.1080/01630563.2016.1144070Search in Google Scholar PubMed PubMed Central

[2] R. Andreev, P. Elbau, M. V. de Hoop, L. Qiu and O. Scherzer, Generalized convergence rates results for linear inverse problems in Hilbert spaces, Numer. Funct. Anal. Optim. 36 (2015), no. 5, 549–566. 10.1080/01630563.2015.1021422Search in Google Scholar

[3] S. W. Anzengruber, B. Hofmann and P. Mathé, Regularization properties of the sequential discrepancy principle for Tikhonov regularization in Banach spaces, Appl. Anal. 93 (2014), no. 7, 1382–1400. 10.1080/00036811.2013.833326Search in Google Scholar

[4] M. Burger, J. Flemming and B. Hofmann, Convergence rates in ℓ1-regularization if the sparsity assumption fails, Inverse Problems 29 (2013), no. 2, Article ID 025013. 10.1088/0266-5611/29/2/025013Search in Google Scholar

[5] M. Burger and S. Osher, Convergence rates of convex variational regularization, Inverse Problems 20 (2004), no. 5, 1411–1421. 10.1088/0266-5611/20/5/005Search in Google Scholar

[6] A. Chambolle, V. Caselles, D. Cremers, M. Novaga and T. Pock, An introduction to total variation for image analysis, Theoretical Foundations and Numerical Methods for Sparse Recovery, Radon Ser. Comput. Appl. Math. 9, Walter de Gruyter, Berlin (2010), 263–340. 10.1515/9783110226157.263Search in Google Scholar

[7] H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems, Math. Appl. 375, Kluwer Academic, Dordrecht, 1996. 10.1007/978-94-009-1740-8Search in Google Scholar

[8] J. Flemming, Generalized Tikhonov Regularization and Modern Convergence Rate Theory in Banach Spaces, Shaker, Aachen, 2012. Search in Google Scholar

[9] J. Flemming, A converse result for Banach space convergence rates in Tikhonov-type convex regularization of ill-posed linear equations, J. Inverse Ill-Posed Probl. 26 (2018), no. 5, 639–646. 10.1515/jiip-2017-0116Search in Google Scholar

[10] J. Flemming, Existence of variational source conditions for nonlinear inverse problems in Banach spaces, J. Inverse Ill-Posed Probl. 26 (2018), no. 2, 277–286. 10.1515/jiip-2017-0092Search in Google Scholar

[11] J. Flemming and D. Gerth, Injectivity and weak*-to-weak continuity suffice for convergence rates in ℓ1-regularization, J. Inverse Ill-Posed Probl. 26 (2018), no. 1, 85–94. 10.1515/jiip-2017-0008Search in Google Scholar

[12] J. Flemming and B. Hofmann, A new approach to source conditions in regularization with general residual term, Numer. Funct. Anal. Optim. 31 (2010), no. 1–3, 254–284. 10.1080/01630561003765721Search in Google Scholar

[13] J. Flemming, B. Hofmann and P. Mathé, Sharp converse results for the regularization error using distance functions, Inverse Problems 27 (2011), no. 2, Article ID 025006. 10.1088/0266-5611/27/2/025006Search in Google Scholar

[14] K. Frick, D. A. Lorenz and E. Resmerita, Morozov’s principle for the augmented Lagrangian method applied to linear inverse problems, Multiscale Model. Simul. 9 (2011), no. 4, 1528–1548. 10.1137/100812835Search in Google Scholar

[15] M. Grasmair, Generalized Bregman distances and convergence rates for non-convex regularization methods, Inverse Problems 26 (2010), no. 11, Article ID 115014. 10.1088/0266-5611/26/11/115014Search in Google Scholar

[16] B. Hofmann, B. Kaltenbacher, C. Pöschl and O. Scherzer, A convergence rates result for Tikhonov regularization in Banach spaces with non-smooth operators, Inverse Problems 23 (2007), no. 3, 987–1010. 10.1088/0266-5611/23/3/009Search in Google Scholar

[17] B. Hofmann and P. Mathé, Analysis of profile functions for general linear regularization methods, SIAM J. Numer. Anal. 45 (2007), no. 3, 1122–1141. 10.1137/060654530Search in Google Scholar

[18] B. Hofmann and P. Mathé, Parameter choice in Banach space regularization under variational inequalities, Inverse Problems 28 (2012), Article ID 104006. 10.1088/0266-5611/28/10/104006Search in Google Scholar

[19] T. Hohage and F. Weidling, Characterizations of variational source conditions, converse results, and maxisets of spectral regularization methods, SIAM J. Numer. Anal. 55 (2017), no. 2, 598–620. 10.1137/16M1067445Search in Google Scholar

[20] K. Ito and B. Jin, Inverse Problems. Tikhonov Theory and Algorithms, Ser. Appl. Math. 22, World Scientific, Hackensack 2015. 10.1142/9120Search in Google Scholar

[21] S. Kindermann, Convex Tikhonov regularization in Banach spaces: New results on convergence rates, J. Inverse Ill-Posed Probl. 24 (2016), no. 3, 341–350. 10.1515/jiip-2015-0038Search in Google Scholar

[22] P. Mathé and S. V. Pereverzev, Geometry of linear ill-posed problems in variable Hilbert scales, Inverse Problems 19 (2003), no. 3, 789–803. 10.1088/0266-5611/19/3/319Search in Google Scholar

[23] Y. Meyer, Oscillating Patterns in Image Processing and Nonlinear Evolution Equations, Univ. Lecture Ser. 22, American Mathematical Society, Providence, 2001. 10.1090/ulect/022Search in Google Scholar

[24] A. Neubauer, T. Hein, B. Hofmann, S. Kindermann and U. Tautenhahn, Improved and extended results for enhanced convergence rates of Tikhonov regularization in Banach spaces, Appl. Anal. 89 (2010), no. 11, 1729–1743. 10.1080/00036810903517597Search in Google Scholar

[25] E. Resmerita, Regularization of ill-posed problems in Banach spaces: Convergence rates, Inverse Problems 21 (2005), no. 4, 1303–1314. 10.1088/0266-5611/21/4/007Search in Google Scholar

[26] E. Resmerita and O. Scherzer, Error estimates for non-quadratic regularization and the relation to enhancement, Inverse Problems 22 (2006), no. 3, 801–814. 10.1088/0266-5611/22/3/004Search in Google Scholar

[27] L. I. Rudin, S. Osher and E. Fatemi, Nonlinear total variation based noise removal algorithms, Phys. D 60 (1992), no. 1–4, 259–268. 10.1016/0167-2789(92)90242-FSearch in Google Scholar

[28] O. Scherzer, M. Grasmair, H. Grossauer, M. Haltmeier and F. Lenzen, Variational Methods in Imaging, Appl. Math. Sci. 167, Springer, New York, 2009. Search in Google Scholar

[29] T. Schuster, B. Kaltenbacher, B. Hofmann and K. S. Kazimierski, Regularization Methods in Banach Spaces, Radon Ser. Comput. Appl. Math. 10, Walter de Gruyter, Berlin, 2012. 10.1515/9783110255720Search in Google Scholar

Received: 2018-05-03
Revised: 2019-01-31
Accepted: 2019-02-08
Published Online: 2019-03-08
Published in Print: 2019-04-01

© 2019 Walter de Gruyter GmbH, Berlin/Boston

Articles in the same Issue

  1. Frontmatter
  2. An inverse problem for the integro-differential Dirac system with partial information given on the convolution kernel
  3. Magnetoencephalography inverse problem in the spheroid geometry
  4. Space of images of the mixed Riesz hyperbolic B-potential and analytic continuation
  5. A gradient-based approach for identification of the zeroth-order coefficient 𝑝(𝑥) in the parabolic equation 𝑢𝑡 = (𝑘(𝑥)𝑢𝑥)𝑥 − 𝑝(𝑥)𝑢 from Dirichlet-type measured output
  6. Convergence of Levenberg–Marquardt method for the inverse problem with an interior measurement
  7. Inverse problem for one-dimensional wave equation with matrix potential
  8. Cauchy difference priors for edge-preserving Bayesian inversion
  9. On an inverse problem in photoacoustics
  10. An inverse dipole source problem in inhomogeneous media: Application to the EEG source localization in neonates
  11. Penalty-based smoothness conditions in convex variational regularization
https://doi.org/10.1515/jiip-2018-0039
Keywords for this article
Variational regularization; convex penalty; error estimates; convergence rates; smoothness conditions; index functions; linear ill-posed problems

Articles in the same Issue

  1. Frontmatter
  2. An inverse problem for the integro-differential Dirac system with partial information given on the convolution kernel
  3. Magnetoencephalography inverse problem in the spheroid geometry
  4. Space of images of the mixed Riesz hyperbolic B-potential and analytic continuation
  5. A gradient-based approach for identification of the zeroth-order coefficient 𝑝(𝑥) in the parabolic equation 𝑢𝑡 = (𝑘(𝑥)𝑢𝑥)𝑥 − 𝑝(𝑥)𝑢 from Dirichlet-type measured output
  6. Convergence of Levenberg–Marquardt method for the inverse problem with an interior measurement
  7. Inverse problem for one-dimensional wave equation with matrix potential
  8. Cauchy difference priors for edge-preserving Bayesian inversion
  9. On an inverse problem in photoacoustics
  10. An inverse dipole source problem in inhomogeneous media: Application to the EEG source localization in neonates
  11. Penalty-based smoothness conditions in convex variational regularization
Sign up now to receive a 20% welcome discount
Institutional Access
How does access work?
Have an idea on how to improve our website?
Please write us.
© 2025 De Gruyter Brill
Downloaded on 10.10.2025 from https://www.degruyterbrill.com/document/doi/10.1515/jiip-2018-0039/html
Scroll to top button