Stability and Experimental Comparison of Prototypical Iterative Schemes for Total Variation Regularized Problems

Sören Bartels; Marijo Milicevic

doi:10.1515/cmam-2016-0014

Artikel

Stability and Experimental Comparison of Prototypical Iterative Schemes for Total Variation Regularized Problems

Sören Bartels und Marijo Milicevic

Veröffentlicht/Copyright: 13. April 2016

Veröffentlicht von

Veröffentlichen auch Sie bei De Gruyter Brill

Manuskript einreichen Informationen für Autor*innen Erkunden Sie dieses Fachgebiet

Aus der Zeitschrift Computational Methods in Applied Mathematics Band 16 Heft 3

Abstract

Various iterative methods are available for the approximate solution of non-smooth minimization problems. For a popular non-smooth minimization problem arising in image processing, we discuss the suitable application of three prototypical methods and their stability. The methods are compared experimentally with a focus on choice of stopping criteria, influence of rough initial data, step sizes as well as mesh sizes. An overview of existing algorithms is given.

Keywords: Total Variation; Finite Elements; Iterative Schemes; Stability

MSC 2010: 65K15; 49M29

Funding source: Deutsche Forschungsgemeinschaft

Award Identifier / Grant number: SPP 1748: BA 2268/2-1

Funding statement: The authors acknowledge financial support by the Deutsche Forschungsgemeinschaft for the project “Finite Element Approximation of Functions of Bounded Variation and Application to Model of Damage, Fracture and Plasticity” (BA 2268/2-1) via the priority program “Reliable Simulation Techniques in Solid Mechanics, Development of Non-Standard Discretization Methods, Mechanical and Mathematical Analysis” (SPP 1748), and by the Sino-German Science Center (grant id 1228) on the occasion of the Chinese-German Workshop on Computational and Applied Mathematics in Augsburg 2015.

A Roots of Quartic Equations

Having (3.2) in mind we consider the problem of finding the roots of a quartic equation

(A.1)x4+a⁢x3+b⁢x2+c⁢x+d=0

with a,b,c,d∈ℝ. A comprehensive discussion of this topic can be found, e.g., in [14]. With the substitution x=y-14⁢a we equivalently obtain

(A.2)y4+b~⁢y2+c~⁢y+d~=0

with b~=-38⁢a2+b, c~=18⁢a3-a⁢b2+c and d~=-3256⁢a4+a2⁢b16-a⁢c4+d. Adding 14⁢b~2 on both sides yields

(y2+12⁢b~)2=14⁢b~2-c~⁢y-d~.

Our goal is to produce a perfect square on the right-hand side. The following procedure is due to Lodovico Ferrari. We introduce a new variable z, which is to be specified later, within the square on the left-hand side and obtain

(y2+12⁢b~+12⁢z)2=14⁢b~2-c~⁢y-d~+14⁢z2+z⁢(y2+12⁢b~)

(A.3)=z⁢y2-c~⁢y+14⁢z2+12⁢b~⁢z+14⁢b~2-d~.

Now the right-hand side is a perfect square with respect to y if and only if the associated discriminant vanishes, i.e., if and only if

(A.4)0=4⁢z⁢(14⁢z2+12⁢b~⁢z+14⁢b~2-d~)-c~2

(A.5)=z3+2⁢b~⁢z2+(b~2-4⁢d~)⁢z-c~2.

The cubic polynomial in (A.5) is called cubic resolvent. We see that with z being any root of the cubic resolvent the term in (A.3) simplifies to a perfect square. Indeed, using (A.4), we have

(y2+12⁢b~+12⁢z)2=z⁢(y2-c~z⁢y+c~24⁢z2)=z⁢(y-c~2⁢z)2.

We can take z to be, for instance, a real root of (A.5) and obtain

y2+12⁢b~+12⁢z=±z⁢(y-c~2⁢z),

so we have two quadratic algebraic equations in y and can compute the roots of (A.2). We obtain the roots {xi}1≤i≤4 of our original equation with the relation xi=yi-14⁢a.

B Roots of Cubic Equations

In order to obtain the roots of (A.5) we briefly discuss how to compute the roots of a cubic algebraic equation. Given an equation of the form

(B.1)x3+a⁢x2+b⁢x+c=0

we set x=y-a3 and get

(B.2)y3+b~⁢y+c~=0

with b~=b-a23 and c~=c-a⁢b3+2⁢a327. With a further substitution y=u+v, with – for the time being – arbitrary u and v, we get

y3=(u+v)3=(u+v)⁢(u2+2⁢u⁢v+v2)=u3+v3+3⁢u⁢v⁢(u+v)=u3+v3+3⁢u⁢v⁢y

⟺y3-3⁢u⁢v⁢y-(u3+v3)=0.

Comparing coefficients yields

b~=-3⁢u⁢v,c~=-(u3+v3).

According to Vieta’s formulas, u3 and v3 are the roots of the quadratic algebraic equation

t2+c~⁢t-b~327=0.

Hence we have

u=-c~2+c~24+b~3273,v=-c~2-c~24+b~3273.

The cube roots u and v have to be chosen such that u⁢v=-b~3∈ℝ. Denoting by u1,v1 the cube roots defined by r3⁢ei⁢φ/3 for a complex number r⁢ei⁢φ, the two other cube roots are given by u2=u1⁢e2⁢π⁢i/3, u3=u1⁢e4⁢π⁢i/3 and v2=v1⁢e2⁢π⁢i/3, v3=v1⁢e4⁢π⁢i/3, respectively. Since the product uv has to be real, we get the three feasible pairs (u1,v1), (u2,v3) and (u3,v2). Hence, the three roots of (B.2) are given by

y1=u1+v1,y2=u2+v3,y3=u3+v2.

The roots {xi}1≤i≤3 of (B.1) are then given by xi=yi-a3.

References

[1] Acerbi E. and Fusco N., Semicontinuity problems in the calculus of variations, Arch. Ration. Mech. Anal. 86 (1984), no. 2, 125–145. 10.1007/BF00275731Suche in Google Scholar

[2] Ambrosio L. and Dal Maso G., On the relaxation in B⁢V⁢(Ω;ℝm) of quasi-convex integrals, J. Funct. Anal. 109 (1992), 76–97. 10.1016/0022-1236(92)90012-8Suche in Google Scholar

[3] Ambrosio L., Fusco N. and Pallara D., Functions of Bounded Variation and Free Discontinuity Problems, Oxford Math. Monogr., Clarendon Press, Oxford, 2000. 10.1093/oso/9780198502456.001.0001Suche in Google Scholar

[4] Ambrosio L., Mortola S. and Tortorelli V. M., Functionals with linear growth defined on vector valued BV functions, J. Math. Pures Appl. (9) 70 (1991), 269–323. Suche in Google Scholar

[5] Andreu-Vaillo F., Caselles V. and Mazón J. M., Parabolic Quasilinear Equations Minimizing Linear Growth Functionals, Progr. Math. 223, Birkhäuser, Basel, 2004. 10.1007/978-3-0348-7928-6Suche in Google Scholar

[6] Attouch H., Buttazzo G. and Michaille G., Variational Analysis in Sobolev and BV Spaces, MPS/SIAM Ser. Optim. 6, Mathematical Programming Society, Philadelphia, 2006. 10.1137/1.9780898718782Suche in Google Scholar

[7] Bartels S., Total variation minimization with finite elements: Convergence and iterative solution, SIAM J. Numer. Anal. 50 (2012), 1162–1180. 10.1137/11083277XSuche in Google Scholar

[8] Bartels S., Broken Sobolev space iteration for total variation regularized minimization problems, IMA J. Numer. Anal. (2015), 10.1093/imanum/drv023. 10.1093/imanum/drv023Suche in Google Scholar

[9] Bartels S., Numerical Methods for Nonlinear Partial Differential Equations, Springer, Heidelberg, 2015. 10.1007/978-3-319-13797-1Suche in Google Scholar

[10] Bartels S., Mielke A. and Roubiček T., Quasi-static small-strain plasticity in the limit of vanishing Hardening and its numerical approximation, SIAM J. Numer. Anal. 50 (2012), no. 2, 951–976. 10.1137/100819205Suche in Google Scholar

[11] Bartels S., Nochetto R. H. and Salgado A. J., Discrete total variation flows without regularization, SIAM J. Numer. Anal. 52 (2014), 363–385. 10.1137/120901544Suche in Google Scholar

[12] Beck A. and Teboulle M., A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci. 2 (2009), no. 1, 183–202. 10.1137/080716542Suche in Google Scholar

[13] Benamou J.-D. and Carlier G., Augmented Lagrangian methods for transport optimization, mean field games and degenerate elliptic equations, J. Optim. Theory Appl. 167 (2015), 1–26. 10.1007/s10957-015-0725-9Suche in Google Scholar

[14] Bewersdorff J., Algebra für Einsteiger, 5th ed., Springer, Heidelberg, 2013. 10.1007/978-3-658-02262-4Suche in Google Scholar

[15] Bildhauer M. and Fuchs M., Convex variational problems with linear growth, Geometric Analysis and Nonlinear Partial Differential Equations, Springer, Berlin (2003), 327–344. 10.1007/978-3-642-55627-2_18Suche in Google Scholar

[16] Bildhauer M. and Fuchs M., A variational approach to the denoising of images based on different variants of the TV-regularization, Appl. Math. Optim. 66 (2012), 331–361. 10.1007/s00245-012-9174-0Suche in Google Scholar

[17] Bregman L. M., The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex programming, USSR Comput. Math. Math. Phys. 7 (1967), 200–217. 10.1016/0041-5553(67)90040-7Suche in Google Scholar

[18] Chambolle A., An algorithm for total variation minimization and applications, J. Math. Imaging Vision 20 (2004), 89–97. 10.1023/B:JMIV.0000011321.19549.88Suche in Google Scholar

[19] Chambolle A. and Lions P.-L., Image recovery via total variation minimization and related problems, Numer. Math. 76 (1997), 167–188. 10.1007/s002110050258Suche in Google Scholar

[20] Chambolle A. and Pock T., A first-order primal-dual algorithm for convex problems with applications to imaging, J. Math. Imaging Vision 40 (2011), 120–145. 10.1007/s10851-010-0251-1Suche in Google Scholar

[21] Chambolle A. and Pock T., A remark on accelerated block coordinate descent for computing the proximity operators of a sum of convex functions, SIAM J. Comput. Math. 1 (2015), 29–54. 10.5802/smai-jcm.3Suche in Google Scholar

[22] Chan R. H. and Liang H.-X., Half-quadratic algorithm for ℓp-ℓq problems with applications to TV-ℓ1 image restoration and compressive sensing, Efficient Algorithms for Global Optimization Methods in Computer Vision, Lecture Notes in Comput. Sci. 8293, Springer, Berlin (2014), 78–103. 10.1007/978-3-642-54774-4_4Suche in Google Scholar

[23] Chan T. F., Golub G. H. and Mulet P., A nonlinear primal-dual method for total variation-based image restoration, SIAM J. Sci. Comput. 20 (1999), 1964–1977. 10.1007/3-540-76076-8_137Suche in Google Scholar

[24] Chan T. F. and Mulet P., On the convergence of the lagged diffusivity fixed point method in total variation image restoration, SIAM J. Numer. Anal. 36 (1999), 354–367. 10.1137/S0036142997327075Suche in Google Scholar

[25] Clason C. and Kunisch K., A duality-based approach to elliptic control problems in non-reflexive Banach spaces, ESAIM Control Optim. Calc. Var. 17 (2011), 243–266. 10.1051/cocv/2010003Suche in Google Scholar

[26] Conti S., Ginster J. and Rumpf M., A BV functional and its relaxation for joint motion estimation and image sequence recovery, ESAIM Math. Model. Numer. Anal. 49 (2015), no. 5, 1463–1487. 10.1051/m2an/2015036Suche in Google Scholar

[27] Dal Maso G., DeSimone A. and Mora M. G., Quasistatic evolution problems for linearly elastic-perfectly plastic materials, Arch. Ration. Mech. Anal. 180 (2006), 237–291. 10.1007/s00205-005-0407-0Suche in Google Scholar

[28] Darbon J. and Sigelle M., A fast and exact algorithm for total variation minimization, Pattern Recognition and Image Analysis, Lecture Notes in Comput. Sci., Springer, Berlin (2005), 351–359. 10.1007/11492429_43Suche in Google Scholar

[29] Dobson D. C. and Vogel C. R., Convergence of an iterative method for total variation denoising, SIAM J. Numer. Anal. 34 (1997), no. 5, 1779–1791. 10.1137/S003614299528701XSuche in Google Scholar

[30] Eckstein J. and Bertsekas D. P., On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators, Math. Program. 55 (1992), 293–318. 10.1007/BF01581204Suche in Google Scholar

[31] Ekeland I. and Témam R., Convex Analysis and Variational Problems, Classics Appl. Math. 28, Society for Industrial and Applied Mathematics, Philadelphia, 1999. 10.1137/1.9781611971088Suche in Google Scholar

[32] Elliott C. M. and Smitheman S. A., Numerical analysis of the tv regularization and H-1 fidelity model for decomposing an image into cartoon plus texture, IMA J. Numer. Anal. 29 (2009), 651–689. 10.1093/imanum/drn025Suche in Google Scholar

[33] Feng X. and Prohl A., Analysis of total variation flow and its finite element approximations, ESAIM Math. Model. Numer. Anal. 37 (2003), 533–556. 10.1051/m2an:2003041Suche in Google Scholar

[34] Fortin M. and Glowinski R., Augmented Lagrangian Methods, Stud. Math. Appl. 15, North-Holland, Amsterdam, 1983. Suche in Google Scholar

[35] Glowinski R. and Le Tallec P., Augmented Lagrangians and Operator-Splitting Methods in Nonlinear Mechanics, SIAM, Philadelphia, 1989. 10.1137/1.9781611970838Suche in Google Scholar

[36] Goldfarb D. and Yin W., Second-order cone programming methods for total variation-based image restoration, SIAM J. Sci. Comput. 27 (2005), 622–645. 10.1137/040608982Suche in Google Scholar

[37] Goldstein T., O’Donoghue B., Setzer S. and Baraniuk R., Fast alternating direction optimization methods, SIAM J. Imaging Sci. 7 (2014), 1588–1623. 10.1137/120896219Suche in Google Scholar

[38] Goldstein T. and Osher S., The split Bregman method for L1 regularized problems, SIAM J. Imaging Sci. 2 (2009), 323–343. 10.1137/080725891Suche in Google Scholar

[39] Güler O., Foundations of Optimization, Grad. Texts in Math. 258, Springer, New York, 2010. 10.1007/978-0-387-68407-9Suche in Google Scholar

[40] Hintermüller M., Ito K. and Kunisch K., The primal-dual active set strategy as a semismooth newton method, SIAM J. Optim. 13 (2003), 865–888. 10.1137/S1052623401383558Suche in Google Scholar

[41] Hintermüller M. and Kunisch K., Total bounded variation regularization as a bilaterally constrained optimization method, SIAM J. Appl. Math. 64 (2004), 1311–1333. 10.1137/S0036139903422784Suche in Google Scholar

[42] Nesterov Y., Smooth minimization of non-smooth functions, Math. Program. 103 (2005), 127–152. 10.1007/s10107-004-0552-5Suche in Google Scholar

[43] Papadakis N., Peyré G. and Oudet E., Optimal transport with proximal splitting, SIAM J. Imaging Sci. 7 (2014), 212–238. 10.1137/130920058Suche in Google Scholar

[44] Rockafellar R. T., Monotone operators and the proximal point algorithm, SIAM J. Control Optim. 14 (1976), 877–898. 10.1137/0314056Suche in Google Scholar

[45] Rudin L. I., Osher S. and Fatemi E., Nonlinear total variation based noise removal algorithms, Phys. D 60 (1992), 259–268. 10.1016/0167-2789(92)90242-FSuche in Google Scholar

[46] Thomas M., Quasistatic damage evolution with spatial BV-regularization, Discrete Contin. Dyn. Syst. Ser. S 6 (2013), 235–255. 10.3934/dcdss.2013.6.235Suche in Google Scholar

[47] Tseng P., Applications of a splitting algorithm to decomposition in convex programming and variational inequalities, SIAM J. Control Optim. 29 (1991), 119–138. 10.1137/0329006Suche in Google Scholar

[48] Vogel C. R. and Oman M. E., Iterative methods for total variation denoising, SIAM J. Sci. Comput. 17 (1996), 227–238. 10.1137/0917016Suche in Google Scholar

[49] Wang J. and Lucier B. J., Error bounds for finite-difference methods for Rudin–Osher–Fatemi image smoothing, SIAM J. Numer. Anal. 49 (2011), 845–868. 10.21236/ADA513262Suche in Google Scholar

[50] Wang Y., Yang J., Yin W. and Zhang Y., A new alternating minimization algorithm for total variation image reconstruction, SIAM J. Imaging Sci. 1 (2008), 248–272. 10.1137/080724265Suche in Google Scholar

[51] Wu C. and Tai X.-C., Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and higher order models, SIAM J. Imaging Sci. 3 (2010), 300–339. 10.1137/090767558Suche in Google Scholar

[52] Zhu M., Fast numerical algorithms for total variation based image restoration, Ph.D. thesis, University of California, Los Angeles, 2008. Suche in Google Scholar

Received: 2015-12-23

Revised: 2016-3-18

Accepted: 2016-3-20

Published Online: 2016-4-13

Published in Print: 2016-7-1

Sie haben derzeit keinen Zugang zu diesem Inhalt.

Artikel in diesem Heft

https://doi.org/10.1515/cmam-2016-0014

Schlagwörter für diesen Artikel

Total Variation; Finite Elements; Iterative Schemes; Stability