Martin Burger Institut für Numerische und Angewandte Mathematik European Institute for Molecular Imaging CeNoS Total Variation and related Methods Numerical Schemes

Martin Burger Total Variation 2 Cetraro, September 2008 Numerical schemes Usual starting point are variational formulation or optimality condition Which formulation to be used: - Primal ? - Dual ? - Primal-Dual ? We aim to give a unified perspective on all such methods

Martin Burger Total Variation 3 Cetraro, September 2008 General setup Consider Variational Problem

Martin Burger Total Variation 4 Cetraro, September 2008 General Setup Particular form of K

Martin Burger Total Variation 5 Cetraro, September 2008 General Setup Introduce „gradient“ variable and constraint

Martin Burger Total Variation 6 Cetraro, September 2008 General Setup Minimization problem

Martin Burger Total Variation 7 Cetraro, September 2008 General Setup Optimality

Martin Burger Total Variation 8 Cetraro, September 2008 General Setup Form of F

Martin Burger Total Variation 9 Cetraro, September 2008 Linear Analogue F quadratic

Martin Burger Total Variation 10 Cetraro, September 2008 Iterative Schemes for Solution of TV Problems Mainly three classes of iterative schemes: 1.Fixed point methods 2.Thresholding methods 3.Newton Methods

Martin Burger Total Variation 11 Cetraro, September Fixed point methods Solve first and third equation exactly in each step (possibly with preconditioning for A*A ) Do fixed-point iteration for w instead of second equation

Martin Burger Total Variation 12 Cetraro, September Fixed point methods Rewrite subgradient relation in some form Eliminating v

Martin Burger Total Variation 13 Cetraro, September Fixed point methods Matrix form

Martin Burger Total Variation 14 Cetraro, September Thesholding methods Solve first equation exactly in each step (possibly with preconditioning for A*A ) Do fixed-point iteration for v instead of second equation Possibly add a damping term in w for the last equation

Martin Burger Total Variation 15 Cetraro, September Thesholding methods C is damping matrix, possible perturbation T is thresholding operator

Martin Burger Total Variation 16 Cetraro, September Newton type methods Approximate F in a reasonably smooth way and perform (inexact) Newton iteration Linearized coupled system solved in each iteration step

Martin Burger Total Variation 17 Cetraro, September Newton type methods For consistency (superlinear convergence)

Martin Burger Total Variation 18 Cetraro, September Newton type methods Matrix form

Martin Burger Total Variation 19 Cetraro, September 2008 Singular case Our case has the same structure except the nonlinearity and multi-valuedness in the second equation Several numerical approaches and distinctions can be understood by approximating the pointwise relation

Martin Burger Total Variation 20 Cetraro, September 2008 Primal Approximation Simple approach: approximate F by smooth function Equation q equals derivative of this smooth function Example for Euclidean norm:

Martin Burger Total Variation 21 Cetraro, September 2008 Primal Approximation General approach to obtain a differentiable approximation with Lipschitz gradient is Moreau-Yosida regularization Example for Euclidean norm: Huber norm

Martin Burger Total Variation 22 Cetraro, September 2008 Fixed point form Alternative without approximation

Martin Burger Total Variation 23 Cetraro, September 2008 Fixed point form Leads to shrinkage

Martin Burger Total Variation 24 Cetraro, September 2008 Fixed point form Equivalent fixed point relation

Martin Burger Total Variation 25 Cetraro, September 2008 SOCP / LP formulations Roughly introduce new variable f and inequality Subgradients characterized by minimization of Note: we want to minimize f + something, hence optimal solution will always satisfies

Martin Burger Total Variation 26 Cetraro, September 2008 SOCP / LP formulations Example: usual total variation, F equals Euclidean norm Constraint is a quadratic (second-order cone) condition

Martin Burger Total Variation 27 Cetraro, September 2008 SOCP / LP formulations Example: anisotropic TV Analogous introduction, now many f yields even LP

Martin Burger Total Variation 28 Cetraro, September 2008 SOCP / LP formulations: Interior Points Interior point methods further approximate the constrained problem via an unconstrained problem with additional barrier term ( - log of the constraint) enforcing that the solution is in the interior of the constraint set This can be rewritten as using a smoothed version of F

Martin Burger Total Variation 29 Cetraro, September 2008 Dual Approaches Alternative interpretation of subgradient relation

Martin Burger Total Variation 30 Cetraro, September 2008 Dual approaches Primal and bidual are the same under suitable conditions

Martin Burger Total Variation 31 Cetraro, September 2008 Dual Approximation Penalty Methods

Martin Burger Total Variation 32 Cetraro, September 2008 Dual Approximation Barrier methods

Martin Burger Total Variation 33 Cetraro, September 2008 Dual Fixed Point Construct fixed-point form

Martin Burger Total Variation 34 Cetraro, September 2008 Dual Fixed Point Adding constants we see that q minimizes

Martin Burger Total Variation 35 Cetraro, September 2008 Dual Fixed Point for Primal Relation Consider primal relation in special case

Martin Burger Total Variation 36 Cetraro, September 2008 Dual Fixed Point for Primal Relation Fixed point equation