1. Martin Burger Institut für Numerische und Angewandte Mathematik European Institute for Molecular Imaging CeNoS Total Variation and related Methods Numerical Schemes

2. Martin Burger Total Variation 2 Cetraro, September 2008 Numerical schemes Usual starting point are variational formulation or optimality condition Which formulation to be used: - Primal ? - Dual ? - Primal-Dual ? We aim to give a unified perspective on all such methods

3. Martin Burger Total Variation 3 Cetraro, September 2008 General setup Consider Variational Problem

4. Martin Burger Total Variation 4 Cetraro, September 2008 General Setup Particular form of K

5. Martin Burger Total Variation 5 Cetraro, September 2008 General Setup Introduce „gradient“ variable and constraint

6. Martin Burger Total Variation 6 Cetraro, September 2008 General Setup Minimization problem

7. Martin Burger Total Variation 7 Cetraro, September 2008 General Setup Optimality

8. Martin Burger Total Variation 8 Cetraro, September 2008 General Setup Form of F

9. Martin Burger Total Variation 9 Cetraro, September 2008 Linear Analogue F quadratic

10. Martin Burger Total Variation 10 Cetraro, September 2008 Iterative Schemes for Solution of TV Problems Mainly three classes of iterative schemes: 1.Fixed point methods 2.Thresholding methods 3.Newton Methods

11. Martin Burger Total Variation 11 Cetraro, September 2008 1. Fixed point methods Solve first and third equation exactly in each step (possibly with preconditioning for A*A ) Do fixed-point iteration for w instead of second equation

12. Martin Burger Total Variation 12 Cetraro, September 2008 1. Fixed point methods Rewrite subgradient relation in some form Eliminating v

13. Martin Burger Total Variation 13 Cetraro, September 2008 1. Fixed point methods Matrix form

14. Martin Burger Total Variation 14 Cetraro, September 2008 2. Thesholding methods Solve first equation exactly in each step (possibly with preconditioning for A*A ) Do fixed-point iteration for v instead of second equation Possibly add a damping term in w for the last equation

15. Martin Burger Total Variation 15 Cetraro, September 2008 2. Thesholding methods C is damping matrix, possible perturbation T is thresholding operator

16. Martin Burger Total Variation 16 Cetraro, September 2008 2. Newton type methods Approximate F in a reasonably smooth way and perform (inexact) Newton iteration Linearized coupled system solved in each iteration step

17. Martin Burger Total Variation 17 Cetraro, September 2008 2. Newton type methods For consistency (superlinear convergence)

18. Martin Burger Total Variation 18 Cetraro, September 2008 2. Newton type methods Matrix form

19. Martin Burger Total Variation 19 Cetraro, September 2008 Singular case Our case has the same structure except the nonlinearity and multi-valuedness in the second equation Several numerical approaches and distinctions can be understood by approximating the pointwise relation

20. Martin Burger Total Variation 20 Cetraro, September 2008 Primal Approximation Simple approach: approximate F by smooth function Equation q equals derivative of this smooth function Example for Euclidean norm:

21. Martin Burger Total Variation 21 Cetraro, September 2008 Primal Approximation General approach to obtain a differentiable approximation with Lipschitz gradient is Moreau-Yosida regularization Example for Euclidean norm: Huber norm

22. Martin Burger Total Variation 22 Cetraro, September 2008 Fixed point form Alternative without approximation

23. Martin Burger Total Variation 23 Cetraro, September 2008 Fixed point form Leads to shrinkage

24. Martin Burger Total Variation 24 Cetraro, September 2008 Fixed point form Equivalent fixed point relation

25. Martin Burger Total Variation 25 Cetraro, September 2008 SOCP / LP formulations Roughly introduce new variable f and inequality Subgradients characterized by minimization of Note: we want to minimize f + something, hence optimal solution will always satisfies

26. Martin Burger Total Variation 26 Cetraro, September 2008 SOCP / LP formulations Example: usual total variation, F equals Euclidean norm Constraint is a quadratic (second-order cone) condition

27. Martin Burger Total Variation 27 Cetraro, September 2008 SOCP / LP formulations Example: anisotropic TV Analogous introduction, now many f yields even LP

28. Martin Burger Total Variation 28 Cetraro, September 2008 SOCP / LP formulations: Interior Points Interior point methods further approximate the constrained problem via an unconstrained problem with additional barrier term ( - log of the constraint) enforcing that the solution is in the interior of the constraint set This can be rewritten as using a smoothed version of F

29. Martin Burger Total Variation 29 Cetraro, September 2008 Dual Approaches Alternative interpretation of subgradient relation

30. Martin Burger Total Variation 30 Cetraro, September 2008 Dual approaches Primal and bidual are the same under suitable conditions

31. Martin Burger Total Variation 31 Cetraro, September 2008 Dual Approximation Penalty Methods

32. Martin Burger Total Variation 32 Cetraro, September 2008 Dual Approximation Barrier methods

33. Martin Burger Total Variation 33 Cetraro, September 2008 Dual Fixed Point Construct fixed-point form

34. Martin Burger Total Variation 34 Cetraro, September 2008 Dual Fixed Point Adding constants we see that q minimizes

35. Martin Burger Total Variation 35 Cetraro, September 2008 Dual Fixed Point for Primal Relation Consider primal relation in special case

36. Martin Burger Total Variation 36 Cetraro, September 2008 Dual Fixed Point for Primal Relation Fixed point equation