1. Martin Burger Total Variation 1 Cetraro, September 2008 Numerical Schemes Wrap up approximate formulations of subgradient relation

2. Martin Burger Total Variation 2 Cetraro, September 2008 Numerical Schemes Primal Approximation Primal Fixed Point Dual Approximation Dual Fixed Point Dual Fixed Point for Primal Relation

3. Martin Burger Total Variation 3 Cetraro, September 2008 1. Fixed point methods Matrix form

4. Martin Burger Total Variation 4 Cetraro, September 2008 Fixed Point Schemes I Primal Gradient Method Based on approximation of F: Fixed-point approach for first optimality equation

5. Martin Burger Total Variation 5 Cetraro, September 2008 Fixed Point Schemes I Primal Gradient Method Based on Approximation, Rudin-Osher- Fatemi 89 + easy to implement, efficient iteration steps + global convergence (descent method for variational problem) - dependent on approximation - slow convergence - severe step size restrictions (explicit approximation of differential operator)

6. Martin Burger Total Variation 6 Cetraro, September 2008 Fixed Point Schemes I Primal Gradient Method Based on Approximation, Rudin-Osher- Fatemi 89 Special case of fixed point methods with choice

7. Martin Burger Total Variation 7 Cetraro, September 2008 Fixed Point Schemes I Primal Gradient Method Based on Approximation, Rudin-Osher- Fatemi 89 + easy to implement, efficient iteration steps + global convergence (descent method for variational problem) - dependent on approximation - slow convergence - severe step size restrictions (explicit approximation of differential operator)

8. Martin Burger Total Variation 8 Cetraro, September 2008 Fixed Point Schemes II Dual Gradient Projection Method Dual methods eliminate also u

9. Martin Burger Total Variation 9 Cetraro, September 2008 Fixed Point Schemes II Dual Gradient Projection Method Do gradient step on the quadrativc functional and project back to constraint set M Note: can be interpreted as a scheme where the first equation is always satisfied, i.e. special case with

10. Martin Burger Total Variation 10 Cetraro, September 2008 Fixed Point Schemes II Dual Gradient Projection Method, Chambolle 05, Chan et al 08, Aujol 08 + easy to implement, efficient iteration steps + global convergence (descent method for dual problem) + no approximation necessary - slow convergence - needs inversion of A*A, hence good for ROF, bad for inverse problems Obvious generalization for last point: use preconditioning of A*A

11. Martin Burger Total Variation 11 Cetraro, September 2008 Fixed Point Schemes II Chambolle‘s Method Dual method, again eliminates u

12. Martin Burger Total Variation 12 Cetraro, September 2008 Fixed Point Schemes III Chambolle‘s Method Complicated derivation from dual minimization problem in original paper Note: can be interpreted as a scheme where the first equation is always satisfied, in addition using dual fixed point form for the primal subgradient relation, i.e. special case with

13. Martin Burger Total Variation 13 Cetraro, September 2008 Fixed Point Schemes III Chambolle‘s Method, Chambolle 04 + easy to implement, efficient iteration steps + global convergence (descent method for dual problem) + no approximation necessary - slow convergence - needs inversion of A*A, hence good for ROF, bad for inverse problems Obvious generalization for last point: use preconditioning of A*A

14. Martin Burger Total Variation 14 Cetraro, September 2008 Fixed Point Schemes IV Inexact Uzawa method, Zhu and Chan 08 Primal gradient descent, dual projected gradient ascent in the reduced Lagrangian (for u and w) Coincides with dual gradient projection if A = I and appropriate choice of damping parameter 

15. Martin Burger Total Variation 15 Cetraro, September 2008 Fixed Point Schemes V Primal Lagged Diffusivity with Approximation, Vogel et al 95-97 Approximate smoothed primal optimality condition Semi-implicit treatment of differential operator

16. Martin Burger Total Variation 16 Cetraro, September 2008 Fixed Point Schemes V Special case with choice

17. Martin Burger Total Variation 17 Cetraro, September 2008 Fixed Point Schemes V Primal Lagged Diffusivity with Approximation + acceptable step-size restrictions + global convergence (descent method for variational problem) - dependent on approximation - still slow convergence - differential equation with changing parameter to be solved in each step

18. Martin Burger Total Variation 18 Cetraro, September 2008 2. Thesholding methods C is damping matrix, possible perturbation T is thresholding operator

19. Martin Burger Total Variation 19 Cetraro, September 2008 Thresholding Methods I Primal Thresholding Method, Daubechies-Defrise-DeMol 03 Only used for D = -I Introduce

20. Martin Burger Total Variation 20 Cetraro, September 2008 Thresholding Methods I Primal Thresholding Method, Daubechies-Defrise-DeMol 03 Hence, special case with

21. Martin Burger Total Variation 21 Cetraro, September 2008 Thresholding Scheme I Primal Thresholding Method + easy to implement, efficient iteration steps if D= - I - slow convergence - cannot be generalized to cases where D is not invertible

22. Martin Burger Total Variation 22 Cetraro, September 2008 Thresholding Methods II Alternating Minimization, Yin et al 08, Amat-Pedregal 08 Use quadratic penalty for the gradient constraing (Moreau- Yosida regularization) Alternate minimization with respect to the variables

23. Martin Burger Total Variation 23 Cetraro, September 2008 Thresholding Methods II Alternating Minimization, Yin et al 08, Amat-Pedregal 08 Use quadratic penalty for the gradient constraing (Moreau- Yosida regularization) Alternate minimization with respect to the variables

24. Martin Burger Total Variation 24 Cetraro, September 2008 Thresholding Methods II Alternating Minimization, Yin et al 08, Amat-Pedregal 08 Introduce Hence, special case with

25. Martin Burger Total Variation 25 Cetraro, September 2008 Thresholding Scheme II Primal Thresholding Method + efficient iteration steps if D*D and A*A can be jointly inverted easily (e.g. by FFT) + treats differential operator implicitely, no severe stability bounds -Linear convergence - Smoothes the regularization functional

26. Martin Burger Total Variation 26 Cetraro, September 2008 Thresholding Methods III Split Bregman, Goldstein-Osher 08 Original motivation from Bregman iteration, can be rewritten as

27. Martin Burger Total Variation 27 Cetraro, September 2008 Thresholding Methods III Split Bregman, Goldstein-Osher 08 Difficult to solve directly, hence subiteration with thresholding After renumbering

28. Martin Burger Total Variation 28 Cetraro, September 2008 Thresholding Scheme III Split Bregman, Goldstein-Osher 08 + efficient iteration steps if D*D and A*A can be jointly inverted easily (e.g. by FFT) + treats differential operator implicitely, no severe stability bounds + does not need smoothing - Linear convergence

29. Martin Burger Total Variation 29 Cetraro, September 2008 2. Newton type methods Matrix form

30. Martin Burger Total Variation 30 Cetraro, September 2008 Newton-type Methods Primal or dual + fast local convergence - global convergence difficult - dependent on approximation (Newton-matrix degenerates) - needs inversion of large Newton matrix Good choice with efficient preconditioning for linear system in each iteration step