1 Total variation minimization Numerical Analysis, Error Estimation, and Extensions Martin Burger Johannes Kepler University Linz SFB Numerical-Symbolic-Geometric Scientific Computing Radon Institute for Computational & Applied Mathematics Westfälische Wilhelms Universität Münster

Total variation minimization Obergurgl, September Stan Osher, Jinjun Xu, Guy Gilboa (UCLA) Lin He (Linz / UCLA) Klaus Frick, Otmar Scherzer (Innsbruck) Carola Schönlieb (Vienna) Don Goldfarb, Wotao Yin (Columbia) Collaborations

Total variation minimization Obergurgl, September Total variation methods are popular in imaging (and inverse problems), since - they keep sharp edges - eliminate oscillations (noise) - create new nice mathematics Many related approaches appeared in the last years, e.g. ℓ 1 penalization / sparsity techniques Introduction

Total variation minimization Obergurgl, September Total variation and related methods have some shortcomings - difficult to analyze and to obtain error estimates - systematic errors (clean images not reconstructed perfectly) - computational challenges - some extensions to other imaging tasks are not well understood (e.g. inpainting) Introduction

Total variation minimization Obergurgl, September Starting point of the analysis is the ROF model for denoising Rudin-Osher Fatemi 89/92, Acar-Vogel 93, Chambolle-Lions 96, Vogel 95/96, Scherzer-Dobson 96, Chavent-Kunisch 98, Meyer 01,… ROF Model

Total variation minimization Obergurgl, September ROF Model Reconstruction (code by Jinjun Xu) cleannoisy ROF

Total variation minimization Obergurgl, September First question for error estimation: estimate difference of u (minimizer of ROF) and f in terms of Estimate in the L 2 norm is standard, but does not yield information about edges Estimate in the BV-norm too ambitious: even arbitrarily small difference in edge location can yield BV-norm of order one ! Error Estimation

Total variation minimization Obergurgl, September We need a better error measure, stronger than L2, weaker than BV Possible choice: Bregman distance Bregman 67 Real distance for a strictly convex differentiable functional – not symmetric Symmetric version Error Estimation

Total variation minimization Obergurgl, September Total variation is neither symmetric nor differentiable Define generalized Bregman distance for each subgradient Symmetric version Kiwiel 97, Chen-Teboulle 97 Error Estimation

Total variation minimization Obergurgl, September Since TV seminorm is homogeneous of degree one, we have Bregman distance becomes Error Estimation

Total variation minimization Obergurgl, September Bregman distance for TV is not a strict distance, can be zero for In particular d TV is zero for contrast change Resmerita-Scherzer 06 Bregman distance is still not negative (TV convex) Bregman distance can provide information about edges Error Estimation

Total variation minimization Obergurgl, September Let v be piecewise constant with white background and color values on regions Then we obtain subgradients of the form with signed distance function and Error Estimation

Total variation minimization Obergurgl, September Bregman distances given by In the limit we obtain for being piecewise continuous Error Estimation

Total variation minimization Obergurgl, September For estimate in terms of we need smoothness condition on data Optimality condition for ROF Error Estimation

Total variation minimization Obergurgl, September Subtract q Estimate for Bregman distance, mb-Osher 04 Error Estimation

Total variation minimization Obergurgl, September In practice we have to deal with noisy data f (perturbation of some exact data g) Estimate for Bregman distance Error Estimation

Total variation minimization Obergurgl, September Optimal choice of the penalization parameter i.e. of the order of the noise variance Error Estimation

Total variation minimization Obergurgl, September Direct extension to deconvolution / linear inverse problems under standard source condition mb-Osher 04 Extension: stronger estimates under stronger conditions, Resmerita 05 Nonlinear inverse problems, Resmerita-Scherzer 06 Error Estimation

Total variation minimization Obergurgl, September Natural choice: primal discretization with piecewise constant functions on grid Problem 1: Numerical analysis (characterization of discrete subgradients) Problem 2: Discrete problems are the same for any anisotropic version of the total variation Discretization

Total variation minimization Obergurgl, September In multiple dimensions, nonconvergence of the primal discretization for the isotropic TV (p=2) can be shown Convergence of anisotropic TV (p=1) on rectangular aligned grids Fitzpatrick-Keeling 1997 Discretization

Total variation minimization Obergurgl, September Alternative: perform primal-dual discretization for optimality system (variational inequality) with convex set Primal-Dual Discretization

Total variation minimization Obergurgl, September Discretization Discretized convex set with appropriate elements (piecewise linear in 1D, Raviart- Thomas in multi-D) Primal-Dual Discretization

Total variation minimization Obergurgl, September In 1 D primal, primal-dual, and dual discretization are equivalent Error estimate for Bregman distance by analogous techniques Note that only the natural condition is needed to show Primal / Primal-Dual Discretization

Total variation minimization Obergurgl, September In multi-D similar estimates, additional work since projection of subgradient is not discrete subgradient. Primal-dual discretization equivalent to discretized dual minimization (Chambolle 03, Kunisch-Hintermüller 04). Can be used for existence of discrete solution, stability of p mb 06/07 ? Primal / Primal-Dual Discretization

Total variation minimization Obergurgl, September For most imaging applications Cartesian grids are used. Primal dual discretization can be reinterpreted as a finite difference scheme in this setup. Value of image intensity corresponds to color in a pixel of width h around the grid point. Raviart-Thomas elements on Cartesian grids particularly easy. First component piecewise linear in x, pw constant in y,z, etc. Leads to simple finite difference scheme with staggered grid Cartesian Grids

Total variation minimization Obergurgl, September ROF minimization has a systematic error, total variation of the reconstruction is smaller than total variation of clean image. Image features left in residual f-u g, clean f, noisy u, ROFf-u Extension I: Iterative Refinement & ISS

Total variation minimization Obergurgl, September Idea: add the residual („noise“) back to the image to pronounce the features decreased to much. Then do ROF again. Iterative procedure Osher-mb-Goldfarb-Xu-Yin 04 Extension I: Iterative Refinement & ISS

Total variation minimization Obergurgl, September Improves reconstructions significantly Extension I: Iterative Refinement & ISS

Total variation minimization Obergurgl, September Extension I: Iterative Refinement & ISS

Total variation minimization Obergurgl, September Simple observation from optimality condition Consequently, iterative refinement equivalent to Bregman iteration Extension I: Iterative Refinement & ISS

Total variation minimization Obergurgl, September Choice of parameter less important, can be kept small (oversmoothing). Regularizing effect comes from appropriate stopping. Quantitative stopping rules available, or „stop when you are happy“ – S.O. Limit to zero can be studied. Yields gradient flow for the dual variable („inverse scale space“) mb-Gilboa-Osher-Xu 06, mb-Frick-Osher-Scherzer 06 Extension I: Iterative Refinement & ISS

Total variation minimization Obergurgl, September Non-quadratic fidelity is possible, some caution needed for L 1 fidelity He-mb-Osher 05, mb-Frick-Osher-Scherzer 06 Error estimation in Bregman distance mb-Resmerita 06, in prep Further details see talk of Klaus Frick Extension I: Iterative Refinement & ISS

Total variation minimization Obergurgl, September Extension I: Inverse Scale Space Movie by M. Bachmayr, Master Thesis 06

Total variation minimization Obergurgl, September Application to other regularization techniques, e.g. wavelet thresholding is straightforward Starting from soft shrinkage, iterated refinement yields firm shrinkage, inverse scale space becomes hard shrinkage Osher-Xu 06 Bregman distance natural sparsity measure, source condition just requires sparse signal, number of nonzero components is smoothness measure in error estimates Extension I: Iterative Refinement & ISS

Total variation minimization Obergurgl, September Total variation, inverse scale space, and shrinkage techniques can be combined nicely See talk by Lin He Extension I: Iterative Refinement & ISS

Total variation minimization Obergurgl, September Total variation will prefer isotropic structures (circles, spheres) or special anisotropies In many applications one wants sharp corners in different directions. Adaptive anisotropy is needed Can be incorporated in ROF and ISS. See talk by Benjamin Berkels Extension II: Anisotropy

Total variation minimization Obergurgl, September Difficult to construct total variation techniques for inpainting Original extensions of ROF failed to obtain natural connectivity (see book by Chan, Shen 05) Inpainting region, image f (noisy) given on Try to minimize Extension III: Inpainting

Total variation minimization Obergurgl, September Optimality condition will have the form with A being a linear operator defining the norm In particular p = 0 in D ! Extension III: Inpainting

Total variation minimization Obergurgl, September Different iterated approach (motivated by Cahn-Hilliard inpainting, Bertozzi et al 05 ) Minimize in each step First term for damping, second for fidelity (fit to f where given, and to old iterate in the inpainting region), third term for smoothing Extension III: Inpainting

Total variation minimization Obergurgl, September Continuous flow for damping parameter to zero Fourth order flow for H -1 norm Stationary solution (existence ?) satisfies Extension III: Inpainting

Total variation minimization Obergurgl, September Result: Penguins Extension III: Inpainting

Total variation minimization Obergurgl, September Original motivation: Osher-Marquinha 01 used preconditioned gradient flow for ROF Stationary state assumed to be ROF minimizer Computational observation: not always true ! Trivial observation: for initial value u(0) = 0 the flow remains zero for all time ! Extension IV: Manifolds

Total variation minimization Obergurgl, September Embarrassing observation: flow always created by transport from initial value Important observation: Stationary state minimizes ROF on the manifold Extension IV: Manifolds

Total variation minimization Obergurgl, September Surprising observation: for f being the indicator function of a convex set, the flow is equivalent to the gradient flow of the L 1 version of ROF No loss of contrast ! More detailed analysis for general images needed Possible extension to ROF minimization on other manifolds by metric gradient flows Extension IV: Manifolds

Total variation minimization Obergurgl, September Download and Contact Papers and Talks: from October: wwwmath1.uni-muenster.de/num