1. Total Variation and Geometric Regularization for Inverse Problems
Regularization in Statistics
September 7-11, 2003
BIRS, Banff, Canada
Tony Chan Department of Mathematics, UCLA

2. Outline TV & Geometric Regularization (related concepts)
PDE and Functional/Analytic based
Geometric Regularization via Level Sets Techniques
Applications (this talk):
Image restoration
Image segmentation
Elliptic Inverse problems
Medical tomography: PET, EIT

3. Regularization: Analytical vs Statistical
Controls “smoothness” of continuous functions
Function spaces (e.g. Sobolov, Besov, BV)
Variational models -> PDE algorithms
Statistical:
Data driven priors
Stochastic/probabilistic frameworks
Variational models -> EM, Monte Carlo

4. Taking the Best from Each?
Concepts are fundamentally related:
e.g. Brownian motion  Diffusion Equation
Statistical frameworks advantages:
General models
Adapt to specific data
Analytical frameworks advantages:
Direct control on smoothness/discontinuities, geometry
Fast algorithms when applicable

5. Total Variation Regularization
Measures “variation” of u, w/o penalizing discontinuities.
|.| similar to Huber function in robust statistics.
1D: If u is monotonic in [a,b], then TV(u) = |u(b) – u(a)|, regardless of whether u is discontinuous or not.
nD: If u(D) = char fcn of D, then TV(u) = “surface area” of D.
(Coarea formula)
Thus TV controls both size of jumps and geometry of boundaries.
Extensions to vector-valued functions
Color TV: Blomgren-C 98; Ringach-Sapiro, Kimmel-Sochen

6. The Image Restoration Problem
A given Observed image z
Related to True Image u
Through Blur K
And Noise n
Initial
Blur
Blur+Noise
Inverse Problem: restore u, given K and statistics for n.
Keeping edges sharp and in the correct location is a key problem !

7. Total Variation Restoration
Regularization:
Variational Model:
* First proposed by Rudin-Osher-Fatemi ’92.
* Allows for edge capturing (discontinuities along curves).
* TVD schemes popular for shock capturing.
Gradient flow:
anisotropic diffusion
data fidelity

8. Comparison of different methods for signal denoising & reconstruction

9. Image Inpainting (Masnou-Morel; Sapiro et al 99)
Disocclusion
Graffiti Removal

10. Unified TV Restoration & Inpainting model
(C- J. Shen 2000)

11. TV Inpaintings: disocclusion

12. Examples of TV Inpaintings
Where is the Inpainting Region?

13. TV Zoom-in
Inpaint Region: high-res points that are not low-res pts

14. Edge Inpainting Inpaint region: points away from Edge Tubes
edge tube T
No extra data are
needed. Just inpaint!
Inpaint region: points away from Edge Tubes

15. Extensions Color (S.H. Kang thesis 02)
“Euler’s Elastica” Inpainting (C-Kang-Shen 01)
Minimizing TV + Boundary Curvature
“Mumford-Shah” Inpainting (Esedoglu-Shen 01)
Minimizing boundary + interior smoothness:

18. Geometric Regularization
Minimizing surface area of boundaries and/or volume of objects
Well-studied in differential geometry: curvature-driven flows
Crucial: representation of surface & volume
Need to allow merging and pinching-off of surfaces
Powerful technique: level set methodology (Osher/Sethian 86)

19. Level Set Representation (S. Osher - J. Sethian ‘87)
Inside C
Outside C
Outside C C
C= boundary of an open domain
Example: mean curvature motion
* Allows automatic topology changes, cusps, merging and breaking.
Originally developed for tracking fluid interfaces.

20. Application: “active contour”
Initial Curve Evolutions Detected Objects

21. Basic idea in classical active contours
Curve evolution and deformation (internal forces):
Min Length(C)+Area(inside(C))
Boundary detection: stopping edge-function (external forces)
Example:
Snake model (Kass, Witkin, Terzopoulos 88)
Geodesic model (Caselles, Kimmel, Sapiro 95)

22. Limitations
- detects only objects with sharp edges defined by gradients
- the curve can pass through the edge
- smoothing may miss edges in presence of noise
- not all can handle automatic change of topology
Examples

23. A fitting term “without edges”
where
Fit > Fit > Fit > Fit ~ 0
Minimize: (Fitting +Regularization)
Fitting not depending on gradient detects “contours without gradient”

24. An active contour model “without edges”
(C. + Vese 98)
Fitting + Regularization terms (length, area)
C = boundary of an open and bounded domain
|C| = the length of the boundary-curve C

25. Mumford-Shah Segmentation 89
S=“edges”
MS reg: min boundary + interior smoothness
CV model = p.w. constant MS

26. The level set formulation of the active contour model
Variational Formulations and Level Sets
(Following Zhao, Chan, Merriman and Osher ’96)
The Heaviside function
The level set formulation of the active contour model

27. Using smooth approximations for the Heaviside and Delta functions
The Euler-Lagrange equations
Using smooth approximations for the Heaviside and Delta functions

28. Experimental Results Automatically detects interior contours!
Advantages
Automatically detects interior contours!
Works very well for concave objects
Robust w.r.t. noise
Detects blurred contours
The initial curve can be placed anywhere!
Allows for automatical change of topolgy

29. A plane in a noisy environment
Europe nightlights

30. Multiphase level set representations and partitions allows for triple junctions, with no vacuum and no overlap of phases
4-phase segmentation
2 level set functions
2-phase segmentation
1 level set function

31. Example: two level set functions and four phases

32. An MRI brain image Phase 11 Phase 10 Phase 01 Phase 00
mean(11)= mean(10)= mean(01)= mean(00)=103

34. References for PDE & Level Sets in Imaging
* IEEE Tran. Image Proc. 3/98, Special Issue on PDE Imaging
* J. Weickert 98: Anisotropic Diffusion in Image Processing
* G. Sapiro 01: Geometric PDE’s in Image Processing
Aubert-Kornprost 02: Mathematical Aspects of Imaging Processing
Osher & Fedkiw 02: “Bible on Level Sets”
Chan, Shen & Vese Jan 03, Notices of AMS