1. Image Matting with the Matting Laplacian
Chen-Yu Tseng 曾禎宇
Advisor: Sheng-Jyh Wang

2. Image Matting with the Matting Laplacian
A. Levin, D. Lischinski, Y. Weiss. A Closed Form Solution to Natural Image Matting. IEEE T. PAMI, vol. 30, no. 2, pp , Feb
Spectral Matting
A. Levin, A. Rav-Acha, D. Lischinski. Spectral Matting. IEEE T. PAMI, vol. 30, no. 10, pp , Oct
Matting for Multiple Image Layers
D. Singaraju, R. Vidal. Estimation of Alpha Mattes for Multiple Image Layers. IEEE T. PAMI, vol. 33, no. 7, pp , July 2011.
Center for Imaging Science, Department of Biomedical Engineering,
The Johns Hopkins University

3. Spectral Matting Result
Image Matting
Extracting a foreground object from an image along with an opacity estimate for each pixel covered by the object
Input Image
Conventional
Segmentation Result
Spectral Matting Result

4. Image Compositing Equationx = +
Input Image L1 L2 L3 α1 α2 α3
Alpha Mattes
Image Layers

5. Trimap (user’s constraint)
Methodology
Supervised Matting
Unsupervised Matting
Spectral Matting
Input Image
Trimap (user’s constraint)
Alpha Matte
Input Image
Matting Components

6. Local Models for Alpha Mattes
𝐼 𝑖 = 𝛼 𝑖 𝐹 𝑖 +(1− 𝛼 𝑖 ) 𝐵 𝑖
𝛼, 𝐹, and 𝐵 are unknown  ill-posed problem = x +
𝛼 𝑖 = 𝐼 𝑖 − 𝐵 𝑖 𝐹 𝑖 − 𝐵 𝑖 ≈𝑎 𝐼 𝑖 +𝑏 ,∀𝑖∈𝑤
Assume a and b are constant
in a small window

7. Color Line Assumption
Omer and M. Werman. Color Lines: Image Specific Color Representation. CVPR, 2004.
Input
Color Distributions

8. Local Models for Alpha Mattes for Multiple Layers
Two color lines
A color point and a color point
Two color points and a single color line
Four color points B G R

9. Local Models Two color lines A color plane and a color point
Two color points and a single color line
Four color points
𝐹 𝑖
𝐼 𝑖 = 𝛼 𝑖 𝐹 𝑖 +(1− 𝛼 𝑖 ) 𝐵 𝑖
Color point
𝐼 𝑖
Unknown color point
𝐵 𝑖
Color plane

10. Local Models Two color lines A color plane and a color point
Two color points and a single color line
Four color points
𝐹 𝑗 1
𝐼 𝑗 = 𝛼 𝑗 1 𝐹 𝑗 𝛼 𝑗 2 𝐹 𝑗 2
Color point
𝐹 𝑗 1 = 𝐶 1
𝐼 𝑖
𝐹 𝑗 2 = 𝛽 𝑗1 𝐶 2 + 𝛽 𝑗2 𝐶 3 +
(1− 𝛽 𝑗1 − 𝛽 𝑗2 ) 𝐶 3
𝐹 𝑗 2
Color plane

12. Local Models for Alpha Mattes for Multiple Layers
Two color lines
A color point and a color point
Two color points and a single color line
Four color points B G R

13. The Matting Laplacian

14. Overview of Spectral Matting
Input Data
Matting Laplacian
Construction
Input Image
Local Adjacency
Spectral Graph
Analysis
Data Component
Generation
Output Components
Laplacian Matrix
Components

15. Spectral Clustering Scatter plot of a 2D data set K-means Clustering
U. von Luxburg. A tutorial on spectral clustering. Technical report, Max Planck Institute for Biological Cybernetics, Germany, 2006.

16. Graph Construction Similarity Graph ε-neighborhood Graph
Connected Groups
Similarity Graph
Similarity Graph
Vertex Set
Weighted
Adjacency Matrix
Similarity Graph
ε-neighborhood Graph
k-nearest neighbor Graphs
Fully connected graph

17. Graph Laplacian 𝒇 𝑇 𝐿𝒇= 1 2 𝑖,𝑗=1 𝑛 𝑤 𝑖𝑗 𝑓 𝑖 − 𝑓 𝑗 2
W: adjacency matrix
L: Laplacian matrix
For every vector 𝒇
D: degree matrix
𝒇 𝑇 𝐿𝒇= 1 2 𝑖,𝑗=1 𝑛 𝑤 𝑖𝑗 𝑓 𝑖 − 𝑓 𝑗 2

18. Example 𝒇 𝑇 𝐿𝒇= 1 2 𝑖,𝑗=1 𝑛 𝑤 𝑖𝑗 𝑓 𝑖 − 𝑓 𝑗 2 W: adjacency matrix
L: Laplacian matrix 1 2 -1 1 1 2 3 4
𝒇 𝑇 𝐿𝒇= 1 2 𝑖,𝑗=1 𝑛 𝑤 𝑖𝑗 𝑓 𝑖 − 𝑓 𝑗 2
Cost
Function 5
Similarity Graph
Good Assignment
Poor Assignment 𝒇
𝒇 * 1 1 2 1 1 2 3 3 4 4 5 5

19. Laplacian Eigenvectors
arg min 𝑓 𝒇 𝑇 𝐿𝒇
s.t. 𝒇 𝑇 𝒇=1
𝒇: Eigenvector
λ: Eigenvalue
𝐿𝒇=λ𝒇
L is symmetric and positive semi-definite.
The smallest eigenvalue of L is 0, the corresponding eigenvector is the constant one vector 1.
L has n non-negative, real-valued eigenvalues
0= λ 1 ≦ λ 2 ≦ ≦ λ n.
Input Image
Smallest eigenvectors

20. From Eigenvectors to Matting Components
Smallest eigenvectors
K-means
Projection into eigs space

21. Overview of Spectral Matting
Input Data
Graph
Construction
Input Image
Local Adjacency
Spectral Graph
Analysis
Data Component
Generation
Output Components
Laplacian Matrix
Components

22. Matting Laplacian α F x x +
1-α B =

23. Matting Laplacian
𝐼 𝑖
𝐼 𝑗
𝜇 𝑘
Color Distribution

24. Matting Laplacian
Typical affinity function
Matting affinity function

25. Linear Transformation
Brief Summary
K-means Clustering
&
Linear Transformation
Input Image
Laplacian Matrix
Smallest Eigenvectors
Matting Components

26. Supervised Matting Cost function with user-specified constraint:
Foreground
Background
Unknown
Input
Trimap

27. Supervised Matting
𝜶 𝑇 𝐿𝜶= 1 2 𝑖,𝑗=1 𝑛 𝑤 𝑖𝑗 𝛼 𝑖 − 𝛼 𝑗 2

28. Estimation Alpha Matte for Two Layers

29. Estimation Alpha Matte for Multi-Layers
Karusch-Kuhn-Tucker (KKT) condition

30. The vector of 1s lies in the null space of L,
Assumption
Construction
The vector of 1s lies in the null space of L,
the solution automatically satisfies the constraint

31. Constrained Alpha Matte Estimation
Image matting for n≥2 image layers with positivity + summation constraints

32. Karusch-Kuhn-Tucker (KKT) conditions
For 0 < 𝛼 𝑘 𝑖 < 1
Λ 𝑘 0 (i,i)=0 and Λ 𝑘 1 (i,i)=0
Conventional Approaches
Directly Clipping
Equivalent to
Introducing Lagrange Multipliers
Refinement is neglected
in conventional approaches

33. Experiments

35. Algorithm 1.
(b) Algorithm 2.
(c) Spectral Matting.
(d) SM-enhance.

36. Algorithm 1.
(b) Algorithm 2.
(c) Spectral Matting.
(d) SM-enhance.

38. Summary Image Matting with the Matting Laplacian
Construction of the Matting Laplacian
Image Compositing Model
Local-Color Affine Model
Supervised Closed-form Matting
Two-layer
Multiple-layer
Spectral Matting
Extended Applications

39. Depth Estimation Compositing Image Likelihood Prior Input Image
Estimated
Depth
MAP
Prior
Refined Result
Confidence
Map

40. Input Image
Transmission Prior
Output Image
Refined Transmission

41. Graph Laplacian and Non-linear Filters
Global
Optima
Local
Optima
Global Optima
Local Optima
Gaussian-based
Bilateral Filter
Matting-Laplacian-based
Guided Filter (K. He, ECCV 2010)