Download presentation
Presentation is loading. Please wait.
Published byJanel Gibson Modified over 9 years ago
1
Image Matting and Its Applications Chen-Yu Tseng Advisor: Sheng-Jyh Wang 2012-10-29
2
Image Matting A process to extract foreground objects from an image, along with an alpha matte (the opacity of the foreground color) Input ImageAlpha MatteExtracted Foreground
3
Two Approaches of Image Matting Supervised Matting With User’s Guidance Unsupervised Matting Without User’s Guidance Input ImageUser’s Guidance e.g. Trimap: White Foreground Black Background Unknown Gray
4
Two Schemes of Supervised Matting Propagation-based Scheme Infer Alpha Matte with Propagation through a Graphical Model A Global-based Approach Sampling-based Scheme Infer Alpha Matte with Some Color Samples A Local-based Approach Foreground Pixel Background Pixel Unknown Pixel Foreground Color Set Background Color Set Unknown Pixel
5
Propagation-based scheme - Matting Laplacian Approach A Graphical Model with Connectivity between Pixels The Connectivity Is Learned from the Image Structure Capability for Dealing with Both Supervised Matting (Inference Problem) Unsupervised Matting (Decomposition Problem) Foreground Pixel Background Pixel Unknown Pixel
6
Reference of Matting Laplacian Approach First proposed by Levin et al. for supervised matting (closed-form matting) A. Levin, D. Lischinski, Y. Weiss. “A Closed Form Solution to Natural Image Matting,” IEEE T. PAMI, vol. 30, no. 2, pp. 228-242, Feb. 2008. Extended to unsupervised matting (spectral matting) A. Levin, A. Rav-Acha, D. Lischinski. “Spectral Matting,” IEEE T. PAMI, vol. 30, no. 10, pp. 1699-1712, Oct. 2008. Extended to learning-based matting Y. Zheng and C. Kambhamettu. “Learning based digital matting,” In ICCV, pages 889–896, 2009. Extended to multi-layer matting D. Singaraju, R. Vidal. “Estimation of Alpha Mattes for Multiple Image Layers,” IEEE T. PAMI, vol. 33, no. 7, pp. 1295-1309, July 2011.
7
Matting Laplacian Input Image Estimating Pair-wise Affinity Graphical Model Node: Image Pixels Edge: Affinity Supervised Matting Background Foreground Matting Laplacian Matrix: Recording the Connectivity between Pair of Pixels
8
Introduction of Graph Laplacian 2 2 3 3 1 1 4 4 5 5 A Graph with Five Vertexes 01100 10100 11000 00001 00010 1 2 3 4 5 12345 Vertex Index
9
Introduction of Graph Laplacian 2 3 1 4 5 A Graph with Five Vertexes 01100 10100 11000 00001 00010 1 2 3 4 5 12345 Vertex Index 2 00 2 00 200 0001 000 1 20000 02000 00200 00010 00001
10
Cutting Cost Function with Graph Laplacian Cost Function for Cutting Criterion Low-cost Assignment High-cost Assignment 2 2 3 3 1 1 4 4 5 5 2 2 3 3 1 1 4 4 5 5
11
Construction of Matting Laplacian Color-model-based Approach (Original) Estimating Affinity Based on Relative Color Distance Learning-based Approach (Extended) Learning Affinity Based on Image Structure
12
Construction of Matting Laplacian Color-model-based Approach Color Distribution Input Image A. Levin, D. Lischinski, Y. Weiss. “A Closed Form Solution to Natural Image Matting,” IEEE T. PAMI, vol. 30, no. 2, pp. 228-242, Feb. 2008. g r b
13
Construction of Matting Laplacian Learning-based Approach Learning Affinity among Local Pixels Linear Alpha-color Model for Single Pixel: Extending to a Local Patch q Assuming all Pixels Sharing the Same Linear Coefficient
14
Construction of Matting Laplacian Learning-based Approach Derived Linear Coefficient Rewritten Linear Model
15
Construction of Matting Laplacian Local Cost Function Input Image Patch q Local Linear Model
16
Construction of Matting Laplacian Local Global Input Image Patch q
17
Supervised Matting (Closed-form Matting) Foreground Pixel Background Pixel Unknown Pixel Input Image ForegroundBackgroundUnknown 10- 110 Cost Function for Supervised Matting Affinity Cost Data Cost Optimal Solution
18
Experimental Results Input ImageAlpha MatteSynthesized Result
19
Unsupervised Matting (Spectral Matting) Solving Alpha Matte without User’s Guidance Procedures Decomposing Image into Several Matting Components Combining Matting Components into Alpha Matte
20
Spectral Clustering 1.L is symmetric and positive semi-definite. 2.The smallest eigenvalue of L is 0, the corresponding eigenvector is the constant one vector 1. 3.L has n non-negative, real-valued eigenvalues 0= λ 1 ≦ λ 2 ≦... ≦ λ n. 2 2 3 3 1 1 4 4 5 5 A Graph Example 2 00 2 00 200 0001 000 1 1 2 3 4 5 12345 0.047 0.577 0 0
21
Spectral Clustering & Matting Components 2 0000 2 0000 20000 0001100 000 00 0000011 00000 1 1 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 1 1 Zero-Eigenvectors Binary Indicating Vectors Linear Transformation
22
Overview of Spectral Matting Input Image Smallest Eigenvectors Matting Components K-means Clustering & Linear Transformation Matting Laplacian
23
Spectral Clustering & K-means Input Image s-smallest Eigenvectors … Pixel i s-dimensional Space K-means Clustering
24
Generating Matting Components Smallest Eigenvectors Projection into Eigen Space K-means ………
25
Reconstructing Alpha Matte from Matting Components =++ Input Image Matting Components Selected Matting ComponentsAlpha Matte
26
Reconstructing Alpha Matte by Grouping Matting Components Matting cost function Alpha Matte Generation Evaluating All Grouping Hypothesis to Derive the Optimal Alpha Matte
27
Results by Levin et al.
28
Summary Constructing Matting Laplacian Solving Supervised Matting Problem Solving Unsupervised Matting Problem
29
Proposed Approaches Efficient Cell-based Framework for Reducing Computations Multi-scale Analysis Extended Applications (Depth Image Reconstruction) Input Image Reconstructed Depth Depth Reconstruction from Single Image Depth Reconstruction in Shape From Focus (SFF) Input ImageReconstructed Depth
30
Cell-based Framework Image Pixel-wise Data Distribution Cell-wise Data Distribution Conventional Matting Laplacian Cell-based Matting Laplacian Pixel-wise Affinity Cell-wise Affinity
31
Multi-scale Affinity Learning Image & Computation Patterns Pixel-based Approach Cell-based Approach
32
Multi-scale Affinity Learning … Finest Level Coarsest Level … Cell-based Graph
33
Results of Reconstructed Alpha Matte 1 st Rank 2 nd Rank (a) Grouping Results by Levin et al. (b) Grouping Results by Levin et al. with Coarse-to-fine Scheme. (c) Ours Input
34
Results (a) Input images(b) Levin’s result(c) Our result
35
Proposed Approaches Efficient Cell-based Framework for Reducing Computations Multi-scale Analysis Extended Applications (Depth Image Reconstruction) Input Image Reconstructed Depth Depth Reconstruction from Single Image Depth Reconstruction in Shape From Focus (SFF) Input ImageReconstructed Depth
36
Depth Reconstruction in Shape From Focus (SFF) Optical Direction Multi-focus Image Sequence Optical Direction Focus Value W1W1 W2W2 W2W2 W1W1
37
Low-SNR Problem Spatially Varying Precision Low-texture Low-SNR Leading Noisy Result Input Image Observation High- precision Low- precision
38
Proposed Maximum-a-posteriori Estimation Multi-focus Image Sequence Learning-based Graph Local Learning Inference Reconstructed Depth
39
Proposed Maximum-a-posteriori Estimation PosteriorLikelihoodPrior Local Observation with Spatial-varying Precision Learned from Image
40
Likelihood Model Input Observation Precision Result High- precision Low- precision PosteriorLikelihoodPrior Local Observation with Spatial-varying Precision
41
Prior Model PosteriorLikelihoodPrior Learning from Input Image Learning-based Graph Local Learning Multi-focus Image Sequence
42
Maximum-a-posteriori Estimation for Depth Reconstruction Input ImageObservationReconstructed Depth
43
Results of Shape from Focus Input Image M. Mahmood, 2012T. Aydin, 2008Ours S. Nayar, 1994
44
Conclusions Construction of Matting Laplacian Conventional Approach Multi-scale Cell-based Approach Supervised Matting Spectral Matting Depth Reconstruction
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.