1. HCI Project : An Iterative Optimization Approach for Unified Image Segmentation and Matting

2. Abstract
Extracting a matte by previous approaches require the input image to be pre-segmented into three regions (trimap).
This pre-segmentation based approach fails for images with large portions of semi-transparent foreground.
In this paper we combine the segmentation and matting problem together and propose a unified optimization approach based on Belief Propagation.

3. Introduction
The observed image I(z) (z = (x, y)) is modeled as a linear combination of foreground image F(z) and background image B(z) by an alpha map: I(z) = αzF(z) + (1 − αz)B(z)
Image Matting
estimating an opacity (alpha value) and foreground and background colors for each pixel in the image.

4. Limitations of a Trimap
To generate good mattes, all these approaches require the user to ”carefully” specify the trimap.
it is almost impossible to manually create an optimal trimap.

5. Limitations of a Trimap (cont.)
Automatically generated trimaps based on the binary segmentation result is non-optimal, since it always has uniform thickness regardless of local image characteristics.

6. MRF Construction Each pixel in and are treated as a node in the MRF
Minimize the total energy of the following function
：How well the estimated alpha value , and foreground and background color for fit with the actual color
：The smoothness energy which penalizes inconsistent alpha value changes between two neighbors and

7. Markov Random Field
上層 – 原值
下層 – 估計值

8. Predefined arguments
Discretize the possible alpha value to 25 levels between 0 and 1,denoted as , k=1,…,25
Each level corresponds to a possible state for a node in the MRF
The local neighborhood area is defined to have a radius of r=20

9. in detail Compute the likelihood of each alpha level as
The set of valid foreground samples, are then weighted by their uncertainty and distance, by

10. in detail
The smoothness cost is defined as

11. Belief Propagation Optimization
Use loopy belief propagation (BP) to solve problem
Finding a labeling with minimum energy corresponds to the MAP estimation problem
It works by passing messages along links in the constructed path

12. BP Algorithm(1)
In each iteration, new messages are computed for each possible state
H (p) \ q denotes the neighbors of p other than q
c is a normalization factor

13. BP Algorithm(2)
After T iterations a belief vector is computed for each node
The state the maximizes at each
node is selected as the estimated level

14. BP Algorithm(3) If =1, set the color as a new foreground sample
If =0, set the color as a new background sample
Otherwise, choose the pair of foreground and background colors from the group of samples

15. BP Algorithm(4) Then, the uncertainty value u(p) is updated as
and are weights for the selected pair of foreground and background samples

16. Iterative Belief Propagation for Image Matting

17. Extension to Video
is small
as definite foreground

18. Result and Comparisons
Propose approach
Bayesian

19. Extracted foreground and novel background

20. Foreground and background is similar
Not work well

21. Start with initial trimap
Rough trimap the user created
Bayesian
Proposed
approach

22. Summary and conclusion
Proposed a approach to solve image matting problem
combines the problems of segmentation and matting into a unified formula
Does not require a well specified trimap