1. Saliency detection with background model
Donghun Yeo
CV Lab.

2. Contents Definition of Saliency Detection
Saliency Detection via Dense and Sparse Reconstruction
Saliency Detection via Absorbing Markov Chain

3. Definition of Saliency Detection
Locate important and interesting regions or objects in an image.
Generally the output is saliency value for each pixel.
input image GT
Result of [Mai et al. 2013]
input image
GT of saliency value
[Mai et al. 2013] Saliency Aggregation: A Data-driven Approach, CVPR 2013

4. Saliency Detection via Absorbing Markov Chain
Bowen Jiang* (DUT), Lihe Zhang (DUT, China), huchuan Lu (DUT,China), Chuan Yang, Ming-Hsuan Yang (UC Merced, USA).
Image boundary pixels are background!

5. Principles of Markov Chain
A set of states
Transition matrix : 𝑚×𝑚
𝑝 𝑖𝑗 : the probability of moving from state 𝑠 𝑖 to state 𝑠 𝑗

6. Absorbing Markov Chain
Absorbing state
A state is absorbing when which has no outgoing edge.
Absorbing Markov chain
A Markov chain is absorbing if it has at least one absorbing state.
Saliency Detection via Absorbing Markov Chain 1
0.6
0.7
0.4
0.3
Make boundary superpixels as absorbing states,
Use mean absorbing time to compute the saliency

7. Principles of Markov Chain
Absorbing Markov chain
Absorbing state 𝑠 𝑖 : 𝑝 𝑖𝑖 =1
Transient state : not absorbing
A Markov chain with 𝑟 absorbing states and 𝑡 transient states
By renumbering the states so that the transient states come first,
Expected number of times that a chain spends in the transient states 𝑗 given that the chain starts in the transient state 𝑖 : 𝑛 𝑖𝑗
Expected number of times that a chain starting in transient state 𝑖 spends before absorption : , where c is l dim vector all of whose elements 1

8. Graph Construction Generate superpixels
Duplicate the boundary superpixels around the image borders
The duplicated superpixels are absorbing states
Others are transient states
Each node is connected to the transient nodes which neighbor it and transient nodes which share boundaries with its neighboring nodes.
The nodes in the boundaries are fully connected with each other

9. Graph Construction : mean color similarity : Affinity 𝑨
: Transition matrix P

10. Saliency Detection
Saliency : Expected number of times that a chain starting in transient state 𝑖 spends before absorption
The normalized 𝑦(𝑖) into [0,1]

11. Problems!
When the salient region appears at the boundaries : They claims that this method is robust to this kind of situation (images in the paper.)
But it is not robust !!!

12. New proposal
Discover the boundary superpixels which are part of salient region.

13. Another Principle in Absorbing Markov Chain
Absorbing state 𝑠 𝑖 : 𝑝 𝑖𝑖 =1
Transient state : not absorbing
A Markov chain with 𝑟 absorbing states and 𝑡 transient states
By renumbering the states so that the transient states come first,
Expected number of times that a chain spends in the transient states 𝑗 given that the chain starts in the transient state 𝑖 : 𝑛 𝑖𝑗
Probability of that the chain starting in the transient state 𝑖 is absorbed into the absorbing state 𝑗 : 𝑏 𝑖𝑗
𝐵=𝑁𝑅

15. Experiments Evaluation Metric Precision-Recall curve F-measure,
Mean Precision , Mean Recall, Mean F-measure

16. Experiments - ASD, SED

17. Experiments – MSRA Dataset

18. Effect of Update Processing

19. Results

20. ExcutionTime

21. Try to focus on the second problem.

22. Absorbing Markov Chain
Background proposal
Saliency Detection via Background Object Proposal
Framework
Superpixel로 image를 분할
Object proposals [Manen et al.]을 이용해 각 Boundary superpixel을 seed로 boundary object region을 찾아, background modeling
Boundary에 추가적으로 내부 region의 정보까지 이용하여 Global contrast의 장점을 가져옴
앞서 구해낸 Background 정보와 Absorbing Markov chain algorithm [Bowen Jiang et al.] 이용해 Saliency를 계산
AMCBP
Superpixel
Background Proposal
Absorbing Markov Chain

23. Object Proposal
Find the object containing a seed superpixel by attaching neighbors of them.

24. Object Proposal

25. Background Proposal
Propose an object from boundary superpixels as the seeds.
Each boundary superpixel selected twice (any) as the seed.

26. Absorbing Markov Chain
Background proposal
Saliency Detection via Background Object Proposal
Framework
Superpixel로 image를 분할
Object proposals [Manen et al.]을 이용해 각 Boundary superpixel을 seed로 boundary object region을 찾아, background modeling
Boundary에 추가적으로 내부 region의 정보까지 이용하여 Global contrast의 장점을 가져옴
앞서 구해낸 Background 정보와 Absorbing Markov chain algorithm [Bowen Jiang et al.] 이용해 Saliency를 계산
AMCBP
Superpixel
Background Proposal
Absorbing Markov Chain

27. Absorbing Markov Chain
MSRA1000 dataset 실험 결과 이미지
Original Image
Ours
Absorbing Markov Chain
GSSP
HyperContext
우리의 방법이 Abosorbing Markov Chain에서 배경을 영역의 Saliency를 제거하는데, 더 좋은 역할을 한다는 것을 볼 수 있다.

28. Absorbing Markov Chain
BSD300 dataset 실험 결과 이미지
Original Image
Ours
Absorbing Markov Chain
GSSP
HyperContext
우리의 방법이 Abosorbing Markov Chain에서 배경을 영역의 Saliency를 제거하는데, 더 좋은 역할을 한다는 것을 볼 수 있다.

29. Precision-Recall Curve
MSRA1000 dataset 실험 결과
Precision-Recall Curve
F-measure

30. Precision-Recall Curve
BSD300 dataset 실험 결과
Precision-Recall Curve
F-measure

31. What is the main problem?

32. Salient object가 image boundary에 존재 할 때
Original Image
Superpixel
Background Proposal
Saliency

33. Contextual Hypergraph Modelling for Salient Object Detection
Xi Li, Yao Li, Chunhua Shen, Anthony Dick, Anton van den Hengel, Contextual Hypergraph Modelling for Salient Object Detection, ICCV 2013

34. Contexture Hypergraph
Contexture Hypergraph [Xi Li et al.]
Clustering을 통해서 image를 여러 영역으로 segmentation
각각의 segment에 대해 주변 영역과의 차이를 이용해 saliency 값을 계산. 단점
이 방법에서도 image boundary에 접하는 region은 배경이 확률이 높다고 가정하고 계산.

35. Contexture Hypergraph
Foreground/Background
Saliency of Foreground

36. How to select salient segment among them – In Contexture Hypergraph
Set the saliency of boundary superpixels to be -1(or -inf)
the saliency of the segment = average the saliency values of all the superpixels -1

37. Futurework
How to determine which one is the salient segment.

38. Difficulty of BSD300