ENSEMBLE SEGMENTATION USING EFFICIENT INTEGER LINEAR PROGRAMMING Ju-Hsin Hsieh Advisor : Sheng-Jyh Wang 2013/07/22 Amir Alush and Jacob Goldberger, “ Ensemble Segmentation Using Efficient Integer Linear Programming ”, IEEE Transactions on PAMI, 2012.

Outline ◦Introduction ◦Method ◦Experiment result ◦Conclusion ◦Reference 2

Outline ◦Introduction What is segmentation? Challenge Main idea ◦Method ◦Experiment result ◦Conclusion ◦Reference 3

What is segmentation? ◦Partitioning of an image into several constituent components. ◦Assign each pixel in the image to one of the image components. 4

Outline ◦Introduction What is segmentation? Challenge Main idea ◦Method ◦Experiment result ◦Conclusion ◦Reference 5

Challenge ◦Segmentation is not a well-defined task. 6

Challenge ◦Segmentations have different numbers of segments and are inconsistent. ◦How to estimate the quality of each segmentation algorithm in an unsupervised manner? 34 segments 77 segments 7

Outline ◦Introduction What is segmentation? Challenge Main idea ◦Method ◦Experiment result ◦Conclusion ◦Reference 8

Main idea ◦Combine segmentations of the same image obtained by different algorithms. ◦Average of all the segmentations. ◦The quality of segmentation is based on the consistency of the segmentation compared to the other algorithms. 9

Main idea Average segmentation Input image 10

Outline ◦Introduction ◦Method Probabilistic framework - Definition - EM algorithm Integer Linear Programming Processing Procedure Additional information ◦Experiment result ◦Conclusion ◦Reference 11

Probabilistic framework 12

Probabilistic framework 13

Outline ◦Introduction ◦Method Probabilistic framework - Definition - EM algorithm Integer Linear Programming Processing Procedure Additional information ◦Experiment result ◦Conclusion ◦Reference 14

Probabilistic framework 15

Probabilistic framework 16

Probabilistic framework 17

Outline ◦Introduction ◦Method Probabilistic framework - Definition - EM algorithm Integer Linear Programming Processing Procedure Additional information ◦Experiment result ◦Conclusion ◦Reference 18

Integer Linear Programming 19

Integer Linear Programming Transitive relation If x ij = x jk = 1 then x ik = 1 The complexity of ILP is high. 20

Outline ◦Introduction ◦Method Probabilistic framework - Definition - EM algorithm Integer Linear Programming Processing Procedure Additional information ◦Experiment result ◦Conclusion ◦Reference 21

Processing Procedure G = ( V, E ) with { w ij } 1. Divided into “positively connected components” Negative weight Positive weight 2. Transform to “Single Edge Partition Tree” 3. Divided into subgraphs 22

Processing Procedure G = ( V, E ) with { w ij } 1. Divided into “positively connected components” 2. Transform to “Single Edge Partition Tree” 3. Divided into subgraphs 23

Processing Procedure c (V 1,E 1 ) c (V 2,E 2 ) Crossing edge E 12 G( V, E ) Negative edge G( V, E ) 24

Processing Procedure 1. Divided into “positively connected components” ◦Approach 25

Processing Procedure G = ( V, E ) with { w ij } 1. Divided into “positively connected components” 2. Transform to “Single Edge Partition Tree” 3. Divided into subgraphs 26

Processing Procedure 2. Transform to “Single Edge Partition Tree” ◦Approach Case 1 Cycle-free graph(tree) V1V1 V2V2 V3V3 V4V4 V V1V1 V2V2 V3V3 V4V4 V5V5 27

Processing Procedure 2. Transform to “Single Edge Partition Tree” ◦Approach Case 2 V1V1 V2V2 V3V3 V4V4 V V1V1 V2V2 V3V3 V4V4 28

Processing Procedure 2. Transform to “Single Edge Partition Tree” ◦Approach Case 3 V1V1 V2V2 V3V3 V4V4 V V1V1 V3V3 V4V4 29

Processing Procedure G = ( V, E ) with { w ij } 1. Divided into “positively connected components” 2. Transform to “Single Edge Partition Tree” 3. Divided into subgraphs 30

Processing Procedure 3. Divided into subgraphs V1V1 V2V2 V3V3 V4V4 V5V5 V1V1 V2V2 V3V3 V4V4 V5V5 31

Outline ◦Introduction ◦Method Probabilistic framework - Definition - EM algorithm Integer Linear Programming Processing Procedure Additional information ◦Experiment result ◦Conclusion ◦Reference 32

Additional information ◦Image spatial consistency  Neighboring pixels are more likely to be in the same cluster than pixels that are far apart. ◦Approach  Use mean-shift algorithm to oversegment the image into small, homogeneous regions, known as superpixels.  Merging the MS superpixels, based on consensus among the experts. 33

Averaging Multiple Unreliable Segmentations ( AMUS ) AMUS Averaging Segmentation 34

Averaging Multiple Unreliable Segmentations ( AMUS ) G = ( V, E ) with { w ij } Divided into “positively connected components” Transform to “Single Edge Partition Tree” Divided into subgraphs Use MS to get superpixels Apply ILP to each subgraphs 35

Outline ◦Introduction ◦Method ◦Experiment result AMUS algorithm Compare with other algorithms ◦Conclusion ◦Reference 36

AMUS algorithm Result Averaging segmentation 37

Outline ◦Introduction ◦Method ◦Experiment result AMUS algorithm Compare with other algorithms ◦Conclusion ◦Reference 38

Compare with other algorithms Image AMUS CTM TBES MNC UCM PRI(probabilistic Rand index) VOI(Variation of information ) GCE(Global Consistency Error) Boundary-based F-measure 39

Outline ◦Introduction ◦Method ◦Experiment result AMUS algorithm Compare with other algorithms ◦Conclusion ◦Reference 40

Conclusion ◦Segmentation is not a well-defined task. ◦This paper present a method for combining several segmentations of an image into a single one ( the averaging segmentation ) in order to achieve a more reliable and accurate segmentation result. ◦This paper also reports the reliability of each segmentation. 41

Outline ◦Introduction ◦Method ◦Experiment result AMUS algorithm Compare with other algorithms ◦Conclusion ◦Reference 42

Reference ◦Amir Alush and Jacob Goldberger, “ Ensemble Segmentation Using Efficient Integer Linear Programming ”, IEEE Transactions on PAMI,