1. Image Segmentation Seminar III Xiaofeng Fan

2. Today ’ s Presentation Problem Definition Problem Definition Approach Approach Segmentation Methods Segmentation Methods Applications Applications Question? Question?

3. Definition What is Image Segmentation What is Image Segmentation Image segmentation is a partitioning of an image into related sections or regions. Category: Category: According the goal: Complete segmentation Complete segmentation Partial Segmentation Partial Segmentation

4. Approaches to Image Segmentation Most image segmentation approaches can be placed in one of three classes: Characteristic feature thresholding or clustering; Characteristic feature thresholding or clustering; Boundary detection; Boundary detection; Region based algorithm. Region based algorithm.

5. Region-based algorithm Straightforward region growing Straightforward region growing Hybrid region growing and spectral clustering approach Hybrid region growing and spectral clustering approach

6. The two ways to segmentation Recursive Hierarchical Image Segmentation Recursive Hierarchical Image Segmentation Swendsen-Wang algorithm Swendsen-Wang algorithm Swendsen-Wang algorithm Swendsen-Wang algorithm

7. HYBRID OF REGION GROWING AND SPECTRAL CLUSTERING APPROACH A high-level outline of the new hybrid image segmentation approach is as follows: Label each image pixel as a separate region and set the global criterion value, critval, equal to zero. Label each image pixel as a separate region and set the global criterion value, critval, equal to zero. Calculate the dissimilarity criterion value between each spatially adjacent region. Calculate the dissimilarity criterion value between each spatially adjacent region. Dissimilarity Criterion : For two regions i and j, characterized by the mean vectors Xi = (x1i, x2i,..., xci)T and Xj = (x1j, x2j,..., xcj)T, the Euclidean Spectral Distance (ESD) is defined as: Find the smallest dissimilarity criterion value, and merge all pairs of spatially adjacent regions with this criterion value. Find the smallest dissimilarity criterion value, and merge all pairs of spatially adjacent regions with this criterion value. Calculate the dissimilarity criterion value between all pairs of non-spatially adjacent regions. Calculate the dissimilarity criterion value between all pairs of non-spatially adjacent regions. Merge all pairs of non-spatially adjacent regions with dissimilarity criterion value less than or equal to the criterion value found in step 3. Merge all pairs of non-spatially adjacent regions with dissimilarity criterion value less than or equal to the criterion value found in step 3.

8. HYBRID OF REGION GROWING AND SPECTRAL CLUSTERING APPROACH (con) If the number of regions remaining is less than the preset value minregions, go to step 7. Otherwise go to step 2. If the number of regions remaining is less than the preset value minregions, go to step 7. Otherwise go to step 2. Let prevcritval = critval. Reset critval to be the current global criterion value. If prevcritval = zero, go to step 2. Otherwise calculate cvratio = critval/prevcritval. If cvratio is greater than the preset threshold convfact, save the region label map from the previous iteration as a "raw" segmentation result. If the number of regions remaining is two or less, save the region label map from the current iteration as the coarsest instance of the final hierarchical segmentation result, and go to step 8. Otherwise go to step 2. Let prevcritval = critval. Reset critval to be the current global criterion value. If prevcritval = zero, go to step 2. Otherwise calculate cvratio = critval/prevcritval. If cvratio is greater than the preset threshold convfact, save the region label map from the previous iteration as a "raw" segmentation result. If the number of regions remaining is two or less, save the region label map from the current iteration as the coarsest instance of the final hierarchical segmentation result, and go to step 8. Otherwise go to step 2. Calculate the global criterion value over each region separately. For the region with maximum global criterion value, search backwards from the last "raw" segmentation result (from step 7) to an iteration where this region is split into two or more regions. Replace the labeling for this region with the labeling from the found iteration and store as the next more detailed level of the final hierarchical segmentation. Repeat this step until the preset number of levels in the final hierarchical segmentation is obtained. Calculate the global criterion value over each region separately. For the region with maximum global criterion value, search backwards from the last "raw" segmentation result (from step 7) to an iteration where this region is split into two or more regions. Replace the labeling for this region with the labeling from the found iteration and store as the next more detailed level of the final hierarchical segmentation. Repeat this step until the preset number of levels in the final hierarchical segmentation is obtained.

9. Improvement: Step 4 requires the calculation of the dissimilarity criterion value between each region and every other region in the image. Divide-and-conquer method is devised to reduce the computational load. Step 4 requires the calculation of the dissimilarity criterion value between each region and every other region in the image. Divide-and-conquer method is devised to reduce the computational load. The divide-and-and conquer algorithm: The divide-and-and conquer algorithm: 1. Specify the number of levels of recursion required (nblevels), pad the image size to even; 2. If level > 1, divide the image data into quarters. Otherwise go to step 4; 3. Run step 2 to step 1 completely processing, reassemble the image segmentation results; 4. Execute the image segmentation algorithm with the following modification: If level < nblevels, terminate the algorithm when the number of regions reaches minregions/2, and do not check for critval or output any "raw" segmentation results.

10. Swendsen-Wang algorithm Swendsen-Wang algorithm Basic character: 1.Given an adjacency graph, algorithm compute a local probability at each edge for how likely the two vertices (image elements) belong to the same pattern. 2.At each step, the algorithm splits, merges, or regroups a connected component 3.The algorithm “ mixes ” rapidly at low components density.

11. The Swendsen-Wang Cuts Algorithm 1. Initialize a graph partition 2. Repeat, for current state A Swendsen-Wang Cuts: SWC-1 Input: G o =, discriminative probabilities q e, e  E o, and generative posterior probability p(W|I). Output: Samples W~p(W|I). 3. Repeat for each subgraph G l =, l=1,2,...,n in A 4. For e  E l turn e=“on” with probability q e. 5. Partition G l into n l connected components: g li =, i=1,...,n l 6. Collect all the connected components in CP={V li : l=1,...,n, i=1,...,n l }. 7. Select a connected component V 0  CP at random 8. Propose to reassign V 0 to a subgraph G l’, l' follows a probability q(l'|V 0,A,G o ) 9. Accept the move with probability α(A  B) or α(A  C). State A CP

12. The Swendsen-Wang Cuts Algorithm State A State C State B A CB

13. State AState B Swendsen-Wang Cuts: Main Theorem Theorem. Accepting the proposed move with probability results in an ergodic and reversible Markov Chain.

14. The Swendsen-Wang Cuts Algorithm The Swendsen-Wang Cuts Algorithm Advantages: ergodic and reversible in the space of graph partitions achieves fast mixing at low temperature - eliminates the slow simulated annealing procedure converges in 3-30 seconds in a PC for image segmentation - about 400 times faster than the classical Gibbs sampler - 20-40 times faster than the DDMCMC algorithm. the algorithm can optimize over the number of models works for general forms of posterior probabilities. - more general than the existing graph cut approaches.

15. Example:

16. Thank you Question?