1. Qualifying Exam: Contour Grouping Vida Movahedi Supervisor: James Elder Supervisory Committee: Minas Spetsakis, Jeff Edmonds York University Summer 2009

2. Contents Introduction Preliminary Concepts –Pre-processing –Gestalt cues Methods –Local & Heuristic –Local & Probabilistic –Global Saliency Evaluation Conclusion & open problems

3. Contents Introduction Preliminary Concepts –Pre-processing –Gestalt cues Methods –Local & Heuristic –Local & Probabilistic –Global Saliency Evaluation Conclusion & open problems

4. Introduction Segmentation Partition an image into regions, each corresponding to an object or entity Figure-Ground segmentation

5. Segmentation Methods Regional Segmentation –Use regional info, optimize labelling of regional tokens, e.g. clustering –Depending on uniformity in object region Active Contour Models –Use regional (external) & boundary (internal) info, optimize deformation of model –Sensitivity to initialization, too smooth Contour Grouping –Use boundary info (& regional info), optimize grouping of contour fragments

6. Problem Definition Input: Color image Goal:Figure-ground segmentation Method: Contour Grouping Other available info: None - No motion, stereo or video information - No user interactions - No assumptions on object types, shapes, color, etc. - No assumptions on background or lighting conditions

7. Challenges High-dimensional data space, lots of information, many cues Unknown cue integration Global optimization in a non-convex multidimensional space Camera, imaging, quantization noise Clutter in natural scenes Occluded or overlapping objects

8. Contents Introduction Preliminary Concepts –Pre-processing –Gestalt cues Methods –Local & Heuristic –Local & Probabilistic –Global Saliency Evaluation Conclusion & open problems

9. Steps Image Grouping Algorithm Saliency Computations Optimization Algorithm Figure/Ground Segmentation Pre-processing Edge Detection Line /Curve Approximation Learned Parameters or Distributions

10. Pre-processing Image  Edge Map  Line Map  Contour

11. Gestalt Cues How is grouping done in human vision? Proximity Similarity –Brightness –Contrast Good continuation –Parallelism –Co-circularity

12. Contents Introduction Preliminary Concepts –Pre-processing –Gestalt cues Methods –Local & Heuristic –Local & Probabilistic –Global Saliency Evaluation Conclusion & open problems

13. Grouping Methods Local Heuristic methods –Defining a heuristic cost for contour hypotheses, find the optimal one Local Probabilistic methods –Find posterior probability of contour hypotheses given cues, find the optimal one Global methods –An extra step of calculating global saliencies based on local measures

14. Contents Introduction Preliminary Concepts –Pre-processing –Gestalt cues Methods –Local & Heuristic –Local & Probabilistic –Global Saliency Evaluation Conclusion & open problems

15. Local & Heuristic Example: Ratio Contour Method (Wang et. al, PAMI’05) Detected/ virtual fragments Contour cost= curvature & gap per unit length Graph model Alternate cycle

16. Local & Heuristic Example: Ratio Contour Method (Wang et. al, PAMI’05) Edge/ Link costs Ratio Contour Algorithm

17. Sample Results for RC method ImageRC RRC RC Image (from Stahl & Wang, TIP’07) (from Wang et al., PAMI’05)

18. Contents Introduction Preliminary Concepts –Pre-processing –Gestalt cues Methods –Local & Heuristic –Local & Probabilistic –Global Saliency Evaluation Conclusion & open problems

19. Local & Probabilistic (Elder et al., PAMI’03) Bayesian Rule: Contour saliency= posterior probability of contour Assumptions: –Markov Chain Assumption –Independence of evidence from cues –Comparing contours of same length

20. Local & Probabilistic (Elder et al., PAMI’03) Graph Model Node weight Link weight Shortest path/cycle Approximate search

21. Sample Results for Probabilistic Methods (from Estrada & Elder- CVPRW’06)

22. Contents Introduction Preliminary Concepts –Pre-processing –Gestalt cues Methods –Local & Heuristic –Local & Probabilistic –Global Saliency Evaluation Conclusion & open problems

23. Global Model Local weights Global weights

24. Global Saliency Edge/Link Affinity Based on collinearity, proximity, etc. Edge/ Link Saliency Relative number of closed random walks which visit that edge/link (Mahamud et al., PAMI’03) Shown to be relevant to the eigenvalues and eigenvectors of the affinity matrix Grouping based on global saliency

25. Some Results of the Untangling method (from Zhu; Song; Shi- ICCV’07)

26. Contents Introduction Preliminary Concepts –Pre-processing –Gestalt cues Methods –Local & Heuristic –Local & Probabilistic –Global Saliency Evaluation Conclusion & open problems

27. Evaluation Empirical discrepancy methods The output of algorithms is compared with a reference segmentation or ground truth Requirements – A ground truth dataset – An error measure

28. SOD: Salient Object Dataset Based on Berkeley Segmentation Dataset (BSD) 300 images, randomly showing 818 segmentations (half of BSD) to each of 7 subjects 12,110 object boundaries obtained

29. Region-based Error Measures Example Not sensitive to some large shape features (e.g., spikes, wiggles)

30. Boundary-based Error Measures Not sensitive to object topology and some large shape features (e.g., loop-backs, wiggles)

31. Mixed Error Measures Example Not sensitive to some large shape features. Does not respect ordering along contours. p j, j=1..N fn are pixels in the false negative region (R B -R A ) q k, k=1..N fp are pixels in the false positive region (R A -R B )

32. Contour Mapping Measure Based upon a matching between all points on the two boundaries Monotonically non-decreasing Allowing one-to-one, many-to- one, and one-to-many matching Error= average distance between matched pairs Dynamic Programming Contour Mapping Distance=7.73

33. Contents Introduction Preliminary Concepts –Pre-processing –Gestalt cues Methods –Local & Heuristic –Local & Probabilistic –Global Saliency Evaluation Conclusion & open problems

34. Conclusion & Open Problems Cue selection and combination Grouping Model –Global saliency –Probabilistic models Optimization Algorithms Hierarchical and multi-scale algorithms Quantitative evaluation