1. 3-D Computer Vision CSc 83020 – Ioannis Stamos 3-D Computer Vision CSc 83020 Clustering methods and boundary representations

2. 3-D Computer Vision CSc 83020 – Ioannis Stamos Image Segmentation Generate clusters (regions) of pixels that correspond to meaningful entities. Generate clusters (regions) of pixels that correspond to meaningful entities. Use metrics of “closeness” between values. Use metrics of “closeness” between values. Use algorithms for combining “close” values. Use algorithms for combining “close” values. Apply other constraints (connectivity). Apply other constraints (connectivity).

3. 3-D Computer Vision CSc 83020 – Ioannis Stamos Example (Forsyth & Ponce)

4. 3-D Computer Vision CSc 83020 – Ioannis Stamos Simple Clustering Methods Divisive clustering Divisive clustering Everything is a big cluster at beginning Everything is a big cluster at beginning Split recursively Split recursively Agglomerative clustering Agglomerative clustering Each data (pixel) is a cluster Each data (pixel) is a cluster Merge Merge

5. 3-D Computer Vision CSc 83020 – Ioannis Stamos Simple clustering

6. 3-D Computer Vision CSc 83020 – Ioannis Stamos Simple clustering What is a good inter-cluster distance? What is a good inter-cluster distance? How many clusters are there? How many clusters are there?

7. 3-D Computer Vision CSc 83020 – Ioannis Stamos Simple clustering

8. 3-D Computer Vision CSc 83020 – Ioannis Stamos Clustering by K-means Assume that the number of clusters (k) is known. Assume that the number of clusters (k) is known. Each cluster has a center C i (i=1..k) Each cluster has a center C i (i=1..k) Each data-point is a vector x j (j=1..Number of pixels) Each data-point is a vector x j (j=1..Number of pixels) Examples: Examples: xj=[x-coord, y-coord, gray-value] xj=[x-coord, y-coord, gray-value] or xj=[gray-value] or xj=[red-value, green-value, blue-value] Assume that elements are close to center of clusters. Assume that elements are close to center of clusters. Minimize: Minimize:

9. 3-D Computer Vision CSc 83020 – Ioannis Stamos Clustering by K-means Iterative algorithm: Iterative algorithm: Allocate each point to center of closest cluster (assuming centers are known) Allocate each point to center of closest cluster (assuming centers are known) Calculate centers of clusters (assuming allocations are known) Calculate centers of clusters (assuming allocations are known) How do we start? How do we start?

10. 3-D Computer Vision CSc 83020 – Ioannis Stamos Clustering by K-means

11. 3-D Computer Vision CSc 83020 – Ioannis Stamos Example (Forsyth & Ponce)

12. 3-D Computer Vision CSc 83020 – Ioannis Stamos Example (Forsyth & Ponce)

13. 3-D Computer Vision CSc 83020 – Ioannis Stamos Example (Forsyth & Ponce)

14. 3-D Computer Vision CSc 83020 – Ioannis Stamos RANSAC RANdom SAmple Consensus RANdom SAmple Consensus Model fitting method Model fitting method Line-fitting example Line-fitting example Fitting a line to a set of edges with 50% outliers Fitting a line to a set of edges with 50% outliers Least squares would fail Least squares would fail Solution: M-estimator or RANSAC Solution: M-estimator or RANSAC

15. 3-D Computer Vision CSc 83020 – Ioannis Stamos RANSAC (line-fitting example) Two edges (wout normal) define a line. Two edges (wout normal) define a line. General idea: General idea: Pick two points. Pick two points. Write the equation of the line. Write the equation of the line. Check how many other points are “close” to line. Check how many other points are “close” to line. If number of “close” points is above threshold, done If number of “close” points is above threshold, done Otherwise, pick two new points. Otherwise, pick two new points. Questions: Questions: Which points to pick? Which points to pick? Complexity in worst case? Complexity in worst case?

16. 3-D Computer Vision CSc 83020 – Ioannis Stamos RANSAC (line-fitting example)

17. 3-D Computer Vision CSc 83020 – Ioannis Stamos RANSAC (line-fitting example) How large should k (max. number of iterations) be?

18. 3-D Computer Vision CSc 83020 – Ioannis Stamos RANSAC (line-fitting example) How large should k (max. number of iterations) be? Assume that w is the probability of picking a “correct” point (i.e. a point on the line). Assume that w is the probability of picking a “correct” point (i.e. a point on the line). Since we are picking n (=2 for lines) points, 1-w n is the probability of picking n “wrong” points. Since we are picking n (=2 for lines) points, 1-w n is the probability of picking n “wrong” points. If we iterate k times we want the probability of failure to be small: i.e. If we iterate k times we want the probability of failure to be small: i.e. (1-w n ) k = z => k = log(z)/log(1-w n ) (1-w n ) k = z => k = log(z)/log(1-w n ) If z=0.1 and w=0.5 then k=8 (n=2) If z=0.1 and w=0.5 then k=8 (n=2) If z=0.01 and w=0.5 then k=16 (n=2) If z=0.01 and w=0.5 then k=16 (n=2) If z=0.001 and w=0.1 then k = 687 (n=2) If z=0.001 and w=0.1 then k = 687 (n=2) How is the formula affected by n? How is the formula affected by n?

19. 3-D Computer Vision CSc 83020 – Ioannis Stamos RANSAC (Conclusions) When can this method be successful? When can this method be successful? Can we detect circles? Can we detect circles? In that case how many points do you need to fit a circle? In that case how many points do you need to fit a circle? Can we detect other shapes? Can we detect other shapes?

20. 3-D Computer Vision CSc 83020 – Ioannis Stamos Boundary representation of regions

21. 3-D Computer Vision CSc 83020 – Ioannis Stamos Representation of 2-D Geometric Structures To MATCH image boundary/region with MODEL boundary/region, they must represented in the same manner. Boundary Representation Snakes – Extraction of arbitrary contours from image. Region Representation

22. 3-D Computer Vision CSc 83020 – Ioannis Stamos Representation Issues Compact: Easy to Store & Match. Compact: Easy to Store & Match. Easy to manipulate & compute properties. Easy to manipulate & compute properties. Captures Object/Model shape. Captures Object/Model shape. Computationally efficient. Computationally efficient.

23. 3-D Computer Vision CSc 83020 – Ioannis Stamos Boundary Representation Polylines: concatenation of line segments. Breakpoint Matching on the basis of: # of line segments lengths of line segments angle between consecutive segments

24. 3-D Computer Vision CSc 83020 – Ioannis Stamos Running Least Squares Method eiei Move along the boundary At each point find line that fits previous points (Least Squares) Compute the fit error E=Sum(e i ) using previous points If E exceeds threshold, declare breakpoint and start a new running line fit.

25. 3-D Computer Vision CSc 83020 – Ioannis Stamos Approximating Curves with Polylines Draw Straight line between end-points of curve For every curve point find distance to line. If distance is less than tolerance level for all points, Exit Else, pick point that is farthest away and use as breakpoint. Introduce new segments. Recursively apply algorithm to new segments.

26. 3-D Computer Vision CSc 83020 – Ioannis Stamos Ψ-s Curve s ψ x y s Ψ 12 3 4 5 6 x y 1 2 3 4 5 6 π 2π

27. 3-D Computer Vision CSc 83020 – Ioannis Stamos Ψ-s Curve s ψ x y s Ψ 12 3 4 5 6 x y 1 2 3 4 5 6 Ψ-s is periodic (2π) Horizontal section in Ψ-s curve => straight line in the Image Non horizontal section in Ψ-s => arc in Image π 2π

28. 3-D Computer Vision CSc 83020 – Ioannis Stamos Slope-Density Function s ψ x yH(Ψ) Ψ π2π HISTOGRAM Lines Arcs

29. 3-D Computer Vision CSc 83020 – Ioannis Stamos Slope-Density Function s ψ x yH(Ψ) Ψ π2π HISTOGRAM H(Ψ) shifts as objects rotates. H(Ψ) wraps around Lines Arcs

30. Fourier Descriptors s ψ x y Find: Ψ(s) Define: Φ(s)= Ψ(s)-(2πs)/P P: Perimeter. 2π: Period of Φ(s) Φ(s) is a Continuous, Periodic function. Fourier Series for Periodic Functions: Fourier Coefficients: Φ k ’s capture shape information Match shapes by matching Φ k ’s Use finite number of Φ k ’s

31. Fourier Descriptors Input Shape Power Spectrum Reconstruction # of coefficients ΦkΦk Note: Reconstructed shapes are often not closed since only a finite # of Φ k ’s are used.

32. B-Splines Piecewise continuous polynomials used to INTERPOLATE between Data Points. Smooth, Flexible, Accurate. s=0 s=1 s=2 x0x0 x1x1 x2x2 xixi x s Spline X(s) Data Point We want to find X(s) from points xi Cubic Polynomials are popular: BASIS FUNCTIONS COEFFICIENTS N

33. B-Splines Bi(s) has limited support (4 spans) i-2i-1ii+1 i+2 Each span (i->i+1) has only 4 non-zero Basis Functions: B i-1 (s), B i (s), B i+1 (s), B i+2 (s) ii+1 B i (s) B i-1 (s) B i+2 (s) B i+1 (s) t=0 t=1 t s:

34. 3-D Computer Vision CSc 83020 – Ioannis Stamos B-Splines ii+1 B i (s) B i-1 (s)B i+2 (s) B i+1 (s) t=0 t=1 t Cubic Polynomials 1/6 4/6

35. 3-D Computer Vision CSc 83020 – Ioannis Stamos B-Splines ii+1 B i (s) B i-1 (s)B i+2 (s) B i+1 (s) t=0 t=1 t Cubic Polynomials 1/6 4/6

36. 3-D Computer Vision CSc 83020 – Ioannis Stamos B-Splines If we compute v 0, … v N => continuous representation for the curve.

37. B-Splines We have our N+1 data points: And in matrix form: We can solve for the v i ’s from the x i ’s Boundary condition for closed curves: v 0 =v N, v 1 =v N+1.

38. B-Splines So, for any i we can find: xixi x i+1 x i(t) t t=0 t=1 Note: Local support. Spline passes through all data points xi. B-Spline demo: http://www.doc.ic.ac.uk/~dfg/AndysSplineTutorial/BSplines.html

39. 3-D Computer Vision CSc 83020 – Ioannis StamosSnakes Elastic band of arbitrary shape. Located near the image contour.

40. 3-D Computer Vision CSc 83020 – Ioannis Stamos Snakes – Implementation How can we minimize  Discrete version: sum over the points. Greedy minimization: For each point compute the best move over a small area.

41. 3-D Computer Vision CSc 83020 – Ioannis Stamos Snakes – Implementation How can we minimize  Discrete version: sum over the points. Greedy minimization: For each point compute the best move over a small area. Set β i =0 for points of high curvature (corners)

42. 3-D Computer Vision CSc 83020 – Ioannis Stamos Snakes – Implementation How can we minimize  Discrete version: sum over the points. Greedy minimization: For each point compute the best move over a small area. Set βi=0 for points of high curvature (corners) Stop when a user-specified fraction of points does not move.

43. 3-D Computer Vision CSc 83020 – Ioannis Stamos Synthetic Results Real Experiment Snakes demo: http://www.markschulze.net/snakes/

44. 3-D Computer Vision CSc 83020 – Ioannis Stamos Region Representation 1 1 11111 11111 1111 1 Spatial Occupancy Array Easy to implement. Large Storage area. Can apply set operations (unite/intersect). Expensive for matching!

45. 3-D Computer Vision CSc 83020 – Ioannis Stamos Quad Trees Efficient encoding of Spatial Occupancy Array using Resolution Pyramids. Black: Fully Occupied. White: Empty. Gray: Partially Occupied.

46. 3-D Computer Vision CSc 83020 – Ioannis Stamos Quad Tree Level 0 Level 1 Level 2 Level 3 Quad Tree Generation: Start with level 0. If Black or White, Terminate. Else declare Gray node & expand with four sons. For each son repeat above step. NWNE SWSE