1. Superquadric Recovery in Range Images via Region Growing influenced by Boundary Information Master-Thesis Christian Cea Bastidas

2. Master Thesis Contents Motivation and Objectives Superquadric and Rim Overview of the Proposed Solution Superquadric Fitting and Rim Fitting Proposed and Alternative Solution Evaluation Methodology and Comparison Summary

3. Christian Cea Bastidas Master Thesis Motivation 1. may be modeled with a low fitting error, using a type of surface called Superquadric 2. are not covered in the image by another object To solve the Segmentation and Recovery Problem which consists in extracting from a 3D image the objects that:

4. Christian Cea Bastidas Master Thesis Objectives 1. To develop an algorithm which solves the stated problem by completing the solution of the existing algorithm Seed Generation. 2. To compare the improved solution with that of the well known approach Recover-and- Select Segmentation ( Leonardis, 1990 ).

5. Christian Cea Bastidas Master Thesis Contents Motivation and Objectives Superquadric and Rim Overview of the Proposed Solution Superquadric Fitting and Rim Fitting Proposed and Alternative Solution Evaluation Methodology and Comparison Summary

6. Christian Cea Bastidas Master Thesis Superquadric : Modeling Element Superquadrics, a generalization of the quadric, were chosen as Modeling Object because: 1. They possess a simple mathematical formulation 2. The presence of superquadric-like objects is recurrent in many applications. 3. Its representation capacity can be easily incremented by means of Deformations.

7. Christian Cea Bastidas Master Thesis Superquadric : Definition Parametric representation Observation : Superellipsoids, a special type of Superquadric has been considered, which are closed and connected.

8. Christian Cea Bastidas Master Thesis Superquadric : Examples (I) The number of edges increases as distance themselves from 1.

9. Christian Cea Bastidas Master Thesis Superquadric : Transformations Euclidean Transform Global Deformations ( Bending and Tapering ) 6 new parameters => Superquadric needs 11 parameters

10. Christian Cea Bastidas Master Thesis Superquadric : Examples (II) A cylinder along its circular and parabolic deformations

11. Christian Cea Bastidas Master Thesis Rim : Definition The 3D points in a range image are collected by a laser sensor located on a certain plane. The normal to this plane corresponds to the Viewing Point. Assumption : The distance between the laser sensor and the objects is supposed to be large

12. Christian Cea Bastidas Master Thesis Rim : Example (I) The rims have been drawn for the objects in the image. Viewing point is (0,1,0) ( Axis Y )

13. Christian Cea Bastidas Master Thesis Rim : Superquadric Rim A parametric representation of the rim is derived from a more operative definition : => Rim equation Important Property : It permits to sample the rim efficiently !

14. Christian Cea Bastidas Master Thesis Rim : Example (II) Rim of a superquadric in general position Viewing Point = (0,1,0)

15. Christian Cea Bastidas Master Thesis Contents Motivation and Objectives Superquadric and Rim Overview of the Proposed Solution Superquadric Fitting and Rim Fitting Proposed and Alternative Solution Evaluation Methodology and Comparison Summary

16. Christian Cea Bastidas Master Thesis Solution Part 1 : Seed Generation Input : Range Image Output : - Seeds - Edge Map Seed : Set of points which belong with high probability to a single object Edge Map : Points on the rims and edges ( All sets are Undistinguishable ! ) Seed Generation + Edge Detection

17. Christian Cea Bastidas Master Thesis Solution Part 2 : Region Growing(1) Output : Recovered Objects Input : - Seeds - Edge Map Region Growing Key Idea : Alternate fitting of the superquadrics to the range image with the fitting of the superquadric rims to the edge map.

18. Christian Cea Bastidas Master Thesis Solution Part 2 : Region Growing(2) Superquadric Fitting Rim Fitting Rim Filter Superquadric is fitted to O Rim is fitted to the Edge Map using the rim of as first estimate O* : set of points in the range image whose projection on the plane XZ is inside the Rim Projection O ( A Seed ) O* ( Recovered Object ) Stop? O = O* Yes No If O* ~ O or Big Error Fitting the Stop!

19. Christian Cea Bastidas Master Thesis Contents Motivation and Objectives Superquadric and Rim Overview of the Proposed Solution Superquadric Fitting and Rim Fitting Proposed and Alternative Solution Evaluation Methodology and Comparison Summary

20. Christian Cea Bastidas Master Thesis SQ Fitting : The Problem (I) Restriction : The points in S come from the visible part of the object ( Self-Occlusion ) The problem can be stated as follows:

21. Christian Cea Bastidas Master Thesis SQ Fitting : The Problem (II) Minimization Problem : Preliminary Formulation Self-Occlusion →

22. Christian Cea Bastidas Master Thesis SQ Fitting : The Problem (II) Minimization Problem : Preliminary Formulation Self-Occlusion →

23. Christian Cea Bastidas Master Thesis SQ Fitting : Objective Functions 3 Alternatives for the function : Standard Euclidean Distance (SED) - There does not exist closed mathematical formula - S does not contain necessarily the closest point to an arbitrary point because of the Self-Occlusion. ← Unfeasible Radial Euclidean Distance (RED) - It has a closed mathematical formula - A good approximation to SED. Modified Algebraic Distance (MED) ← Selected - Closed mathematical formula and simple derivatives - Broadly used in the literature

24. Christian Cea Bastidas Master Thesis SQ Fitting : SED and RED But SED and RED are more intuitive error measures SED RED

25. Christian Cea Bastidas Master Thesis SQ Fitting : The Problem (III) Definitive Formulation (Solina and Bajcsy ) Using a modified algebraic distance for : where and reverse the effect of deformations and euclidean transformation respectively.

26. Christian Cea Bastidas Master Thesis SQ Fitting : Type of Problem The formulated problem : For this kind of problem, Levenberg-Marquardt Algorithm is specially suitable. Corresponds to a Nonlinear Least Squares Minimization:

27. Christian Cea Bastidas Master Thesis SQ Fitting : LM Algorithm Iterative Procedure defined by : holds for Nonlinear Least Squares Minimization Requisites : 1. The initial estimate => An Ellipsoide ( Rosenfeld and Kak ) 2. The derivatives in order to evaluate and

28. Christian Cea Bastidas Master Thesis SQ Fitting : Examples - The original points ( in pink ) lies on the visible part of the object - Rounded objects are more easily fitted than objects with edges.

29. Christian Cea Bastidas Master Thesis Rim Fitting : The Problem (I) There exist 2 mayor differences respect to the SQ Fitting : 1. The real rim of an object cannot be isolated from the Edge Map => Objective Function = Sum of the distances from each point in a sampling of the SQ Rim to the Edge Map. Important Assumptions : - Edge Map contains enough points for each rim - The points on the rim sampling must be uniformly distributed.

30. Christian Cea Bastidas Master Thesis Rim Fitting : The Problem (I) There exist 2 mayor differences respect to the SQ Fitting : 1. The real rim of a object cannot be isolated from the Edge Map => Objective Function = Sum of the distances from each point in a sampling of the SQ Rim to the Edge Map. Important Assumptions : - Edge Map contains enough points for each rim - The points on the rim sampling must be uniformly distributed. 2. The fitting is done in 2D: Rim Sampling and Edge Map are projected onto the plane XZ before computing the distances. Reasons : - Efficiency - Rim Filter needs only the Rim Projection

31. Christian Cea Bastidas Master Thesis Rim Fitting : The Problem (II) Mathematical Formulation

32. Christian Cea Bastidas Master Thesis Rim Fitting : Examples The estimate comes from a Superquadric fitting a small region => stays far from the real Rim The estimate comes from a Superquadric fitting a big region => evolves nearly into the real Rim Case 1 Case 2

33. Christian Cea Bastidas Master Thesis Contents Motivation and Objectives Superquadric and Rim Overview of the Proposed Solution Superquadric Fitting and Rim Fitting Proposed and Alternative Solution Evaluation Methodology and Comparison Summary

34. Christian Cea Bastidas Master Thesis Region Growing : Algorithm (1) Stop Criterion Parameters

35. Christian Cea Bastidas Master Thesis Region Growing : Algorithm (2) Output

36. Christian Cea Bastidas Master Thesis Recover-and-Select Segmentation: Part 1: Seed Generation and Expansion 1. Partition the image into n x n regions 2. Fit a superquadric to each region 3. Add new points to a region if they are well approximated by the associated superquadric 4. Go to 2. until no more suitable points are available Seed Generation + Region Expansion

37. Christian Cea Bastidas Master Thesis Recover-and-Select Segmentation: Part 2: Model Selection Model Selection A subset M’ of the generated models M is selected by solving a Binary Quadratic Programming Problem : The idea is to minimize the information quantity I needed to represent the image : Information I = Bits for SQ parameters + Bits for Error Fitting + Bits for Free Points m : decision binary vector

38. Christian Cea Bastidas Master Thesis Contents Motivation and Objectives Superquadric and Rim Overview of the Proposed Solution Superquadric Fitting and Rim Fitting Proposed and Alternative Solution Evaluation Methodology and Comparison Summary

39. Christian Cea Bastidas Master Thesis Evaluation Methodology : Reference Solution The Ideal Solution has 2 parts, one related to the Segmentation and the second one to the Modelling : 1. The objects are segmented manually from the image and their points are stored as sets 2. For each object, the superquadric with the best fitting is found. Thus the set constitutes the second part of the solution. As the best fitting cannot be guarranteed, then the Modelling part is replaced by the from the SQ Fitting. The Segmentation part continues being the ideal one.

40. Christian Cea Bastidas Master Thesis Evaluation Methodology : Indexes 1. Each object O in the solution is matched manually with an object O* in the reference solution. 2. Then the Solution Quality is evaluated in three aspects : Modelling Segmentation Time belongs to the solution and is the superquadric fitted to O. : Convex hull of the projection of the set onto the plane XZ 3. Finally each index is averaged over the objects exposed in the image weighting with the size of each set O ( |O| )

41. Christian Cea Bastidas Master Thesis Algorithm Comparison : Images The difficulty in solving the recovery problem for an image depends on : 1. Number of Objects (No) 2. Average size of the Objects (Size) [ small, medium, large] 3. Shape of the Objects (Shape) [rounded, box-like, mixed] 4. Percentage exposed objects or closely exposed (%E.O.) The algorithms were tested using 8 images with the following characteristics :

42. Christian Cea Bastidas Master Thesis Algorithm Comparison : Index 1 The superquadrics from Alg 1 model better the objects than Alg 2. The exception is the Image 4. For images 5, 6, 7 and 8 the models of Alg 1 are nearly as good as those of the reference solution.

43. Christian Cea Bastidas Master Thesis Algorithm Comparison : Image 4 The seed is completely contained in on one face of the box => A seed should always contain points on “key sectors” of an object

44. Christian Cea Bastidas Master Thesis Algorithm Comparison: Index 2 In general, Alg 1 excels in segmenting, except for image 8. Even for the image 8, this index is still good for Alg 1.

45. Christian Cea Bastidas Master Thesis Algorithm Comparison : Image 8 The rim did not reach the bottom edges of the object.

46. Christian Cea Bastidas Master Thesis Algorithm Comparison : Index 4 Average Recovery Time (ART) Algorithm 1 : 35 sec. Algorithm 2 : 300 sec. Only in one case Alg 2 was faster than Alg 1.

47. Christian Cea Bastidas Master Thesis Algorithm Comparison : Index 4 Image 6 is a important case because : - Complexity of the Image - The seeds are not so big Average Recovery Time = 52 sec.

48. Christian Cea Bastidas Master Thesis Summary 1. The proposed solution showed a best performance both in efficacy and in efficiency in comparison with the classical approach.

49. Christian Cea Bastidas Master Thesis Summary 1. The proposed solution showed a best performance both in efficacy and in efficiency in comparison with the classical approach. 2. The objective function used for the SQ fitting could be improved considering algebraic approximations to the Standard Euclidean Distance.

50. Christian Cea Bastidas Master Thesis Summary 1. The proposed solution showed a best performance both in efficacy and in efficiency in comparison with the classical approach. 2. The objective function used for the SQ fitting could be improved considering algebraic approximations to the Standard Euclidean Distance. 3. The parameterization and sampling of the rim played a key role in the solution.

51. Christian Cea Bastidas Master Thesis Summary 4. In the Rim Fitting the model error is measured only in a two-dimensional subspace. But it is feasible to compute this error in the original space.

52. Christian Cea Bastidas Master Thesis Summary 4. In the Rim Fitting the model error is measured only in a two-dimensional subspace. But it is feasible to compute this error in the original space. 5. The performance of the proposed algorithm depends strongly on the edge map quality and to what extent the seeds contain points on key zones of the objects.

53. Christian Cea Bastidas Master Thesis References

54. Christian Cea Bastidas Master Thesis Superquadric and Rim : Sampling Goal : To generate uniformly distributed points on the surface or curve. Applications : - Plotting and Visualization - Computation of the Closest Point from a given point to the SQ or Rim ( Used in Optimization Problems ) Mechanism : - A SQ can be obtained by multiplying 2 superellipses - The rim can be obtained by multiplying 1 superellipse and 1 point. A Superellipse is a 2D curve which is easier to sample

55. Christian Cea Bastidas Master Thesis Superellipse : Definition

56. Christian Cea Bastidas Master Thesis Superellipse : Examples (I) Superellipses with a=3 and b=9 for different values of ε.

57. Christian Cea Bastidas Master Thesis Superellipse : Sampling (I) Problem : If ө is uniformly sampled then the resulting points on the superellipse are not uniformly distributed. Two sampling mechanisms were tested: Equal-Distance Sampling (Pilu and Fisher) - The superellipse is approximated using 2 models which are easily parameterizable. - Better distribution, but it returns fewer points than the required number. Angle-Center Parameterization (Löffelmann and Gröller) - The superellipse is represented in polar coordinates (r,ω) and ω is uniformly sampled - It returns exactly the required number of points

58. Christian Cea Bastidas Master Thesis Superellipse : Sampling (II) The first mechanism showed the smaller interdistance deviation.

59. Christian Cea Bastidas Master Thesis Superellipse : Sampling (III) The ratio ρ between the required number of points and the obtained was fitted with a 2° polynom in є,b/a. Then the required number of points is adjusted with :

60. Christian Cea Bastidas Master Thesis SE : Sampling Comparison (I) If ө is uniformly sampled then the resulting points are not uniformly distributed.

61. Christian Cea Bastidas Master Thesis SE : Sampling Comparison (II) Equal-Distance Sampling produces uniformly distributed points

62. Christian Cea Bastidas Master Thesis SE : Sampling Comparison (III) Angle-Center Parameterization produces points not satisfactorily distributed

63. Christian Cea Bastidas Master Thesis SE : Superquadric Sampling (I) A uniformly distributed sampling for the superquadric is obtained by making the spherical product between the samplings of the superellipses and Observation:

64. Christian Cea Bastidas Master Thesis SE : Superquadric Sampling (II) Superquadric Sampling based on Equal-Distance Sampling

65. Christian Cea Bastidas Master Thesis is derived from the Rim Equation, = 0.001, R is the rotation matrix and is a constant 1. If The point and the superellipse 2. If The superellipse and the point SE : Rim Sampling (I) A nearly uniformly distributed sampling for the rim can be obtained as the spherical product between:

66. Christian Cea Bastidas Master Thesis SE : Rim Sampling (II) The sampling quality is acceptable The sampling quality is low

67. Christian Cea Bastidas Master Thesis SE : Rim Sampling (III) The problems appear when and they are solved using special parameterizations for the concerned rims The figures show the attained results after applying the new parameterizations for the special rims.

68. Christian Cea Bastidas Master Thesis Rim Anomalies In the figure the discarded points appear in red