1. Slide 1 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Shape Recovery from Medical Image Data Using Extended Superquadrics Talib Bhabhrawala Advisor : Dr. Venkat Krovi Department of Mechanical and Aerospace Engineering State University of New York at Buffalo Master of Science Thesis Defense December 14 th, 2004

2. Slide 2 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Introduction Background Methodology Results Conclusion Introduction Background Methodology Development Case Studies Interfaces Conclusion & Future Work Overview

3. Slide 3 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Ubiquitous availability of computation and communication infrastructure Introduction Superquadrics Methodology Results Conclusion Introduction Create, manipulate & distribute such data.

4. Slide 4 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Model Based Reconstruction –Building geometric shape models from raw input data –Data reduction, Analysis, Manipulation, Storage Computer Vision & Animation Life sciences Engineering Introduction Superquadrics Methodology Results Conclusion Introduction Point Cloud Data is adequate for Visualization Application Areas –Models & Methods are defined by the final application –Visualization – Surface Geometry –Dynamic & Finite Element Analysis – Volumetric Information

5. Slide 5 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Desired Characteristics Low Order Models –Computational ease. –Fitting, Visualizing & Analysis Parametric Models - Intuitive and Easy to use - Meaningful and repeatable - Great Success in Engineering Models whose nature is approximation - Tractability for infinite dimensional data

6. Slide 6 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo How can we leverage the same framework to additionally parametrically explore multi-resolution hierarchical indexing, storage, searching, reconstruction and retrieval? Introduction Superquadrics Methodology Results Conclusion Research Issues Which kind of a parametric approximation framework would be most suitable for rapid, easy, accurate and computationally inexpensive shape modeling and conversion to volumetric solid model from a dense sampling of the surface?

7. Slide 7 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo - flexible family of parametric objects - using low order parameterization, variety of shapes maybe obtained - simple mathematical representation - good explicit and implicit form Superquadrics (SQ) Introduction Superquadrics Methodology Results Conclusion powerful & compact shape representation

8. Slide 8 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo A 3D surface can be obtained by the spherical product of two 2D curves. When a half circle in a plane orthogonal to the (x, y) plane. is crossed with the full circle in (x, y) plane Introduction Superquadrics Methodology Results Conclusion Spherical Product

9. Slide 9 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo A superellipse is a closed curve defined by Introduction Superquadrics Methodology Results Conclusion Superellipses

10. Slide 10 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Superellipsoids are obtained from superellipses a 1, a 2, a 3 - scaling factors ε 1, ε 2 - relative roundness & squareness. Introduction Superquadrics Methodology Results Conclusion Superquadrics

11. Slide 11 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Introduction Superquadrics Methodology Results Conclusion Superquadrics

12. Slide 12 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Valuable single implicit function. The object is continuous everywhere. Point membership classification can be done Inside–outside function. Introduction Superquadrics Methodology Results Conclusion Implicit Representation

13. Slide 13 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Advantages – can model a diverse set of objects – compact representations – controllability and intuitive meaning – can be recovered from 3D information robustly Introduction Superquadrics Methodology Results Conclusion Limitations – Basic representation can only model symmetrical shapes SQ Discussion

14. Slide 14 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Superquadric’s applications –Computer environments (Montiel, 1997; Pentland, 2000) –Graphics & vision (Chella, 2000; Jacklic, 2000) Local and Global Deformations –Nonlinear deformable models (Solina & Bajcsy, 1991) –Simulating equations of motion (Terzopoulos, 1993) Increasing the DOF –Segmentation (e.g. Löffelman and Gröller, 1994) –blending multiple models (DeCarlo & Metaxas, 1998) –free form deformations (Bardinet et al., 1994) Introduction Superquadrics Methodology Results Conclusion Literature

15. Slide 15 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo To represent more complex shapes there is a trade off between - degrees of freedom & expressive power Zhou and Kambhamettu (2001) first examined - exponents need not be fixed - possibly be spatially varying functions - extended superquadrics Introduction Superquadrics Methodology Results Conclusion Literature

16. Slide 16 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Analogous to a SQ it is defined by & are the latitude & longitude angles Exponents are now functions of these angles. Introduction Superquadrics Methodology Results Conclusion Extended Superquadrics (ESQ)

17. Slide 17 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Measure the difference between a modeled shape and the given data set where Introduction Superquadrics Methodology Results Conclusion Inside-Outside Function

18. Slide 18 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo The shape of the exponent functions have to be controllable We introduce a spline as the exponent function. This interpolated curve acts like a look up table for the algorithm Introduction Superquadrics Methodology Results Conclusion Exponent Functions

19. Slide 19 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Introduction Superquadrics Methodology Results Conclusion Intuitive Example

20. Slide 20 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Problem Statement Recovering a superquadric model from a set of 3D points – Superquadric model – Vector of superquadric parameters – Input Points – Minimize Least square distance between SQ surface & data points

21. Slide 21 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo SQ in a local coordinate system SQ in the general position Transform the points to the object coordinated system Introduction Superquadrics Methodology Results Conclusion Five parameters which define the size & shape Initial Model Definition

22. Slide 22 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Applying the Inverse Transform & using Euler angles Additional six parameters which define the position and orientation Case of Extended Superquadrics Exponent is a spline interpolating ‘p’ control points increases the total number of parameters by 2(p-1). Number of parameters are now 9+2(p-1). Introduction Superquadrics Methodology Results Conclusion Initial Model Definition

23. Slide 23 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Farthest range point along each coordinate axis which gives an estimate of a 1, a 2 & a 3 Object recognition and pose estimation Obtain the rotation matrix and eigen vectors. Orient axes along minimal & maximal moment of inertia Introduction Superquadrics Methodology Results Conclusion Moment Based Estimation

24. Slide 24 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo The error-of-fit function is define using the inside–outside function EOF varies quickly where the exponents are large and slowly where exponents are small Introduction Superquadrics Methodology Results Conclusion Added to remove the bias Error of Fit Function

25. Slide 25 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Ambiguity in Description –A set of exponent functions in conjunction with scaling parameters can generate the same shape as another set To solve the ambiguity, the minimum volume constraint is added Introduction Superquadrics Methodology Results Conclusion Error of Fit Function Metric to be minimized for the “fitting”

26. Slide 26 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Minimize where Variables Introduction Superquadrics Methodology Results Conclusion Optimization Problem

27. Slide 27 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo The conventional method used is the Levenberg-Marquardt algorithm Fast and accurate Problems of local minima Heavily dependent on initial estimates Introduction Superquadrics Methodology Results Conclusion Choice of Optimization Method

28. Slide 28 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Inspired by Biological Evolution and its principles The evolution of life on earth can be regarded as one long optimization process though it’s up to debate if this process has reached a optimum yet… Introduction Superquadrics Methodology Results Conclusion Genetic Algorithms

29. Slide 29 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Salient Features Requires little insight into the problem Ideal if a problem is non convex or has a very large multimodal solution space Heuristics Based, does not require derivatives Provides with “Good” Solutions Ideal exploratory tool to examine new approaches Genetic Algorithms

30. Slide 30 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Genetic Algorithms Components of a GA Population size: 20 – 100 Crossovers: 50-60% Mutations: < 5% Generations: 20 – 2000 - Encoding technique (double vector, binary) - Object function (environment) - Genetic operators (selection, mutation, crossover) “Typical” tuning parameters initialize population; evaluate population; while TerminationCriteriaNotSatisfied { select parents for reproduction; perform crossover & mutation; evaluate population; }

31. Slide 31 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Introduction Superquadrics Methodology Results Conclusion Shape Recovery Algorithm

32. Slide 32 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Introduction Superquadrics Methodology Results Conclusion 2D Case Study

33. Slide 33 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Introduction Superquadrics Methodology Results Conclusion 3D Case Study

34. Slide 34 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Difficult to fit a complex model using a single extended superquadric Segment an object into primitives Maximum Error Partitioning Line d minimize Two Superquadrics to approximate the data EOF1= 0.575 EOF2= 0.326 Introduction Superquadrics Methodology Results Conclusion Iterative Segmentation & Recovery

35. Slide 35 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Introduction Superquadrics Methodology Results Conclusion Volume Segmentation Using ESQ 2D contours are obtained and are stacked Topological Accuracy is high Loses Compact representation Laborious process & model has inconsistencies Requires a post-processing step

36. Slide 36 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Introduction Superquadrics Methodology Results Conclusion PC Based Interface

37. Slide 37 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Introduction Superquadrics Methodology Results Conclusion Web Based Interface

38. Slide 38 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Flexible enough for an asymmetric object that deform smoothly on spheres Variable coefficients of the continuous exponents offer a compact parameter space and broad coverage The descriptive parameterization is directly incorporated into the model formulation Introduction Superquadrics Methodology Results Conclusion Conclusion

39. Slide 39 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo A more intuitive and robust segmentation scheme Techniques for creating “tailored” models from such simple general purpose models More intelligent precursor steps to improve convergence speed of the algorithm Systematic way to extract and store characteristic signatures of shape Introduction Superquadrics Methodology Results Conclusion Future Work

40. Slide 40 of 40 Automation, Robotics and Mechatronics Lab, SUNY at Buffalo Thank You! Acknowledgments: Dr. V. Krovi, Dr. C. Bloebaum & Dr. A. Patra