1. Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D Non- Rigid Deformable Surface Estimation Using RGBD Data

2.  We propose an algorithm that uses energy minimization to estimate the current configuration of a highly non-rigid object.  Our approach relies on a 3D nonlinear energy minimization framework to solve for the configuration using a semi-implicit scheme adapted from Fua and colleagues (Pilet et al. 2005). Overview

3. Previous Work on Pose Estimation for Robotics Elbrechter et al. (IROS 2011) use a soft-body-physics model with visual tracking to manipulate a piece of paper. Bersch et al. (IROS 2011) describe a method to bring a T-shirt into a desired configuration by alternately grasping the item with two hands, using a fold detection algorithm. Both approaches require predefined fiducial markers.

4. The purpose of this approach is to minimize the energy equation of a mesh model that involves 4 terms: S moothness term Energy Minimization Approach C orrespondence term D epth term B oundary term

5. The purpose of this approach is to minimize the energy equation of a mesh model that involves 4 terms: Energy Minimization Approach

6. Mesh Initialization Energy Minimization Approach

7. S moothness term Energy Minimization Approach

8. S moothness term C orrespondence term D epth term B oundary term

9. S moothness term Energy Minimization Approach

10. S moothness term Energy Minimization Approach

11. C orrespondence term Energy Minimization Approach

12. C orrespondence term Energy Minimization Approach

13. D epth term Energy Minimization Approach Front ViewTop View

14. D epth term Energy Minimization Approach Front ViewTop View

15. B oundary term Without BoundaryWith Boundary Energy Minimization Approach

16. B oundary term Without BoundaryWith Boundary Energy Minimization Approach

17. Minimize energy equation

18. Experimental Results We captured RGBD video sequences of shirts and posters to test our proposed method’s ability to handle different non-rigid objects in a variety of scenarios. Four experiments were conducted: 1)Illustrating the contribution of the depth term 2)Illustrating the contribution of the boundary term 3)Partial self-occlusion 4)Textureless shirt sequence

19. Experimental Results Illustrating the contribution of the depth term

20. Experimental Results Illustrating the contribution of the boundary term

21. Experimental Results Partial self-occlusion

22. Experimental Results Textureless shirt sequence

23. Experimental Results Video

24. Conclusion  We have presented an algorithm to estimate the 3D configuration of a highly non-rigid object through a video sequence using feature point correspondence, depth, and boundary information.  We plan to extend this research to handle a two-sided 3D triangular mesh that covers both the front and the back of the object.

25. Questions?