Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield IROS 2012 Vila Moura, Algarve An Energy Minimization Approach to 3D Non- Rigid Deformable Surface Estimation Using RGBD Data

 We propose an algorithm that uses energy minimization to estimate the current configuration of a non-rigid object.  Our approach relies on a 3D nonlinear energy minimization framework to solve for the configuration using a semi-implicit scheme. Overview

Previous Related Work on Pose Estimation Elbrechter et al. (IROS 2011) compare a purely mathematical representation of the paper manifold with a soft-body-physics model and demonstrate the use of their visual tracking method. Bersch et al. (IROS 2011) describe a method to compute valid grasp poses on the cloth which accounts for deformability.

The purpose of this approach is to minimize the energy equation of a mesh model that involves 4 terms: Smoothness term data Correspondence term Depth term Boundary term Energy Minimization Approach

The purpose of this approach is to minimize the energy equation of a mesh model that involves 4 terms: Energy Minimization Approach

Mesh Initialization Energy Minimization Approach

Smoothness term Energy Minimization Approach

Smoothness term Energy Minimization Approach

data Correspondence term Energy Minimization Approach

Depth term Energy Minimization Approach Front ViewTop View

Boundary term Without BoundaryWith Boundary Energy Minimization Approach

Experimental Results We captured RGBD video sequences of shirts and posters to test our proposed method’s ability to handle different non-rigid objects in a variety of scenarios. Four experiments were conducted: 1)Illustrating the contribution of the depth term 2)Illustrating the contribution of the boundary term 3)Partial self-occlusion 4)Textureless shirt sequence

Experimental Results Illustrating the contribution of the depth term

Experimental Results Illustrating the contribution of the boundary term

Experimental Results Partial self-occlusion

Experimental Results Textureless shirt sequence

Conclusion  We have presented an algorithm to estimate the 3D configuration of a non-rigid object through a video sequence using feature point correspondence, depth, and boundary information.  The next step is to integrate this algorithm into a robotic system that can grasp and handle non-rigid objects in an unstructured environment.

Questions?