Planning for Deformable Parts

Holding Deformable Parts How can we plan holding of deformable objects?

Deformable parts “Form closure” does not apply: Can always avoid collisions by deforming the part.

Deformation Space: A Generalization of Configuration Space. Based on Finite Element Mesh. D-Space

Deformable Polygonal parts: Mesh Planar Part represented as Planar Mesh. Mesh = nodes + edges + Triangular elements. N nodes Polygonal boundary.

D-Space A Deformation: Position of each mesh node. D-space: Space of all mesh deformations. Each node has 2 DOF. D-Space: 2N- dimensional Euclidean Space. 30-dimensional D- space

Nominal mesh configuration Deformed mesh configuration Deformations Deformations (mesh configurations) specified as list of translational DOFs of each mesh node. Mesh rotation also represented by node displacements. Nominal mesh configuration (undeformed mesh): q 0. General mesh configuration: q. q0q0 q

D-Space: Example Simple example: 3-noded mesh, 2 fixed. D-Space: 2- dimensional Euclidean Space. D-Space shows moving node’s position. x y Physical space D-Space q0q0

Topological Constraints: D T x y Physical space D-Space Mesh topology maintained. Non-degenerate triangles only. DTDT

Topology violating deformation Undeformed part Allowed deformation Self Collisions and D T

D-Obstacles x y Physical space D-Space Collision of any mesh triangle with an object. Physical obstacle A i has an image DA i in D-Space. A1A1 DA 1

D-Space: Example Physical space x y D-Space

Potential Energy Needed to Escape from a Stable Equilibrium Consider: Stable equilibrium q A, Equilibrium q B. Capture Region: K(q A )  D free, such that any configuration in K(q A ) returns to q A. q A qBqB q U(q) K( q A )

U A (q A ) = Increase in Potential Energy needed to escape from q A. = minimum external work needed to escape from q A. U A : Measure of “Deform Closure Grasp Quality” q A qBqB q U(q) UAUA Potential Energy Needed to Escape from a Stable Equilibrium K( q A )

Deform Closure qAqA qBqB q U(q)

MOTION PLANNING FOR DEFORMABLE OBJECTS From slides by Ilknur Kaynar-Kabul

Introduction  Algorithms so far  the world was assumed to be made of rigid objects  Why deformable objects?  Deformable moving objects (wires, metal sheets)  Deformable obstacles (e.g., human-body tissue structures)  Need for physical model

First Paper: Planning paths for elastic objects under manipulation constraints (Lamiraux and Kavraki) Energy model

Second Paper: Probabilistic Roadmap Motion Planning for Deformable Objects (O. Burchan Bayazit, Jyh-Ming Lien, Nancy M.Amato)

Planning Paths for Elastic Objects Under Manipulation Constraints Florent Lamiraux Lydia E. Kavraki Rice University Int. J. Robotics Research, 2001.

Introduction Goal: Plan paths for elastically deformable objects in a static environment What is hard? Representing the shape of an object with a possibly infinitely dimensional configuration space Computing object shapes under actuator loading conditions Collision checking for a shape-changing object

Problem Definition What objects are considered? Elastically deformable objects constrained by two actuators Shape is determined by the lowest energy state for a given configuration of the actuators Only the actuators are responsible for deformations (object cannot touch obstacles) ACTUATORS constrain the position of a subset of the points of the object

Problem Definition In general, the configuration of an elastic object can be infinite-dimensional and cannot be represented by a vector Configuration Rest configuration q 0 Configuration q corresponds to mapping object through deformation VoVo VqVq Configuration q 0 Configuration q

Problem Definition Manipulation Constraints Actuators constrain a subset of points V 0 p in V 0 Denote M as set of possible actuator positions and m is one of these positions in M For all x in V 0 p there is a mapping X m from V 0 to V q For a given position m of the actuators, points V 0 p are moved to X m (V 0 p ) The position of the other points of the objects should be such that the elastic energy of the object is minimized.

Problem Definition Stable Equilibrium Configurations Motion is slow enough to consider quasi-static paths – only stable equilibrium configurations Stable equilibrium configurations are shapes at which the elastic energy is minimized Minimum EnergyCannot form this with two actuators

Problem Definition Elastically Admissible Configurations Elastic materials have a range of elastic deformation, large deformations may exceed this range and permanently deform A range of elastic e(x) is defined for each point x Admissible configurations are those in which e(x) is within the elastic range for all x in V 0

Problem Definition In path planning, “collision-free paths” are not enough – other conditions must be met Manipulability: every point along the path must meet the actuator constraints Quasi-static equilibrium: every point along the path must be in stable equilibrium (a minimum energy shape) Elastic admissibility: no points along the path exceed the elastic limits of the material

Path Planning Algorithm Algorithm PRM approach is used, similar to conventional planners Initial/Final configurations are chosen Random free stable equilibrium configurations are chosen as nodes in roadmap Nodes are connected by a local planner to form edges Decompose deformation and position of object to save computing time

Path Planning Algorithm Algorithm The following steps are repeated until q init and q goal are in the same connected component of the roadmap: Node generation Node connection Enhancement

Path Planning Algorithm Node Generation A random manipulator position is chosen and minimum energy shape calculated and admissibility is checked

Path Planning Algorithm Node Generation Random rigid-body motions are evaluated for collision-free configurations Rigid body motion is applied

Path Planning Algorithm Node Generation Random rigid-body motions are evaluated for collision-free configurations collision

Path Planning Algorithm Node Generation N collision-free configurations are found for the same deformation

Path Planning Algorithm Node Connection Each newly generated node is tested for connection with its K closest neighbors Distance function is evaluated

Path Planning Algorithm Node Connection Distance function should account for rigid body transformation and deformation Distance total = distance transformation + distance deformation

Path Planning Algorithm Node Connection Connections are performed by a deterministic local planner that generates quasi-static paths between pairs of configurations. Edges

Path Planning Algorithm Node Connection Local planner checks for collisions and admissibility Not a valid edge Violates elasticity limits Not a valid edge There exist collision

Path Planning Algorithm Enhancement Under the assumption that unconnected nodes are in difficult parts of the configuration space, add more nodes in these difficult areas

Path Planning Algorithm Enhancement Randomly walk away from unconnected nodes in the same configuration for a certain distance, reflecting off obstacles A total of M enhancement nodes are added

Path Planning Algorithm Path finding for a given q init and q goal A graph search can yield a sequence of edges leading from q init and q goal Concatenation of local paths results in a global path We look for a path that minimizes the number of distinct deformations of the nodes of V belonging to the path -> reduce unnecessary deformations

Path Planning Algorithm Distance Metric Distance d(p,q) = d d (p,q) + d r (p,q) d d is deformation distance, defined as the maximum distance a point moves in the local frame during a deformation d r is rigid body translation and rotation distance, defined as the Euclidean distance in R 6

Path Planning Algorithm Collision Checking With the decoupled motions, a standard collision checking algorithm can be applied, the research in this paper used RAPID (OBB-trees) By keeping deformation separate from position, deformations can be stored and reused speeding up collision checking

Experimental Results Bending Plate 7 Dimensional problem 6 for placement 1 for deformation

Experimental Results Bending Plate The actuators are along the 2 opposite long edges They are always parallel Actuators constrain the distance d <= L between the opposite edges Thus, deformation is one dimensional

Experimental Results Bending Plate N = 200 K = 40 M = 100 Avg run time – 22.7 min Avg # nodes – 12,500

Experimental Results Bending Plate 9 Dimensional problem 6 for placement 3 for deformation

Experimental Results Bending Plate Manipulation constraints specify both the position and tangent direction of 2 opposite edges of the plate One end of the curve is fixed and the other can move freely (translation along x1 and x3, rotation about x2)

Experimental Results Bending Plate N = 200 K = 40 M = 100 Avg run time – 4 hrs 12 min Avg # nodes – 33,600

Experimental Results Bending Plate Avg run time – 4 hrs 12 min Space of deformation is of higher dimension Large number of minimizations involved for computing deformation paths The free space inside the box is constrained

Experimental Results Elastic Pipe – one end fixed 5 Dimensional problem All 5 dimensions for deformation

Experimental Results Elastic Pipe – one end fixed Manipulation constraints specify both the position and tangent direction of the ends of the pipe. No twisting of pipe

Experimental Results Elastic Pipe – one end fixed Nodes = 200 K = 40 M = 0 Avg run time – 14.2

MOVIES

Conclusions Very general algorithm, easily tunable Improved energy minimization would be very helpful Many representations from graphics do not conserve surface area or volume

Disadvantages and Future Work Planner is not suitable for real-time use expensive operations for solving mechanical models generating collision detection data structures Has so far only been applied to simple objects, such as a sheet of metal or a pipe-like robot

Probabilistic Roadmap Motion Planning for Deformable Objects O. Burchan Bayazit Jyh-Ming Lien Nancy M. Amato ICRA2002

Outline Introduction Problem Definition Roadmap construction Shrink factor Penetration depth Query processing Bounding box deformation Geometric deformation Experimental Results Conclusions

Algorithm Motion planning for deformable objects Framework is based on a PRM model Difference from typical motion planning The robot is allowed to change its shape (deform) to avoid collisions as it moves along the path

2 Stage Algorithm An approximate path is found for the rigid version of the robot Might contain collisions Collisions on this path are corrected by deforming the robot

Overview of algorithm 1.Build a feasible roadmap 2.while (a valid path is not found) 3. find a feasible path 4. foreach path configuration 5. if (there is collision) 6. deform robot 7. if (not deformable) 8. remove path segment from roadmap 9.endwhile

Roadmap

If there exist collision -> Deform Robot

If robot cannot be deformed

If robot cannot be deformed -> Delete edge

Roadmap Construction Roadmap may include some colliding configurations Weight the edges to denote the expected difficulty of deforming that edge Weights are selected to denote the expected deformation energy required to deform the edge Computing deformation energy is difficult

Roadmap Construction 2 strategies for constructing the roadmap Use reduced-scale version of the rigid robot until it is collision-free Use the amount of penetration by the original robot and check it is less than some acceptable bound In both cases goal is to build a roadmap containing paths that are Already valid OR Can be made valid by deforming the robot

Shrinkable Robots ‘shrink’ factor required to obtain a free robot in a particular configuration is some indication of deformation energy robot

Shrinkable Robots Can be computed relatively inexpensively Pre-compute a set of scaled models Construct by scaling all vertices and maintaining the robot topology robot

Shrinkable Robots robot Scaled models are used for collision detection in node generation and connection

Node generation and construction For each node Begin with the largest robot and progressively reduce its scale until a collision free robot size is found The roadmap is connected as with traditional PRM. An edge weight is set as the sum of the shrink factors of the two scaled models that are the edge’s endpoints. A larger(smaller) robot model has a smaller(larger) shrink factor

A roadmap generated with shrinkable robots Roadmap configurations are shown in their accepted shrink size with wired surfaces

Penetration Another estimate of deformation energy The deeper the robot penetrates inside an obstacle the harder it will be to deform the robot into a collision free shape Unfortunately, penetration depth computation is hard to compute

Penetration C-space penetration depth is used Allowable Penetration depth

Penetration If any of vectors is collision free, then accept configuration Allowable Penetration depth

Node generation and connection Collision detection is replaced by a test for allowable penetration Advantage: we can use a standard PRM, including all node generation and connection methods

A roadmap generated with penetrating robots

QUERY Rigid body -> roadmap construction Deformable body -> query processing The query process must check for collision If collisions are found, use a deformation method to try to deform the offending configurations into collision-free configurations

PATH QUERY The swept volume of a path found Although the shortest path is on the right side of ball, the planner selected a longer path which had a smaller deformation energy

DEFORMATION Goal is to produce realistic looking deformations fast Not physically complete Precise computation of energy is not done

DEFORMATION 2 objects participate in deformation Deformable object Deformer The deformer pushes (a portion of) the deformable object towards a collision-free state Deformable object changes shape according to these external forces

BOUNDING BOX DEFORMATION Deforms the given model hierarchically First, the bounding box of the model is deformed, and then the model itself is deformed according to the deformation computed for the bounding box

BOUNDING BOX DEFORMATION Combination of Chainmail deformation and Free-form deformation is used First, the bounding box of the model is deformed by Chainmail deformation Then, this model is deformed by FFD according to shape of its bounding box

GEOMETRIC DEFORMATION Continuously moves the colliding polygons of the robot until they are outside the obstacle Which direction to move? Sampling strategy to identify the correct direction 1.Normals of colliding polygons on the robot 2.Normals of the colliding polygons on the obstacle

GEOMETRIC DEFORMATION

Problems If the robot does not fit in a narrow passage, then this method will fail to find a pushing direction unless we use a reduced scale version of the robot Can produce unrealistic deformations because the resulting polygons may be quite large Apply constraints Smooth the polygons

MOVIES