Collaborative Grasp Planning with Multiple Object Representations Peter Brook Matei Ciocarlie Kaijen Hsiao

Uncertainty in Object Perception Sensed Scene Real Scene Collaborative Grasp Planning with Multiple Object Representations 2

Uncertainty in Object Perception Sensed Scene Misdetected Object Collaborative Grasp Planning with Multiple Object Representations 3

Different Object Representations Collaborative Grasp Planning with Multiple Object Representations 4 Known object grasps precomputed using GraspIt! Minimum value bounding box decomposition [Huebner et. al., ICRA 2008] Novel object grasps from point clouds [Hsiao et al., IROS 2010] Grasp Planning via Decomposition Trees [Goldfeder et. al., ICRA 2007] Many more

Why Choose Just One? One object – multiple representations Pool grasps from all representations Evaluate grasps on all representations Combine evaluations for each grasp based on representation confidence A good grasp on the real object should be likely to work on all likely representations Collaborative Grasp Planning with Multiple Object Representations 5

Predicting Grasp Success Collaborative Grasp Planning with Multiple Object Representations 6 A good grasp on the real object should be likely to work on all likely representations

ROS Grasping Pipeline Collaborative Grasp Planning with Multiple Object Representations 7

Our Implementation 8

Grasp Evaluation on Object Representations Evaluate grasp on all representations For point cluster grasps:  Heuristic quality measures For database objects:  Evaluate in simulation (expensive)  Use data-driven regression Collaborative Grasp Planning with Multiple Object Representations 9

Combining Object Representations Collaborative Grasp Planning with Multiple Object Representations 10

Object DetectionGraspIt! energy metric Collaborative Grasp Planning with Multiple Object Representations 11 Based on 892 point clouds of 44 objects Based on 490 recorded grasps of 30 objects Models

Simulation experiments Collaborative Grasp Planning with Multiple Object Representations 12

Simulation experiments Single object on table Point cloud recorded from stereo camera with labeled ground truth object pose We know mapping from simulation score to real success Evaluate grasps in simulation on true object Collaborative Grasp Planning with Multiple Object Representations 13

Simulation Results Collaborative Grasp Planning with Multiple Object Representations 14 How does collaborative planning compare to the existing planning method on known and unknown objects?

PR2 Results Object Category Collaborative Planner Naïve Planner Novel22/2518/25 Database22/2521/25 Collaborative Grasp Planning with Multiple Object Representations 15 Single object on table Success = lift and move to side without dropping

Conclusion and Future Work Framework for integrating different grasping algorithms and representations Encourages detection algorithms to expose their uncertainty Combining models allows robust grasping Collaborative Grasp Planning with Multiple Object Representations 16 Add more!  Representations  Grasp generation and evaluation algorithms Bayesian Grasp Planner Feedback to detection algorithms Feedback to high-level decisions about grasping vs. exploring