Columbia University Robotics Group Robotic Grasp Planning Peter Allen Department of Computer Science Columbia University.

Slides:



Advertisements
Similar presentations
Reactive and Potential Field Planners
Advertisements

Kinematic Synthesis of Robotic Manipulators from Task Descriptions June 2003 By: Tarek Sobh, Daniel Toundykov.
Presented by Xinyu Chang
B659: Principles of Intelligent Robot Motion Spring 2013 David Tidd.
Kinematics & Grasping Need to know: Representing mechanism geometry Standard configurations Degrees of freedom Grippers and graspability conditions Goal.
Meshless Elasticity Model and Contact Mechanics-based Verification Technique Rifat Aras 1 Yuzhong Shen 1 Michel Audette 1 Stephane Bordas 2 1 Department.
Iterative Relaxation of Constraints (IRC) Can’t solve originalCan solve relaxed PRMs sample randomly but… start goal C-obst difficult to sample points.
Instructor: Mircea Nicolescu Lecture 13 CS 485 / 685 Computer Vision.
SA-1 Body Scheme Learning Through Self-Perception Jürgen Sturm, Christian Plagemann, Wolfram Burgard.
A Study on Object Grasp with Multifingered Robot Hand Ying LI, Ph.D. Department of Mechanical Engineering Kagoshima University, Japan.
Picking Up the Pieces: Grasp Planning via Decomposition Trees Corey Goldfeder, Peter K. Allen, Claire Lackner, Raphael Pelosoff.
Manipulator Dynamics Amirkabir University of Technology Computer Engineering & Information Technology Department.
Robot Motion Planning Bug 2 Probabilistic Roadmaps Bug 2 Probabilistic Roadmaps.
Single Point of Contact Manipulation of Unknown Objects Stuart Anderson Advisor: Reid Simmons School of Computer Science Carnegie Mellon University.
Planning operation start times for the manufacture of capital products with uncertain processing times and resource constraints D.P. Song, Dr. C.Hicks.
Chapter 5: Path Planning Hadi Moradi. Motivation Need to choose a path for the end effector that avoids collisions and singularities Collisions are easy.
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Constraint-Based Motion Planning using Voronoi Diagrams Maxim Garber and Ming C. Lin Department of Computer.
Copyright © 2006 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Technology Education Copyright © 2006 by The McGraw-Hill Companies,
CS 485/685 Computer Vision Face Recognition Using Principal Components Analysis (PCA) M. Turk, A. Pentland, "Eigenfaces for Recognition", Journal of Cognitive.
Velocities and Static Force
Introduction to BIM BIM Curriculum 01.
Definition of an Industrial Robot
Introduction to Haptic Rendering JPL - Virtual Environments Laboratory California Institute of Technology ( Cagatay Basdogan,
Constraints-based Motion Planning for an Automatic, Flexible Laser Scanning Robotized Platform Th. Borangiu, A. Dogar, A. Dumitrache University Politehnica.
© Manfred Huber Autonomous Robots Robot Path Planning.
Lecture 2: Introduction to Concepts in Robotics
Columbia GraspIt!: A Versatile Simulator for Robotic Grasping Andrew T. Miller Columbia University.
Towards Cognitive Robotics Biointelligence Laboratory School of Computer Science and Engineering Seoul National University Christian.
Intelligent Vision Systems ENT 496 Object Shape Identification and Representation Hema C.R. Lecture 7.
Major objective of this course is: Design and analysis of modern algorithms Different variants Accuracy Efficiency Comparing efficiencies Motivation thinking.
Evolving Virtual Creatures & Evolving 3D Morphology and Behavior by Competition Papers by Karl Sims Presented by Sarah Waziruddin.
M.S. Thesis Defense Jason Anderson Electrical and Computer Engineering Dept. Clemson University.
ME451 Kinematics and Dynamics of Machine Systems Review of Linear Algebra 2.1, 2.2, 2.3 September 06, 2013 Radu Serban University of Wisconsin-Madison.
Hand Posture Subspaces for Dexterous Robotic Grasping Dipartimento di Ingegneria dell’Informazione Università degli Studi di Siena IIT- Genova 24 January.
Objective Read World Uncertainty Analysis CMSC 2003 July Tim Nielsen Scott Sandwith.
A Passive Approach to Sensor Network Localization Rahul Biswas and Sebastian Thrun International Conference on Intelligent Robots and Systems 2004 Presented.
Intelligent Database Systems Lab N.Y.U.S.T. I. M. Externally growing self-organizing maps and its application to database visualization and exploration.
Review: Neural Network Control of Robot Manipulators; Frank L. Lewis; 1996.
Kinematic Redundancy A manipulator may have more DOFs than are necessary to control a desired variable What do you do w/ the extra DOFs? However, even.
Chapter 7. Learning through Imitation and Exploration: Towards Humanoid Robots that Learn from Humans in Creating Brain-like Intelligence. Course: Robots.
1cs426-winter-2008 Notes  Will add references to splines on web page.
ROBOT VISION LABORATORY 김 형 석 Robot Applications-B
Accurate Robot Positioning using Corrective Learning Ram Subramanian ECE 539 Course Project Fall 2003.
Outline: Introduction Solvability Manipulator subspace when n<6
By: David Gelbendorf, Hila Ben-Moshe Supervisor : Alon Zvirin
Autonomous Robots Robot Path Planning (3) © Manfred Huber 2008.
Advanced Games Development Game Physics CO2301 Games Development 1 Week 19.
1cs426-winter-2008 Notes. 2 Kinematics  The study of how things move  Usually boils down to describing the motion of articulated rigid figures Things.
Simulation and Experimental Verification of Model Based Opto-Electronic Automation Drexel University Department of Electrical and Computer Engineering.
Collaborative Grasp Planning with Multiple Object Representations Peter Brook Matei Ciocarlie Kaijen Hsiao.
Zach Miller Computer Sciences Department University of Wisconsin-Madison Supporting the Computation Needs.
Integrated Hands-On Mechanical System Laboratories Arif Sirinterlikci, Ph.D., Professor of Engineering Tony Kerzmann, Ph.D., Assistant Professor of Mechanical.
1 Manipulation by humans and robots CENG782 - Planning of Robotic Manipulation Instructor: Erol Sahin Adapted from slides from..
Toward humanoid manipulation in human-centered environments T. Asfour, P. Azad, N. Vahrenkamp, K. Regenstein, A. Bierbaum, K. Welke, J. Schroder, R. Dillmann.
Introduction to Parametric Curve and Surface Modeling.
Velocity Propagation Between Robot Links 3/4 Instructor: Jacob Rosen Advanced Robotic - MAE 263D - Department of Mechanical & Aerospace Engineering - UCLA.
Physically-Based Motion Synthesis in Computer Graphics
Character Animation Forward and Inverse Kinematics
CENG 789 – Digital Geometry Processing 08- Rigid-Body Alignment
INTRODUCTION TO GEOGRAPHICAL INFORMATION SYSTEM
Andreas Hermann, Felix Mauch, Sebastian Klemm, Arne Roennau
Direct Manipulator Kinematics
Accurate Robot Positioning using Corrective Learning
Dynamical Statistical Shape Priors for Level Set Based Tracking
Robust Belief-based Execution of Manipulation Programs
Objective of This Course
GENERAL VIEW OF KRATOS MULTIPHYSICS
Where did we stop? The Bayes decision rule guarantees an optimal classification… … But it requires the knowledge of P(ci|x) (or p(x|ci) and P(ci)) We.
Retrieval Performance Evaluation - Measures
Chapter 4 . Trajectory planning and Inverse kinematics
Presentation transcript:

Columbia University Robotics Group Robotic Grasp Planning Peter Allen Department of Computer Science Columbia University

Columbia University Robotics Group Robotic grasping is a complex field hand design: high level (number of fingers, kinematic structure, etc.) and low-level (mechanism design, motors, materials, etc.); hand control algorithms: high level (find an appropriate posture for a given task) and low-level (execute the desired posture); information from sensors (tactile, vision, range sensing, etc.); any pre-existing knowledge of objects shape, semantics and tasks (e.g. a cup is likely to be found on a table, should not be held upside-down, etc.); all of these add up to a Grasp Planning System...and more!

Columbia University Robotics Group Human Grasping vs. Robotic Grasping Human performance provides both a benchmark to compare against, and a working example that we can attempt to learn from. However, it has proven very elusive to replicate: the human hand is a very complex piece of equipment, with amazing capabilities; humans benefit from an unmatched combination of visual and tactile sensing; human continuously practice grasping and manipulation, the amount of data they are exposed to dwarfs anything tried so far in robotics; are we setting the bar too high\ Speed stacking!

Robotic Manipulation Process allowing a robot to make physical changes to the world around it. Includes moving objects, joining objects, reshaping objects, etc. Moving objects can be done by grasping, pushing, carrying, dropping, throwing, etc. Task accomplished by a manipulator with some sort of end-effecter.

Grippers vs. Hands Structured environments Reliable Simple Low cost Unstructured environments Adaptable Complex Expensive SuctionParallel JawMagnet Barrett Utah/MITRobonaut

Simulation for Grasp Planning Integrated grasp analysis –Grasp quality, weak point, force optimization Perform many grasps quickly –Faster than using a real arm and hand Build a library of saved grasps –Recall grasp when object is encountered again

GraspIt!: A Tool for Grasping Research Library of hands and objects Intuitive user interface Visualize grasp wrench space Quality measures evaluate grasp Dynamic simulation Grasp Planning

Library of hands and objects Intuitive user interface Visualize grasp wrench space Quality measures evaluate grasp Dynamic simulation Grasp Planning GraspIt: A Tool for Grasping Research

GraspIt Components World Construction read object models read link models read kinematics assemble robots User Interface view 3D scene change hand pose auto-grip manually move joints Matlab interface Contact Determination detect collisions adjust contact to object surface find contact area add friction cones Grasp Analysis compute grasp wrench space use metrics on space grasp force optimization Wrench Space Visualization create projections of GWS worst case disturbance Rigid Body Dynamics compute object motions Grasp Planner generate and test grasps Control Algorithms PD controllers

Hand Construction The hand kinematics (in D-H notation) specify the transforms between links, and can handle coupled joints. The 3D link geometries are accurately described in CAD model files. The format is flexible and can easily model many of the available complex articulated hands.

Hand Kinematics

Hand/Arm Library BarrettRutgersRobonaut Parallel Jaw DLRPumaPaloma

Grasp Analysis Occurs when a contact is formed or broken Computes space of forces and torques that can be applied by the grasp Quality measures numerically evaluate grasp Provides a means to evaluate grasps –Compare grasps of one hand, one object –Compare grasps of many hands, one object –Compare grasps of many hands, across a task specified object set.

Wrench Spaces In 3-space, a wrench is a 6D vector composed of a force and a torque: The space of wrenches that may need to be applied during a task is the task wrench space. The space of wrenches that can be applied by a grasp is the grasp wrench space. A possible quality measure:

Special Types of Grasps A force-closure grasp completely restrains the object. –Origin is contained within grasp wrench space. A manipulable grasp can impart arbitrary velocities on the object without breaking contact.

Friction Cones Friction at a contact point allows forces in directions other than the contact normal COF,  is determined by the contacting materials Estimate friction cone as convex sum of a force vectors on the boundary assuming a unit normal force,.

2D Example Object: square lamina Grasp: 3 point contacts with a COF of 0.3 In 2D, friction cones do not need to be approximated. x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y d max d1d1 f 1,1 fxfx fyfy 

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Contact Wrenches Each force acts at a position on object. Compute the corresponding wrench with respect to object’s center of gravity:  - torque scale factor, i: contact number j: (1-8) around cone boundary x y

Objective: find total space of wrenches that can be applied by a grasp of unit magnitude. Grasp vector: Define with norm: –Sum magnitude of contact normal forces is 1. Compute grasp wrench space using qhull: Grasp Wrench Space

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Two Measures of Quality Assume task wrench space is unknown. Estimate with wrench space ball - good grasps will resist all wrenches equally well. Previously proposed measures of quality: –Radius, , of the largest wrench space ball that can fit within the unit grasp wrench space. –Volume, v, of unit grasp wrench space. Ball radius  : Hull volume v: 0.902

Grasp Wrench Space Projections To visualize the 6D grasp wrench space project it to 3-space by fixing three coordinates. A useful choice: –See the forces that can be applied without applying a net torque (or vice versa).

Visualizing the Results Numeric quality & Worst case indicator Wrench Space Projection

Comparing Grasps

Simulating Dynamics

Columbia University Robotics Group Low-dimensional subspaces can approximate most of the variance needed for common grasping tasks [Santello] PCA on large dataset of human joint angles during grasping 2 PC’s contain approx. 85% of the variance! Continuous grasp subspace approximates common grasp posture Hand Posture Subspaces A grasp can be a considered a point in a high-dimensional hand configuration space To find a grasp, need to search this space: expensive, intractable 20 DOF4 DOF12 DOF14 DOF e.g. p=a 1 e 1 +a 2 e 2

Columbia University Robotics Group Eigengrasps Can be seen as generalization of grasp taxonomy [Napier ‘56, Cutkosky ‘89, Iberall ‘97, etc.] Eigengrasp method: video: video continuous subspacediscrete points, Cutkosky `89

Columbia University Robotics Group Grasp Planning using Eigengrasps Low-dimensionality basis for grasping Derived from human user studies and mapped to robotic hands Practical implications  dimensionality reduction  Completeness: can it cover the space of good grasps? Search eigengrasp subspace for good grasps  use simulated annealing as search algorithm o stochastic nature allows escape from local minima o no analytical gradient is necessary Planner works independent of hand kinematics M. Ciocarlie, C. Goldfeder, P. Allen, Dimensionality reduction for hand-independent dexterous robotic grasping, IROS 2007

Columbia University Robotics Group Grasp planning using Eigengrasps Apply Eigengrasp concept to a range of hand models: From simple gripper (4 DOF) to human hand (20 DOF) For human hand, use available data For robotic hands, attempt to define grasp subspaces similar to the one obtained in the case of the human hand direct transfer possible for anthropomorphic models

Columbia University Robotics Group

Grasp planning using Eigengrasps Energy function formulation attempts to bring pre-specified contact locations on the palm in contact with the object Simulated annealing search is performed over 8 variables o 6 for wrist position / orientation o 2 eigengrasp amplitudes Our implementation: 100K iterations, ~150 seconds

Columbia University Robotics Group

Grasp planning using Eigengrasps k = 1000 k = 5000k = 8000k = 15000k = 20000k = 25000k = k = k = 75000k = 60000k = 55000k = 40000k = 35000k = Each image shows best solution found until iteration k Video

Columbia University Robotics Group Grasp Planning Examples

Columbia University Robotics Group Grasp planning using Eigengrasps

Columbia University Robotics Group Grasp planning using Eigengrasps We need the final posture for stable (form-closure) grasps Completeness: perfect grasp postures can not be found in EG space Can come very close with simple heuristic: close all fingers! Perform form-closure tests in parallel in a multi-threaded environment  takes advantage of multi-core architecture Form-closure in 23 out of 30 cases

Columbia University Robotics Group Interactive Online Grasping Dexterous hand prosthetics  human control combined with artificial hardware and algorithms  benefits from results in both human and robotic grasping  but also open new interface problems! Task: achieve a stable grasp using a dexterous prosthetic  3D tooltip positioning under neural control [Taylor et al. ‘02]  finger posture has proven more difficult to specify o EMG [Zecca et al. ‘02], neural control [Taylor et al. ‘03] o low-dimensional, noisy Approach:  reduce the dimensionality of the searched posture space  use on-line operator input to simplify hand position Matei Ciocarlie and Peter K. Allen, Hand Posture Subspaces for Dexterous Robotic Grasping, The International Journal of Robotics Research, Jul 2009; vol. 28: pp

Columbia University Robotics Group Neural Control of Prosthetic Hands

Columbia University Robotics Group Interactive Dexterous Grasping Operator  specifies approach direction  has no control over finger posture Automated grasp planner  knows object shape and position  records position input  searches for a stable grasp according to user approach direction  sets finger posture accordingly Operator  changes approach direction, or  completes grasp (binary “close all fingers” command)

Columbia University Robotics Group Interactive Grasp Planning Hand posture: 2 variables (eigengrasp amplitudes) Hand position:  user not expected to fully specify final position o affects interaction, can not handle noise  3 variables to re-parameterize hand approach: o d, θ and φ define a conical search space Total: 5 variables  loops of 2000 Simulated Annealing iterations  continuously update base hand position  search does not get stuck if one loop fails  best pre-grasps tested for form-closure

Columbia University Robotics Group Interactive Grasp Planning Examples

Columbia University Robotics Group Interactive Grasp Planning Examples

Columbia University Robotics Group Interactive Grasp Planning Examples Simulated Annealing search: ~1000 postures / s candidate pre-grasp tests: ~20 grasps / s overall: 5-15 seconds for task completion

Columbia University Robotics Group Is Grasping Indexable? Many previous attempts to taxonomize grasps Is there a finite set of grasps we can pre-compute? If so, can we build an indexable database of grasps? Given a new object to grasp, can we find a similar grasp Some Problems: Lots of objects to grasp… Lots of DOF in a hand (~ in human hand)… Lots of different robotic hands… Intractable? But maybe not….

Columbia University Robotics Group The Columbia Grasp Database We have built a large scale grasp database. High quality form closure grasps for:  Multiple hands  Thousands of objects  Hundreds of thousands of grasps We can use the grasp database for:  Data-driven grasp planning  Rigorous benchmarking of grasp planners

Columbia University Robotics Group Finding Stable Grasps Problem: Collecting grasp data from humans is expensive  Human studies do not scale to large datasets  How can we collect data on robotic hands? Solution: Automate the data collection  Use a powerful offline grasp planner as the source  Works for human or robotic hands, inexpensive Caveat: Database quality depends on the quality of this planner Advantages of Eigengrasps  Stochastic, so can give many grasps for one object  Gives form-closure grasps

Columbia University Robotics Group Objects to Grasp We reused the 3D models from the Princeton Shape Benchmark *  Well known academic dataset of 1,814 models  Encourage collaboration with shape researchers  All models resized to “graspable” sizes We provide grasps at 4 scales  …because grasping is scale dependent .75, 1.0, 1.25 and 1.5 times the size of each model  7,256 3D models in all * Shilane et al., SMI 2004

Columbia University Robotics Group Robotic Hands We provide grasps for 4 hands  Human hand model (20 DOF)  Barrett Hand (4 DOF + disengaging clutch)  Barrett Hand with rubber coating  Willow Garage gripper  More hands to come!

Columbia University Robotics Group The Completed Database Building the database  Took 1 month on 6 multicore computers.  Resulted in ~250,000 distinct form closure grasps For each grasp we saved:  Grasp and pre-grasp joint angles and position  Ferrari-Canny volume and epsilon metrics Available for download now!  grasping.cs.columbia.edu

Columbia University Robotics Group Browsing the Grasp Database

Columbia University Robotics Group Data Driven Grasp Planning Given a new 3D model to grasp  Find nearest geometric neighbors in database  Initial implementation with Zernike descriptors  Collect pre-grasps from neighbor models  Evaluate candidates in GraspIt! simulator

Columbia University Robotics Group

CGDB: Benchmarking That seemed to work, but can we quantify it?  Historically – not easily!  Evaluating a grasp planner is very difficult  No consistent benchmarks for comparison Our proposal: Benchmark on this database  Simulate planned grasps on each model  Report aggregate statistics  Easily repeatable, no expensive hardware required  Quality metrics are only possible in simulation

Columbia University Robotics Group Benchmarking We benchmarked two planners  Eigengrasp Planner and Database Planner We benchmarked two hands  Human and Barrett We report the average quality of the first 25 grasps from each planner over 1814 models (single scale)

Columbia University Robotics Group Planner Benchmarks Barrett HandHuman Hand n th best grasp for data-driven & Eigengrasp planners

Columbia University Robotics Group Results Consistent results over both hands For first grasp  Database planner’s quality matches Eigengrasps  But is much faster (not shown in the chart)  1 minute vs minutes For subsequent grasps  Database planner produces many more good grasps

Columbia University Robotics Group Grasping a Sensed Object Data from a NextEngine laser scanner noisy, missing and occluded regions Do geometric match into database, test these grasps Works for human or robotic hands

Columbia University Robotics Group Next: Using partial range scans… Goldfeder, Ciocarlie, Allen, et. al., Data-Driven Grasping with Partial Sensor Data, IROS 2009 Data Driven Grasping Examples

Columbia University Robotics Group GraspIt! Simulation Engine focused on robotic grasping Can quickly test thousands of potential grasps, using arbitrary hand geometries and object models. Includes dynamics, visualization, GWS calculation Easy to add new hand designs Eigengrasps: Reduced subspace for grasping On-line grasp planning: Add user input to eigengrasps to reduce search space further Applied to a number of robotic and human hand models, using both real and simulated environments Columbia Grasp Database: indexed set of grasps across large model and hand space

Columbia University Robotics Group Acknowledgements Andy Miller: Original GraspIt! design and implementation Matei Ciocarlie: Eigengrasps, online planner, soft finger contacts Corey Goldfeder: Columbia Grasp Data Base NSF and NIH