1. Robot Operating System (ROS) Framework
Automated Robotic Picking in Unstructured Environments Robot Perception, Learning and Planning
Prof. Hao Zhang
Human-Centered Robotics Laboratory
Background
Approach
Human-Centered Robotics
The use of robotic systems in human-social environments to help people live safer, comfortable and more independent lives
Enabling robots with capabilities to live among us and help take over tasks where our current society has shortcomings
Robot Operating System (ROS) Framework
3D model and synthetic point clouds
Object Dataset
Object Recognition
Grasp Planning
Error
Detection
and
Recovery
Motion Planning
Drop item in order bin
Item List
2D scenes from wrist cameras
3D scenes from RGB-D cameras
Search and rescue
Daily life assistance
Daily Life Assistance
Automated Picking in warehouses
(Golem Krang, 2012)
(Care-O-Bot 2011)
(Amazon Picking Challenge, 2015)
Challenges
Complex, dynamic, unstructured multi-human social environment
Real-time requirements under computational constraints
Object Recognition
Detect and classify the items using 3D and 2D robot perception [1,2]
Train an exemplar Support Vector Machine (SVM) classifier to detect and localize the items using the BigBIRD object dataset [3]
Grasping Planning
Compute initial grasping points using Deep Grasping Method [4]
Refine grasping points using observations from multiple perspectives
Motion Planning
Pre-compute trajectory from a start position to the center of each shelf bin
Apply MoveIt! [5] to plan a short trajectory between the bin center and the grasping point
Error Detection and Recovery
Confirm if grasping is successful, else try grasping again
Motivation
Amazon Picking Challenge
Currently, Amazon’ automated warehouses are successful at moving and searching for items within the warehouse
However, item picking and sorting are still manually performed
Automated picking in unstructured environments still remains a difficult challenge
Items to be picked from the shelf
Overview
Given an order list of items, pick the items from a shelf and place them in an order bin
Task must be completed autonomously without any human involvement whatsoever
References
Contact
“The OpenCV Library”, accessed on 03/14/15.
R. B. Rusu and S. Cousins, "3D is here: Point Cloud Library (PCL)," in IEEE International Conference on Robotics and Automation, 2011.
Singh, J. Sha, K. S. Narayan, T. Achim, P. Abbeel, “BigBIRD: A large-scale 3D database of object instances,” in IEEE International Conference on Robotics and Automation, 2014.
I. Lenz, H. Lee, A. Saxena, “Deep Learning for Detecting Robotic Grasps”, in International Journal of Robotics Research, to appear, 2015
“MoveIt!”, accessed on 03/14/15.
Hao Zhang, Ph.D.
Assistant Professor
Dept. of Elec. Engr. & Comp. Sci.
Colorado School of Mines
Phone: (303) HCRobotics Lab: 
Can we make a robot autonomously pick the items in this scenario?
Baxter robot and workspace setup