Robot Operating System (ROS) Framework Automated Robotic Picking in Unstructured Environments Robot Perception, Learning and Planning Prof. Hao Zhang Human-Centered Robotics Laboratory Background Approach Human-Centered Robotics The use of robotic systems in human-social environments to help people live safer, comfortable and more independent lives Enabling robots with capabilities to live among us and help take over tasks where our current society has shortcomings Robot Operating System (ROS) Framework 3D model and synthetic point clouds Object Dataset Object Recognition Grasp Planning Error Detection and Recovery Motion Planning Drop item in order bin Item List 2D scenes from wrist cameras 3D scenes from RGB-D cameras Search and rescue Daily life assistance Daily Life Assistance Automated Picking in warehouses (Golem Krang, 2012) (Care-O-Bot 2011) (Amazon Picking Challenge, 2015) Challenges Complex, dynamic, unstructured multi-human social environment Real-time requirements under computational constraints Object Recognition Detect and classify the items using 3D and 2D robot perception [1,2] Train an exemplar Support Vector Machine (SVM) classifier to detect and localize the items using the BigBIRD object dataset [3] Grasping Planning Compute initial grasping points using Deep Grasping Method [4] Refine grasping points using observations from multiple perspectives Motion Planning Pre-compute trajectory from a start position to the center of each shelf bin Apply MoveIt! [5] to plan a short trajectory between the bin center and the grasping point Error Detection and Recovery Confirm if grasping is successful, else try grasping again Motivation Amazon Picking Challenge Currently, Amazon’ automated warehouses are successful at moving and searching for items within the warehouse However, item picking and sorting are still manually performed Automated picking in unstructured environments still remains a difficult challenge Items to be picked from the shelf Overview Given an order list of items, pick the items from a shelf and place them in an order bin Task must be completed autonomously without any human involvement whatsoever References Contact “The OpenCV Library”, http://opencv.org, accessed on 03/14/15. R. B. Rusu and S. Cousins, "3D is here: Point Cloud Library (PCL)," in IEEE International Conference on Robotics and Automation, 2011. Singh, J. Sha, K. S. Narayan, T. Achim, P. Abbeel, “BigBIRD: A large-scale 3D database of object instances,” in IEEE International Conference on Robotics and Automation, 2014. I. Lenz, H. Lee, A. Saxena, “Deep Learning for Detecting Robotic Grasps”, in International Journal of Robotics Research, to appear, 2015 “MoveIt!”, http://moveit.ros.org, accessed on 03/14/15. Hao Zhang, Ph.D. Assistant Professor Dept. of Elec. Engr. & Comp. Sci. Colorado School of Mines Phone: (303) 273-3581 Email: hzhang@mines.edu HCRobotics Lab: http://hcr.mines.edu Can we make a robot autonomously pick the items in this scenario? Baxter robot and workspace setup