Developing Artificial Intelligence in Robotics Viktoria Golobev & Alina Marchenko Advisor: Prof. Ronen Brafman

Teaching Robots To use the elevator What is it good for? Help disabled people. Delivering mail and packages. Buying grocery. Guiding people in hospitals.

Steps and challenges Detecting the button Calculating path to the button Pushing the button Going back to driving position Recognizing the elevator’s arrival and entering the elevator.

Robot Operating System (Ros) ROS is a trending robot application development platform that provides various features such as message passing, distributed computing, code reusing, and so on. The platform is based on nodes - processes that perform computation using ROS client libraries. The ROS message-passing middleware allows communicating between different nodes through topics. All the communication passes through ROS Master, which is much like a DNS server. The master provides name registration and lookup to the rest of the nodes.

The Robot 2 x HD Cameras Kinect 2 RGBD Camera (mounted on a Pan-Tilt system) SR300 RGBD camera (Above gripper) GPS 9-DoF IMU 3 x Ultrasonic Range Finders (left, rear, right) Emergency button Battery meter

Step 1- detecting the button To recognize the elevator’s button, we put a red sticker on it. The Kinect2 camera publishes messages of type sensor_msgs/PointCloud2 to topic /kinect2/qhd/points. We use OpenCV — a library for real-time computer vision — to recognize red objects We filter objects with the wrong size The PCL (POINT CLOUD) library processes the image and gives us its coordinates With moment function we find the center of the button

Step 2 - Calculating path to the button The button coordinates are with respect to the head camera We use TF — a package that tracks of multiple coordinate frames over time — to convert to base coordinates If the button is to high, we lift armadillos torso to get it. If the button is out of reach, the robot goes back to driving position

Step 3 – Pushing the button We add the button to the robots collision matrix

Step4 - Elevator opening recognition and entering the elevator Armadillo uses a laser scanner to recognize the elevators arrival. The scanner publishes massages of type sensor_msgs/LaserScan.msg to the topic /scan. The sensor_msgs/LaserScan message contains the information about the distance measurements between the robot and it’s surroundings. When the elevator is open the distance grows. We use moveit which is a motion planning framework and gmapping to calculate a safe path into the elevator.

The End