1. Exploration Robot with Stereovision Vladislav Richter richterv2@fit.cvut.cz Miroslav Skrbek FIT, CTU in Prague skrbek@fit.cvut.cz

2. Vladislav Richter, Miroslav Skrbek PESW2014 2 Outline Introduction Robot's hardware Control Application Sensor visualization Kinect

3. Vladislav Richter, Miroslav Skrbek PESW2014 3 Exploration Robot Android Android phone or tablet Wi-Fi 3D image (anaglyph) Commands forward backward rotate set_wheel_speed say get_eye_image …

4. Vladislav Richter, Miroslav Skrbek PESW2014 4 Robot's Hardware Cameras (stereo pair) WiFi antena Roboard RB110 (1GHz PC,Linux) Speaker (speech synthesis) 80 cm distance IR sensor Accelerometer, gyroscope, battery voltage, battery current and temperature sensors inside Audio codec with an amplifier Eye Movement (up/down)

5. Vladislav Richter, Miroslav Skrbek PESW2014 5 Remote Control Application Directed for tablets and mobile phones Android based application –Earlier version Android 2.3 –Latest version Android 4.4 Motor control Sensor visualization Sound localization Voice output

6. Vladislav Richter, Miroslav Skrbek PESW2014 6 Sensor Visualization Accelerometer Gyroscope Motor speed System temperature Battery –Voltage –Current IR distance sensors Sound source location

7. Vladislav Richter, Miroslav Skrbek PESW2014 7 Accelerometer Data Visualizations Accelerometer data The model follows the movement of the robot Terrain

8. Vladislav Richter, Miroslav Skrbek PESW2014 8 Sound Source Localization Bar with four microphones (two stereo pairs) Cross correlation I Cross correlation II Localization Δt1 Δt2

9. Vladislav Richter, Miroslav Skrbek PESW2014 9 Camera Images 3D View Anaglyph Cameras: 640x480, 30 fps, YUYV

10. Vladislav Richter, Miroslav Skrbek PESW2014 10 Nearest object detection Left image Right image OpenCV based stereo correspondence calculation Depth map Nearest objects are white

11. Vladislav Richter, Miroslav Skrbek PESW2014 11 Motor Control Virtual Joystick Movement keypad

12. Vladislav Richter, Miroslav Skrbek PESW2014 12 Advanced Feature – GUI Adaptation Android get_xml_info XML file Description of sensors and commands available on the robot GUI adaptation

13. Vladislav Richter, Miroslav Skrbek PESW2014 13 Kinect on the Robot Wi-Fi Server UDP multi-cast streams - color image - depth map - infrared image - skeleton tracking data - accelerometer - audio UDP stream - motor control - stream control 15 fps

14. Vladislav Richter, Miroslav Skrbek PESW2014 14 Kinect - Depth Map Image depth is 12 bits. To view the full range, a color scheme is used. Depth map

15. Vladislav Richter, Miroslav Skrbek PESW2014 15 Kinect - Color Image Head-up Display

16. Vladislav Richter, Miroslav Skrbek PESW2014 16 Kinect – Infrared Image

17. Vladislav Richter, Miroslav Skrbek PESW2014 17 Kinect - Skeleton Data

18. Vladislav Richter, Miroslav Skrbek PESW2014 18 Conclusion Four-wheeled robot hardware Stereo camera pair for distance measurement Reach set of sensors Sound source location Voice output Android based control application –3D view by anaglyph –Sensor data visualization –Motor control

19. Vladislav Richter, Miroslav Skrbek PESW2014 19 Thank you for your attention