1 Final Conference, 19th – 23rd January 2015 Geneva, Switzerland A Macro-Micro Visual Servoing Controller for Mobile Manipulation Research Package(RP): 10 ESR: Mohammad M. Aref Supervisor: Jouni Mattila Organization: Tampere University of Technology Project: 06/2011 – 06/2014 Presented by: Reza Ghabcheloo

2 Final Conference, 19th – 23rd January 2015 Geneva, Switzerland Background Information

3 Final Conference, 19th – 23rd January 2015 Geneva, Switzerland Project results & Learning outcome –System engineering concepts for mobile manipulation in scientific infrastructures –Control system architecture and fault tolerance –Visual servoing –… other relevant skills through numerous PURESFAE workshops

4 Final Conference, 19th – 23rd January 2015 Geneva, Switzerland … in details Research on mobile manipulators: Sensor fusion and fault tolerance Visual servoing and base manipulator coordination Several SE documents as part of the PURESAFE project deliverables:  “System Requirement Document for Mobile Manipulation in CERN infrastructures”.  “Architectural Design Document of the iMoro Mobile Manipulator”.  “Detailed Design Description Document of the iMoro Mobile Manipulator”.  “Integration and Acceptance Test Document of the iMoro Mobile Manipulator”.  A highly maneuverable Modular Mobile Manipulator System (MMMS)  Designed and built at IHA/TUT, based on the statement of need (PURESAFE ANNEX Ι)  Based on the requirements gathered and presented in the project requirements document and approved by the project partners.  Test platform for verifying reliable high performance motion controller for the mobile platforms Mechatronics design and system integration of iMoro:

5 Final Conference, 19th – 23rd January 2015 Geneva, Switzerland Research use cases Finnish center of excellence: Generic Intelligent Machines Research ( ) Future intelligent mobile working machines and field robots. GIM Development of mobile manipulation technology for ionizing radiation environments

6 Final Conference, 19th – 23rd January 2015 Geneva, Switzerland GIM machine pallet picking Challenges Rough terrain Inaccurate actuators Big dimensions (resolution, calibration) Exception handling Coordinated motion

7 Final Conference, 19th – 23rd January 2015 Geneva, Switzerland Micro – Macro control system architecture Macro (platform) Steering and drive Micro (manipulator) boom, telescope, fork

8 Final Conference, 19th – 23rd January 2015 Geneva, Switzerland Architecture in some details

9 Final Conference, 19th – 23rd January 2015 Geneva, Switzerland Main objective F  F* Error space  two orthogonal spaces: F  F* and B  B* Micro control assuming Macro errors are small: B=B* Control in some details

10 Final Conference, 19th – 23rd January 2015 Geneva, Switzerland Local localization smoothens the vision output (vision: every 300ms) provides 20Hz pose estimate, negligible delays works even when markers in shadow or lost Macro-Micro coordination and exception handling Macro waits for Micro (no need for tight coordination) Terminate/replan in case of delayed detection Redundancy resolution,… Detection and filtering (local localization)

11 Final Conference, 19th – 23rd January 2015 Geneva, Switzerland

12 Final Conference, 19th – 23rd January 2015 Geneva, Switzerland Experiments

13 Final Conference, 19th – 23rd January 2015 Geneva, Switzerland Experiments

14 Final Conference, 19th – 23rd January 2015 Geneva, Switzerland Publications M. M. Aref, R. Oftadeh, R. Ghabcheloo, and J. Mattila, “Fault Tolerant Control Architecture Design for Mobile Manipulation in Scientific Facilities”, International Journal of Advanced Robotic Systems, InTech, December M. M. Aref, R. Ghabcheloo, J. Mattila, “A Macro-Micro Controller for Pallet Picking by an Articulated-Frame-Steering Hydraulic Mobile Machine”, IEEE International Conference on Robotics and Automation, ICRA2014 (May 31-June 5, 2014), Hong Kong Convention and Exhibition Center, Hong Kong. R. Oftadeh, M. M. Aref, R. Ghabcheloo, and J. Mattila, “System Integration for Real-time Mobile Manipulation”, International Journal of Advanced Robotic Systems, vol 11, no. 51, InTech, 2014.

15 Final Conference, 19th – 23rd January 2015 Geneva, Switzerland Publications M. M. Aref, R. Ghabcheloo, A. Kolu, M. Hyvönen, J. Mattila, K. Huhtala, "Position Based Visual Servoing for Pallet Picking by an Articulated- frame-steering Hydraulic Mobile Machine", The 6th IEEE International Conference on Robotics, Automation and Mechatronics, RAM2013 (November 12-15, 2013), Manila, Philippines. R. Oftadeh, M. M. Aref, R. Ghabcheloo, and J. Mattila, “Mechatronic Design of a Four Wheel Steering Mobile Robot with Fault-Tolerant Odometry Feedback” in the proceedings of The 6th IFAC Symposium on Mechatronic Systems (Mechatronics '13) Zhejiang University, Hangzhou, China, April 10-12, R. Oftadeh Mohammad M. Aref, Reza Ghabcheloo, Jouni Mattila,”Unified Framework for Rapid Prototyping of Linux based Real-Time Controllers with Matlab and Simulink”, Accepted to present in the 2012 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), KaoHsiung, Taiwan.

16 Final Conference, 19th – 23rd January 2015 Geneva, Switzerland Currently working on Extension of these ideas to iMoro, a four wheel steering mobile manipulator with 6 DOF arm. –Macro-Micro coordination –Sensor fusion: vision IMU / body kinematic constraint / object motion constraint Addressing manipulator torque limits in tracking performance: aggressive maneuvers

17 Final Conference, 19th – 23rd January 2015 Geneva, Switzerland Much remains to be done

18 Final Conference, 19th – 23rd January 2015 Geneva, Switzerland THANK YOU for your attention Our YouTube Channel