Sérgio Ronaldo Barros dos Santos (ITA-Brazil) Sidney Nascimento Givigi Júnior (RMC-Canada) Cairo Lúcio Nascimento Júnior (ITA-Brazil) Autonomous Construction.

Slides:



Advertisements
Similar presentations
Application a hybrid controller to a mobile robot J.-S Chiou, K. -Y. Wang,Simulation Modelling Pratice and Theory Vol. 16 pp (2008) Professor:
Advertisements

Joshua Fabian Tyler Young James C. Peyton Jones Garrett M. Clayton Integrating the Microsoft Kinect With Simulink: Real-Time Object Tracking Example (
Model Checker In-The-Loop Flavio Lerda, Edmund M. Clarke Computer Science Department Jim Kapinski, Bruce H. Krogh Electrical & Computer Engineering MURI.
Mechatronics 1 Weeks 5,6, & 7. Learning Outcomes By the end of week 5-7 session, students will understand the dynamics of industrial robots.
Learning Parameterized Maneuvers for Autonomous Helicopter Flight Jie Tang, Arjun Singh, Nimbus Goehausen, Pieter Abbeel UC Berkeley.
DARPA Mobile Autonomous Robot SoftwareMay Adaptive Intelligent Mobile Robotics William D. Smart, Presenter Leslie Pack Kaelbling, PI Artificial.
Effective Reinforcement Learning for Mobile Robots Smart, D.L and Kaelbing, L.P.
Monte Carlo Localization for Mobile Robots Karan M. Gupta 03/10/2004
Cam-follower systems: experiments and simulations by Ricardo Alzate University of Naples – Federico II WP6: Applications.
Design of Attitude and Path Tracking Controllers for Quad-Rotor Robots using Reinforcement Learning Sérgio Ronaldo Barros dos Santos Cairo Lúcio Nascimento.
1 Sensor Relocation in Mobile Sensor Networks Guiling Wang, Guohong Cao, Tom La Porta, and Wensheng Zhang Department of Computer Science & Engineering.
Path Planning with the humanoid robot iCub Semester Project 2008 Pantelis Zotos Supervisor: Sarah Degallier Biologically Inspired Robotics Group (BIRG)
עקיבה אחר מטרה נעה Stable tracking control method for a mobile robot מנחה : ולדיסלב זסלבסקי מציגים : רונן ניסים מרק גרינברג.
Tracking a moving object with real-time obstacle avoidance Chung-Hao Chen, Chang Cheng, David Page, Andreas Koschan and Mongi Abidi Imaging, Robotics and.
Motion based Correspondence for Distributed 3D tracking of multiple dim objects Ashok Veeraraghavan.
Automatic Control & Systems Engineering Autonomous Systems Research Mini-UAV for Urban Environments Autonomous Control of Multi-UAV Platforms Future uninhabited.
Efficient Methodologies for Reliability Based Design Optimization
Dynamic Medial Axis Based Motion Planning in Sensor Networks Lan Lin and Hyunyoung Lee Department of Computer Science University of Denver
Interactive Manipulation of Rigid Body Simulations Presenter : Chia-yuan Hsiung Proceedings of SIGGRAPH 2000 Jovan Popovi´c, Steven M. Seitz, Michael.
Abstract In this project we expand our previous work entitled "Design of a Robotic Platform and Algorithms for Adaptive Control of Sensing Parameters".
Sérgio Ronaldo Barros dos Santos, Cairo Lúcio Nascimento Júnior,
Teaching with MATLAB - Tips and Tricks
Operations Research Models
Charles L. Karr Rodney Bowersox Vishnu Singh
Parallelism and Robotics: The Perfect Marriage By R.Theron,F.J.Blanco,B.Curto,V.Moreno and F.J.Garcia University of Salamanca,Spain Rejitha Anand CMPS.
Sérgio Ronaldo Barros dos Santos (ITA-Brazil)
Department of Electrical Engineering, Southern Taiwan University Robotic Interaction Learning Lab 1 The optimization of the application of fuzzy ant colony.
COBXXXX EXPERIMENTAL FRAMEWORK FOR EVALUATION OF GUIDANCE AND CONTROL ALGORITHMS FOR UAVS Sérgio Ronaldo Barros dos Santos,
The Application of The Improved Hybrid Ant Colony Algorithm in Vehicle Routing Optimization Problem International Conference on Future Computer and Communication,
Ibrahim Fathy, Mostafa Aref, Omar Enayet, and Abdelrahman Al-Ogail Faculty of Computer and Information Sciences Ain-Shams University ; Cairo ; Egypt.
UAV Navigation by Expert System for Contaminant Mapping George S. Young Yuki Kuroki, Sue Ellen Haupt.
1 Distributed and Optimal Motion Planning for Multiple Mobile Robots Yi Guo and Lynne Parker Center for Engineering Science Advanced Research Computer.
Zibin Zheng DR 2 : Dynamic Request Routing for Tolerating Latency Variability in Cloud Applications CLOUD 2013 Jieming Zhu, Zibin.
Mobile Agent Migration Problem Yingyue Xu. Energy efficiency requirement of sensor networks Mobile agent computing paradigm Data fusion, distributed processing.
© 2011 Autodesk Freely licensed for use by educational institutions. Reuse and changes require a note indicating that content has been modified from the.
S ystems Analysis Laboratory Helsinki University of Technology Automated Solution of Realistic Near-Optimal Aircraft Trajectories Using Computational Optimal.
Distributed Algorithms for Multi-Robot Observation of Multiple Moving Targets Lynne E. Parker Autonomous Robots, 2002 Yousuf Ahmad Distributed Information.
Learning to Navigate Through Crowded Environments Peter Henry 1, Christian Vollmer 2, Brian Ferris 1, Dieter Fox 1 Tuesday, May 4, University of.
Aeronautics & Astronautics Autonomous Flight Systems Laboratory All slides and material copyright of University of Washington Autonomous Flight Systems.
MURI Telecon, Update 7/26/2012 Summary, Part I:  Completed: proving and validating numerically optimality conditions for Distributed Optimal Control (DOC)
Operations Research The OR Process. What is OR? It is a Process It assists Decision Makers It has a set of Tools It is applicable in many Situations.
1 Motion Fuzzy Controller Structure(1/7) In this part, we start design the fuzzy logic controller aimed at producing the velocities of the robot right.
Coverage Efficiency in Autonomous Robots With Emphasis on Simultaneous Localization and Mapping Mo Lu Computer Systems Lab Q3.
Optimization in Engineering Design 1 Introduction to Non-Linear Optimization.
DEPARTMENT/SEMESTER ME VII Sem COURSE NAME Operation Research Manav Rachna College of Engg.
Operations Research Models and Methods Advanced Operations Research Reference: Operations Research Models and Methods, Operations Research Models and Methods,
Path Planning Based on Ant Colony Algorithm and Distributed Local Navigation for Multi-Robot Systems International Conference on Mechatronics and Automation.
A Protocol for Tracking Mobile Targets using Sensor Networks H. Yang and B. Sikdar Department of Electrical, Computer and Systems Engineering Rensselaer.
Advanced Computer Graphics Spring 2014 K. H. Ko School of Mechatronics Gwangju Institute of Science and Technology.
The Mechanical Simulation Engine library An Introduction and a Tutorial G. Cella.
Dynamic Mission Planning for Multiple Mobile Robots Barry Brumitt and Anthony Stentz 26 Oct, 1999 AMRS-99 Class Presentation Brian Chemel.
Minor Project on Vertical Take-off Landing System SUBMITTED BY:- SHUBHAM SHARMA ( ) ABHISHEK ARORA ( ) VIBHANSHU JAIN ( )
Learning of Coordination of a Quad-Rotors Team for the Construction of Multiple Structures. Sérgio Ronaldo Barros dos Santos. Supervisor: Cairo Lúcio Nascimento.
Department of Electrical Engineering, Southern Taiwan University 1 Robotic Interaction Learning Lab The ant colony algorithm In short, domain is defined.
Space Robotics Seminar On
© 2012 Anwendungszentrum GmbH Oberpfaffenhofen Idea by: Dr. Eng. Mohamed Zayan | 1.
Simulations of Curve Tracking using Curvature Based Control Laws Graduate Student: Robert Sizemore Undergraduate Students: Marquet Barnes,Trevor Gilmore,
1 26/03/09 LIST – DTSI Design of controllers for a quad-rotor UAV using optical flow Bruno Hérissé CEA List Fontenay-Aux-Roses, France
Multi-robot
Introduction to Machine Learning, its potential usage in network area,
Generation and Testing of Gait Patterns for Walking Machines Using Multi-Objective Optimization and Learning Automata Jeeves Lopes dos Santos Cairo L.
Project Overview Introduction Frame Build Motion Power Control Sensors
Sérgio Ronaldo Barros dos Santos Cairo Lúcio Nascimento Júnior
Timothy Boger and Mike Korostelev
Advanced Computer Graphics Spring 2008
An Adaptive Middleware for Supporting Time-Critical Event Response
Synthesis of Motion from Simple Animations
Market-based Dynamic Task Allocation in Mobile Surveillance Systems
Chapter 4 . Trajectory planning and Inverse kinematics
Motion Cueing Standards for Commercial Flight Simulation
Presentation transcript:

Sérgio Ronaldo Barros dos Santos (ITA-Brazil) Sidney Nascimento Givigi Júnior (RMC-Canada) Cairo Lúcio Nascimento Júnior (ITA-Brazil) Autonomous Construction of Structures in a Dynamic Environment using Reinforcement Learning

2/25 Introduction In recent years, there has been a growing interest in a class of applications in which mobile robots are used to assemble and build different types of structures. These applications traditionally involves human performing:  The operation of tools and equipament;  The manipulation and transportation of the resources for manufacturing of structures; and  The careful preplanning of the tasks that will be executed.

3/25 Due to recent advancements in technologies available for UAVs, the problem of autonomous manipulation, transportation and construction is advancing to the aerial domain. Autonomous construction using aerial robots may be useful in several situations, such as:  Reduce the high accident rates in traditional construction;  Enable the construction in extraterrestrial environments or disaster areas; and  Use in military and logistics applications. Introduction

4/25 Quad-rotor Robot All of the movements of quad-rotor can be controlled by the changes of each rotor speed. An inertial frame and a body fixed frame whose origin is in the center of mass of the quad-rotor are used.

5/25 Problem Statement Construction task using mobile robots are characterized by three fundamental problems such as task planning, motion planning and path tracking. However, to obtain the task and path planning that define a specific sequence of operations for construction of different structures is generally very complex. The task planning, motion planning and low-level controllers for robotic assembly are derived off-line through a simulation environment, using Reinforcement Learning (RL) and heuristic search (A*) algorithms, and then the solutions are ported to an actual quad-rotor.

6/25 Problem Statement Proposed Environment The 3-D Structures suggested This work is concentrated on the learning of four different types of 3-D structures: cube, tower, pyramid and wall, similar to those used in construction of scaffolds, tower cranes, skyscrapers, etc.

7/25 Proposed Solution Low-level controllers: Enable the position and path tracking control of the quad-rotor. Task planning: Provide the maneuvers and assembly sequence. Path Planning: Find the optimal path for the robot so that its navigation by the dynamic environment is executed.

8/25 Experimental Infrastructure

9/25 Experimental Infrastructure

10/25 Reinforcement Learning The task planning and low-level controllers for robotic assembly were learned by a reinforcement learning algorithm known as Learning Automata (LA).

11/25 Learning Control of a Quad-rotor The low-level controllers are adapted simultaneously, (attitude and height) and after (position and path tracking) using the nonlinear dynamics model of the target quad-rotor built for the X-Plane Flight Simulator and also the LA algorithm executing in Matlab.

12/25 Learning Control of a Quad-rotor Some considerations that must be taken into account during the learning phase, such as wind and ground effects, as well as the change of mass and center of gravity of the system produced by different types of payloads.

13/25 Learning Control of a Quad-rotor A simulation setup is proposed to the training and evaluation of the control parameters under realistic conditions.

14/25 Learning Control of a Quad-rotor Experimental setup used to test and validate the learned attitude and path tracking controllers in simulation.

15/25 Learning Control of a Quad-rotor Path tracking and height responses obtained by the quad-rotor during the test of the adapted control laws. Test in simulation Experimental Validation

16/25 Learning of the Robotic Assembly The proposed learning system for the autonomous construction of structures. The training process of the task planning is accomplished by a team of automata. Learning Architecture Learning Automata

17/25 Learning of the Robotic Assembly The proposed total cost function to evaluate the structure construction mode is given by: The numeric value of the response quality obtained by the robot during each iteration is computed using:

18/25 Learning of the Robotic Assembly The value of R c (n) ϵ [R p, R G ] is understood as the established limit by the user to change the speed convergence during the training process. A common reinforcement is used to update the action probability distributions of the team of automata.

19/25 Learning of the Robotic Assembly During the learning phase, it is noted that the acquired knowledge by the system relative to assembly of a 3-D structure (tower) increases with each iteration. The learned sequence of maneuvers and assembly for construction of a tower are illustrated in plots below.

20/25 Learning of the Robotic Assembly Experimental setup used to validate the learned task planning and the produced path planning by RL and A* algorithms, simultaneously.

21/25 Learning of the Robotic Assembly The Executed events for the assembly task of a structure.

22/25 Learning of the Robotic Assembly The resulting trajectory of the sequence of maneuvers learned for assembling the tower, through a quad-rotor was successfully performed.

23/25 Conclusions This method allows the autonomous construction of multiple 3-D structures based on the Learning Automata and A* algorithms using a quad-rotor. This approach reduces substantially the effort employed for developing a task and motion planning that permits a robot to efficiently assemble and construct multiple 3-D structures. The use of reinforcement learning for finding different set of actions to build a 3-D structure is very promising.

24/25 Conclusions The proposed learning architecture enables an aerial robot to learn a good sequence of maneuvers and assembly so that the constraints inherent in the structures and environment are overcome. It has been shown that a 3-D structure can be built using the adapted low-level controllers, the learned task planning and also the produced path planning.

25/25 Thank you Questions?