1 Algoritmos Genéticos aplicados em Machine Learning Controle de um Robo (em inglês)

2 Robot Control using Genetic Algorithms

3 Summary Introduction –Robot Control –Khepera Simulator Genetic Model for Path Planning –Chromosome Representation –Evaluation Function –Case Studies Conclusions

4 The Robot Controller Problem Given a robot and a description of an environment, provide commands (motor speeds) to the robot, in order to achieve a path between two specified locations, which is collision-free and satisfies certain optimisation criteria. (x i, y i ) (x f, y f )

5 Optimisation Criteria Robot should: –attempt near-optimal paths –avoid obstacles –perform straight motion Controller should be independent of: –the robot’s environment –target location

6 The Khepera Simulator Freeware mobile robot simulator (designed by Olivier Michel, University of Nice Sophia-Antipolis) User designed worlds Control algorithms can be written in C/C++ Robot’s position and angle reading 8 sensors (S0-S7): [0, 1023] 2 motors (M1, M2): [-10, +10]

7 Simulator Readings: sensors, position and angle S0-S7: [0, 1023] 1000 X Y Robot’s World angle of the robot with the world  : [- ,  ] x y 0 obstacle not obstacle very detected closed

8 Control Mode To evolve the robot’s attitudes as it interacts with the environment Each robot action determines: –how well the controller performs with respect a given task; –the next input stimuli to the controller. learnThe controller should learn as the robot interacts with the environment

9 Controller Model GeneticAlgorithm evolves robot’s attitudes Sensors Position Robot’s Angle Goal Location Motor 2 Motor 1 KheperaSimulator

10 Proposed Model based on human behavior Obstacle detected IF Obstacle detected THEN Avoid collision, forget target ELSE Straight to the target according to the target direction END

11 Sensors Reading Simplification

12 Determining the Target Direction Direction =

13 Model ((S left > L) or (S right > L) or (S back > L)) IF ((S left > L) or (S right > L) or (S back > L)) THEN Obstacle detected, avoid collision, forget target Proximity-sensor = highest value (S left, S right, S back ) ELSE Obstacle not detected (collision-free), straight to the target Target direction =  -  END L=collision threshold=900

14 Genetic Algorithm Modelling Problem Chromosome Representation Evaluation Function Genetic Operators Techniques Parameters

15 Chromosome Representation Which speed should be imposed to each motor in each situation the robot is?

16 Evaluation Function Main objectives: –(V) speed: as high as possible –(D) straight motion: same motor speed for M1 e M2 –(A) action: reach a target and avoid obstacles Calculated based on the contribution for each gene [1,7], at each step.

17 Speed Normalised sum of the absolute value of the motors speeds; Vi increases as both speeds increase Whatever the robot does, it does quickly.

18 Straight Motion It favours high positive speeds to both motors When the robot is not oriented to the target (2,3,4), D=1 avoids contradictory learning

19 Action It considers the benefit of each gene regarding to: –obstacle avoidance –target closeness TPi = total of steps executed by attitude i AAi=action’s fitness at stept of attitude i

20 Action It considers the benefit of each gene regarding to: –obstacle avoidance –target closeness TPi = total of steps executed by attitude i AAi=action’s fitness at stept of attitude i Rates the distance variation to the target between two consecutive steps, and the maximum distance in one step, for collision free/front

21 Action It considers the benefit of each gene regarding to: –obstacle avoidance –target closeness TPi = total of steps executed by attitude i AAi=action’s fitness at stept of attitude i Rates the angle variation between two consecutive steps, and the maximum angle in one step, for collision free left, right, back

22 Action It considers the benefit of each gene regarding to: –obstacle avoidance –target closeness TPi = total of steps executed by attitude i AAi=action’s fitness at stept of attitude i Increases as the distance to the proximity-sensor increases in the step

23 Improving the Target Direction Model 4 possible target directions 0  /4  /2 3  /4  /2 -3  /4 -  /4 -  /2 8 possible target directions

24 Chromosome Representations

25 Genetic Algorithm Integer chromosome Population Size =100 Generations = 50 Crossover Rate = 80 % Mutation Rate = 4% Roulette Wheel Reproduction Elitism Linear scaling of fitness 300 Evaluation Steps for each chromosome Average of 25 Experiments

26 Genetic Algorithm Performance 7 Genes Chromosome

27 Genetic Algorithm Performance 7 Genes Chromosome

28 Genetic Algorithm Performance 11 Genes Chromosome

29 Genetic Algorithm Performance 11 Genes Chromosome

30 Paths Achieved in World 1 Case Study 1 7 Genes Chromosome11 Genes Chromosome

31 Paths Achieved in World 1 Case Study 2 7 Genes Chromosome11 Genes Chromosome

32 Speed Comparison 11 Genes Chromosome 7 Genes Chromosome

33 Paths Achieved in World 2 Case Study 1 7 Genes Chromosome11 Genes Chromosome

34 Paths Achieved in World 2 Case Study 2 7 Genes Chromosome11 Genes Chromosome

35 Speed Comparison 11 Genes Chromosome 7 Genes Chromosome

36 Speed Comparison (%) Case Study 1 Case Study 2 Case Study 3

37 Paths Achieved in World 3 Case Study 1 7 Genes Chromosome11 Genes Chromosome

38 Paths Achieved in World 3 Case Study 2 7 Genes Chromosome11 Genes Chromosome

39 Paths Achieved in World 3 Case Study 3 7 Genes Chromosome11 Genes Chromosome

40 Paths Achieved in World 3 Case Study 4 7 Genes Chromosome11 Genes Chromosome

41 Speed Comparison

42 Conclusions A simple GA was able to gradually evolve the robot control The robot achieved near optimal path towards the goal, avoiding obstacles Retraining is not necessary when the environment changes Controller improved performance with the 11 genes model The robot has no memory about previous unsuccessful paths and may get lost Other tasks can be included in the model (e.g. energy supply) Chromosome codification is limited for few robot’s situations