Parallelized Evolution System Onur Soysal, Erkin Bahçeci Erol Şahin Dept. of Computer Engineering Middle East Technical University ISCIS’03 04/11/2003

ISCIS’03www.kovan.ceng.metu.edu.tr Evolutionary Methods Simulation of Natural Evolution Using evolution mechanisms to solve optimization problems Survival of the fittest Generation n Chrom.1: Chrom.2: Generation n+1 Select Reproduce Mutate Chrom.1: Chrom.2: Population n Population n+1

ISCIS’03www.kovan.ceng.metu.edu.tr Genetic Algorithms Encoding Binary representation Stochastic search Guided by measure of fitness Problem-specific fitness function Genetic operators Mutation Crossover

ISCIS’03www.kovan.ceng.metu.edu.tr Evolutionary Robotics Motivation Controller: Sensor-actuator relationship Difficult to design manually in nontrivial tasks To be in the shoes of a robot? Solutions of evolutionary robotics Evolving controllers SensorsActuators Controller ???

ISCIS’03www.kovan.ceng.metu.edu.tr Genotype  Phenotype: Generic Controller Sensor data Actuator outputs Convert to controller parameters Use the controller in robots Controller Chromosome

ISCIS’03www.kovan.ceng.metu.edu.tr Genotype  Phenotype: Neural Network Controller Sensor data Actuator outputs Convert to neural network weights Use the neural network to control robots Chromosome

ISCIS’03www.kovan.ceng.metu.edu.tr Fitness Evaluation Use controller on physical robots on simulated robots Observe behavior Calculate fitness value Repeat for each generation for each chromosome

ISCIS’03www.kovan.ceng.metu.edu.tr Physics Based Simulation Importance of realistic simulations Pros Much faster runtime compared to physical implementation Behavioral similarity to physical robots Realistic interactions Cons High processing demand! Example libraries: ODE, Vortex, Havok

ISCIS’03www.kovan.ceng.metu.edu.tr Single Machine Limitations Computation required Solving Ordinary Differential Equations Increasing complexity with more collisions Time estimates for single computer Order of minutes for a single evaluation For 100 chromosomes and 100 generations Total time > a week on a single machine

ISCIS’03www.kovan.ceng.metu.edu.tr Cluster Computing Cheap alternative to supercomputers Large-grained problems Example application Search for Extraterrestrial Intelligence at home

ISCIS’03www.kovan.ceng.metu.edu.tr Cluster Computing Solution to Evolutionary Robotics Fitness evaluations dominate computation time Fitness evaluations are independent Master-Slave model Master: GA Slaves: Fitness evaluations

ISCIS’03www.kovan.ceng.metu.edu.tr Parallel Evolution System (PES) Parallel Genetic Algorithm library Distributed fitness evaluation Central genetic algorithm Supports interoperability of multiple platforms

ISCIS’03www.kovan.ceng.metu.edu.tr PES Architecture Server: GA Clients: Fitness evaluation PES-C Client Application PES-S Server Application

ISCIS’03www.kovan.ceng.metu.edu.tr PES Communication Model PES Network Adapter PES-C PES Network Adapter PES-S Ping mechanism Task and result packets ping ping reply task result

ISCIS’03www.kovan.ceng.metu.edu.tr PES-S Architecture Server Application GA Engine Task Manager PES-S Task generator Best solutions Network Adapter

ISCIS’03www.kovan.ceng.metu.edu.tr PES-C Architecture Client Application Network Adapter Task Fitness PES-C

ISCIS’03www.kovan.ceng.metu.edu.tr PES Usage: Server Server Code #include int main() { PES_INIT(); for each generation { PES_EVAL_INDIVIDUALS(); PES_NEXT_GENERATION(); } }

ISCIS’03www.kovan.ceng.metu.edu.tr PES Usage: Client Client Code #include int main() { PES_INIT(); while true { RECEIVE_TASK(); EVAL_TASK(); SEND_RESULT(); } }

ISCIS’03www.kovan.ceng.metu.edu.tr PES Features Multi-platform support Linux and Windows via PVM Improved fault tolerance Ping mechanism to detect non-responding clients Supplies GA facilities Built-in crossover, mutation and selection functions Simplified usage Automatic task distribution / fitness collection Single line for evaluating fitness of whole population Basic load balancing

ISCIS’03www.kovan.ceng.metu.edu.tr Our Application Task: Aggregation Getting 10 robots as close as possible Minimal sensors and communication Modified single layer NN for control No memory/learning No world knowledge Probabilistic inputs Fitness evaluation 2000 steps of simulation Total distance as fitness measure R

ISCIS’03www.kovan.ceng.metu.edu.tr Sample Application Results

ISCIS’03www.kovan.ceng.metu.edu.tr Processor Load Balancing Dynamic simulation Varying number of collisions Varying task complexity Varying processor load Diamonds: tasks Solid lines: Start of new generation

ISCIS’03www.kovan.ceng.metu.edu.tr Fault Tolerance Processor 2 fails Detected at ping at 15 th sec Task restart at 19 th sec Red lines: Ping Blue lines: Generation Numbers: Task index

ISCIS’03www.kovan.ceng.metu.edu.tr Efficiency & Speedup

ISCIS’03www.kovan.ceng.metu.edu.tr Discussions & Future Work Generation gap Asynchronous reproduction Dynamic host list Progress monitor Extension to the Internet

ISCIS’03www.kovan.ceng.metu.edu.tr PES Sources & Documentation Source distribution under Gnu Public License (GPL) User manual Functional reference Sample applications