1 Autonomic Computer Systems Evolutionary Computation Pascal Paysan

2 Optimization  Maximize / minimize objective function subject to constraints  Linear / non-linear / discreet Wish List  Cope with large search and solution spaces  Minimal human intervention  Avoid local minima / maxima  Independent of the initialization  Ability to deal with dynamic environments Motivation

3 Optimization General formulation of an optimization problem: f(x) = objective function g i (x) = constraints Simple example: 1 variable (x), no constraints maximize: subject to: f(x)‏ x global optimum local optima search space best solution

4 Darwinian Evolution Reproduction = replication + (unlimited) heritable variation  Replication of the DNA sequence  Cell replication  Organism reproduction  Variation: mutation, recombination Fitness = Reproduction rate  how fast an organism (or species) is able to reproduce Selection: survival of the fittest  exponential growth + finite resources = competition  outcome: competitive exclusion (survival of the fittest)‏

5 Evolutionary Computation: Basic Concepts Genotype: the genetic material of an individual Phenotype: the ensemble of observable traits Fitness: measure of how good a candidate solution is  tested on a number of test cases (training set)‏  expressed as a fitness function: e.g. error between ideal and obtained solution (on training case); absolute or relative performance measure Selection strategy: Algorithm that selects individuals in the population that will build the next generation  Principle: "survival of the fittest": best fit individuals have a higher chance of being selected  Selected individuals undergo variation through genetic operators to form the next generation

6 Evolutionary Computation Genetic Algorithms (GA)‏  goal: find an optimum solution (e.g. combination of parameters) to an instance of a problem  candidate solutions are typically strings Genetic Programming (GP)‏  goal: find an optimum program able to solve any instance of the problem  candidate solutions are programs

7 Genetic Algorithms: Basic Concepts Genetic operators: variation functions that transform a set of individuals (parents) into a new set (offspring)‏ Common operators:  Mutation: random change in genotype, with low probability  Crossover: recombine portions of two genotypes offspring mutation parent crossover offspring 1 offspring 2 parent 1 parent 2 crossover point

8 Use mutation and offspring to find programs  Generated valid programs  Tree-Based GP Closure – gracefully produce valid outputs given any possible inputs  Division by zero – default value Different representation for the functionality  Linear – Avida, nop patterns  Grammatical Evolution – Grow prg. Using BNF  Algorithmic Chemistry – Random execution Genetic Programming: Basic Concepts

9 Optimization Issues Premature Convergence – Local optimum  Strategy to avoid Ruggedness – Bumpy cost  No reliable gradient Deceptiveness Neutrality – Change don't affects cost Overfitting – Loss of Generality (solution) No Free Lunch – trade-off between performance of the algorithm for a specific problem and generalization for all problems f(x)‏ x global optimum z x y z x y optimization run f(x)‏ x

10 Optimization in Dynamic Environments Challenges: change and uncertainty  noise / errors in fitness  changes in environmental parameters  change in desired optimum Re-optimize (start from scratch) is expensive  Track or discover new optima instead Crucial to keep diversity  if the optimum changes, the population must be able to re-adapt: this requires diversity in the population

11 Genetic algorithm to learn how to walk Sony Aibo