Evolutionary Algorithms

Outline Genetic algorithm Genetic Programming Differential Evolution MOEA/D

Genetic Algorithms “Genetic Algorithms are good at taking large, potentially huge search spaces and navigating them, looking for optimal combinations of things, solutions you might not otherwise find in a lifetime.” - Salvatore Mangano Computer Design, May 1995

Genetic algorithm I am at the top I am not at the top. Height is ... I am not at the top. My high is better! I will continue

John Holland, university of Michigan, 1975. introduces the principle of evolution and genetics into search among possible solutions to given problem. The idea is to simulate the process in natural systems. This is done by the creation within a machine of a population of individuals represented by chromosomes, in essence a set of character strings, that are analogous to the DNA, that we have in our own chromosomes.

Survival of the Fittest Inspired by Darwin's theory of evolution: survival of the fittest. The good solution survive, while bad ones die.

Genetic Algorithms Genetic algorithms are mainly used in the context of AI: Say you have a really huge search space. You want to find the global optimum for some function in that space. The name and inspiration come from natural selection in biology: We start with some random population of solutions. By randomly breeding and mutating the solutions, new ones are produced. Ranking the solutions and retaining only the best implements natural selection. Iterating this process moves towards near-optimal solutions. 7

1 t←0; initialize Pop(t) with N chromosomes Popi(t) GA() 1 t←0; initialize Pop(t) with N chromosomes Popi(t) 2 while not (terminating condition) do 3 for i ←1 to N do fi← f(Popi(t)) 4 for i ←1 to N do 5 NewPopi(t+1)←randomly choose Popi(t)∈Pop(t) with pj=fj/(∑kfk) 6 CrossPop(t+1) ← recombine(NewPop(t+1)) with pc 7 MutPop(t+1) ← mutate(CrossPop(t+1)) with pm 8 Pop(t+1)←MutPop(t+1) 9 t ← t+1 8

Parameters of GA GA has following parameters: Population size (a set of solutions “chromosomes) Encoding (gene, chromosome) Evaluation function (objective function) Selection (reproduction) Crossover rate Mutation rate Crossover and mutation type

tree - genetic programming Representing Genomes... Representation Example string 1 0 1 1 1 0 0 1 array of strings http avala yubc net ~apopovic or > c tree - genetic programming xor b a b

Fitness Function Fitness function is evaluation function, that determines what solutions are better than others. Fitness is computed for each individual. Fitness function is application depended.

Selection The selection operation copies a single individual, probabilistically selected based on fitness, into the next generation of the population. There are few possible ways to implement selection: Roulette wheel selection by the size of fitness value Rank Survival of the strongest Some weak solutions survive Assign a probability that a particular individual will be selected for the next generation More diversity Some bad solutions might have good parts!

Roulette Wheel Selection Main idea: better individuals get higher chance – Chances proportional to fitness – Assign to each individual a part of the roulette wheel – Spin the wheel n times to select n individuals Fitness Chr. 1 3 Chr. 2 1 Chr. 3 2 3/6=50% 33.33% 50% 1/6=16.67% 2/6=33.33% 16.67%

Rank Selection Roulette-wheel has problem when the fitness value differ greatly In rank selection the worst value has fitness 1, the next 2,......, best has fitness N.

Selection - rank Previous generation 0.93 0.51 0.72 0.31 0.12 0.64 Next generation 0.93 0.72 0.64

Selection - Some Weak Solutions Survive Previous generation 0.93 0.51 0.72 0.31 0.12 0.12 0.64 Next generation 0.93 0.72 0.64 0.12

Crossover Crossover is concept from genetics. Crossover is sexual reproduction. Crossover combines genetic material from two parents, in order to produce superior offspring. Few types of crossover: One-point Multiple point Uniform crossover

Crossover: Recombination One-point Crossover: * P1 (0 1 1 0 1 0 0 0) (0 1 0 0 1 0 0 0) C1 P2 (1 1 0 1 1 0 1 0) (1 1 1 1 1 0 1 0) C2 Crossover is a critical feature of genetic algorithms: It greatly accelerates search early in evolution of a population It leads to effective combination (subsolutions on different chromosomes)

Mutation Mutation introduces randomness into the population. Mutation is asexual reproduction. The idea of mutation is to reintroduce divergence into a converging population. Mutation is performed on small part of population, in order to avoid entering unstable state.

Mutation: Local Modification Before: (1 0 1 1 0 1 1 0) After: (0 1 1 0 0 1 1 0) Before: (1.38 -69.4 326.44 0.1) After: (1.38 -67.5 326.44 0.1) Causes movement in the search space (local or global) Restores lost information to the population

About Probabilities... Average probability for individual to crossover is, in most cases, about 80%. Average probability for individual to mutate is about 1-2%. Probability of genetic operators follow the probability in natural systems. The better solutions reproduce more often.

Stopping Criteria Final problem is to decide when to stop execution of algorithm. There are two possible solutions to this problem: First approach: Stop after production of definite number of generations Second approach: Stop when the improvement in average fitness over two generations is below a threshold

Comments on genetic algorithms Advantages Random exploration can find solutions that local search can’t (via crossover primarily) Appealing connection to human evolution E.g., see related area of genetic programming Concept is easy to understand. Supports multi-objective optimization Always an answer; answer gets better with time !!! Inherently parallel; easily distributed Many ways to speed up and improve a GA-based application as knowledge about problem domain is gained Easy to exploit previous or alternate solutions

Disadvantages Sometimes GA is extremely slow, and much slower than usual algorithms Large number of “tunable” parameters Difficult to replicate performance from one problem to another Lack of good empirical studies comparing to simpler methods Useful on some (small?) set of problems but no convincing evidence that GAs are better than hill-climbing /random restarts in general

Genetic Programming Part of larger discipline called Machine Learning. Machine learning can be best described as "the study of computer algorithms that improve automatically through experience" (Mitchell 1996). It attempts to solve the problem - How can computers be made to do what needs to be done without being told exactly how to do it? It saves time by freeing the human from having to design complex algorithms. Not only designing the algorithms but creating ones that give optimal solutions.

Algorithm Randomly create an initial population of programs from the available primitives Repeat Execute each program and ascertain its fitness Select one or two program(s) from the population with a probability based on fitness to participate in genetic operations Create new individual program(s) by applying genetic operations with specified probabilities until an acceptable solution is found or some other stopping condition is met return the best-so-far individual

Tree Structure (+ 1 2 (IF (> TIME 10) 3 4))‏ Programs are represented as tree. (+ 1 2 (IF (> TIME 10) 3 4))‏

Basics Terminal: The variables and constants in the program (x, y and 3) which forms leaves of the tree. Funtion: The arithmetic operations (+, * and max) are internal nodes called functions. Primitive set : Functions and terminals together from primitive set.

Initialization of population Full Method: Selecting internal nodes from function set untill maximum depth is reached. At the end, select node from terminal set. All leaves are at same level. Grow Method: Select nodes from the primitive set until maximum depth is reached. Ramped half-and-half: Half the initial population is constructed using full and half is constructed using grow. Depth limits of trees not fixed.

Initialization of population(Contd.)‏ Initialization need not be entirely random. If properties of the desired solution is known, produce trees accordingly. This creates intial bias in population for faster convergence.

Selection Strategies Tournament Selection: Best individual from a set of randomly selected individuals is chosen to be parent. Advantage: It rescales fitness among the population. A best program doesn't populate the next generation with its children reducing diversity. Disadvantage: A small difference in fitness gets amplified in next generation. Even though it is just marginally better than its competitior.

Selection Strategies(Contd.)‏ Fitness- Proportionate Selection: Select individuals based on their fitness values.

Breeding Next Generation Crossover, mutation and reproduction are main operators applied on parents to generate offsprings. Subtree Crossover: Create the offspring by replacing the subtree rooted at the crossover point in a copy of the first parent with a copy of the subtree rooted at the crossover point in the second parent. Reproduction: Simply copy an elite individual to next generation. Ensures passing of good genes.

Subtree Crossover

Crossover contd. Since most of the nodes of tree are leaves.Hence crossover results in replacement/exchange of small genetic material. John R. Koza suggested 90 times functions and 10 times terminals should be selected for crossover.

Mutation Subtree Mutation: Replace the mutation point by randomly generated tree. Point Mutation: Randomly select a node and replace it with another primitive of same arity.

Contraints on Operators:closure Type Consistency: Subtree crossover can mix and join nodes arbitrarily. Hence all functions used must be type consistent. i.e they return values of same type. Evaluation safety: Subtree after evaluation should produce a valid value to be used. For this, protected versions of functions used. (e.g. “/” with 2nd argument 0 gives 1 as its output).

Sufficiency Sufficiency means solution to the problem can be expressed through combinations of different primitives used. In general sufficiency cannot be guaranteed. In areas where theory is well developed, we can decide on primitives to be used.

Fitness Function Gives a measure of how good a computer program is at solving the given problem. Proper encoding of problem through fitness functions is important for GP's success. Measuring fitness of a given program means executing the program. Evaluation of program tree is done in depth first search order.

GP Parameters Control parameters: Population Size, N Maximum number of generations, Gmax Probability Pc of crossover Probability Pm of mutation Probability Pr of reproduction

Five Preparatory steps to set up GP Determining the set T of terminals. Determining the set F of functions. Determining the fitness measures. Determining the GP parameters. Determining the Termination criteria and result designation

Symbolic Regression using GP Symbolic regression: The process of mechanically creating a computer program that fit certain numerical data. We want to evolve an expression whose values match those of the quadratic polynomial x2 + x + 1 in the range [−1,1].

Independent variable X SYMBOLIC REGRESSION Independent variable X Dependent variable Y -1.00 1.00 -0.80 0.84 -0.60 0.76 -0.40 -0.20 0.00 0.20 1.24 0.40 1.56 0.60 1.96 0.80 2.44 3.00

Preparatory Steps Objective: Find a computer program with one input (independent variable X) whose output equals the given data 1 Terminal set: T = {X, Random-Constants} 2 Function set: F = {+, -, *, %} 3 Fitness: The sum of the absolute value of the differences between the candidate program’s output and the given data (computed over numerous values of the independent variable x from –1.0 to +1.0)‏ 4 Parameters: Population size M = 4 5 Termination: An individual emerges whose sum of absolute errors is less than 0.1

Symbolic Regression (initialization)‏ Generation 0 with 4 individuals Two parental chromosomes exchange part of their genetic information to create new hybrid combinations (recombinant). No loss of genes, but an exchange of genes between two previous chromosomes. No new genes created, preexisting old ones mixed together.

Symbolic regression(fitness)‏ x + 1 x2 + 1 2 x 0.67 1.00 1.70 2.67

Symbolic Regression (Generation1)‏ Reproduction: Copy of (a)‏

Symbolic Regression (Generation1)‏ Mutation of (c)‏: Picking “2” as mutation point

Symbolic Regression (Generation1)‏ Second offspring of crossover of (a) and (b)‏  picking “+” of parent (a) and left-most “x” of parent (b) as crossover points First offspring of crossover of (a) and (b) picking “+” of parent (a) and left-most “x” of parent (b) as crossover points

Symbolic Regression (Generation1)‏ Second offspring of crossover of (a) and (b)‏  picking “+” of parent (a) and left-most “x” of parent (b) as crossover points

Areas where GP will do well The interrelationships among the relevant variables is unknown or poorly understood. Finding the size and shape of the ultimate solution is a major part of the problem. Search domain is very large and solutions are sparsely distributed. There are good simulators to test the performance of tentative solutions to a problem, but poor methods to directly obtain good solutions.

Applications GP was used in designing of an antenna for deployment on NASA’s Space Technology 5 Mission. Designs generated are complex, non-intuitive, and 100% compliant with the mission antenna performance requirements.

Summary Genetic programming now routinely delivers high- return human-competitive machine intelligence. The AI ratio (the “artificial-to-intelligence” ratio) of a problem-solving method as the ratio of that which is delivered by the automated operation of the artificial method to the amount of intelligence that is supplied by the human applying the method to a particular problem. GP has very high AI ratio.

Summary Routine:A problem solving method is routine if it is general and relatively little human effort is required to get the method to successfully handle new problems within a particular domain and to successfully handle new problems from a different domain. Human-Competitiveness: Previously patented, an improvement over a patented invention, or patentable today. Publishable in its own right as a new scientific result.

Homework Experiments(1 or 2): 1. Implement GP for credit scoring problem 2. Implement GP for Stock market forecasting problem Read paper: GoldMiner:A Genetic Programming based algorithm applied to Brazilian Stock Market Zhang Defu, Hifi Mhand, Chen Qingshan, Ye Weiguo. A Hybrid Credit Scoring Model Based on Genetic Programming and Support Vector Machines. Fourth International Conference on Natural Computation 7(2008)8-12. Download website: https://www.researchgate.net/profile/Defu_Zhang3/contributions 55