Mathematical Models of GAs Notes from * Chapter 4 of Mitchell’s An Intro. to GAs * Neal’s Research CS 536 – Spring 2006

GA Theory ? Why GA theory? Because the GA is a black box. Serious/Organized GA theory research is relatively new FOGA 1 in 1990 GAs became fairly popular in the 80s (first ICGA in 1985) Note that mathematical theory of biological evolution has been around since at least 1916 (first issue of Journal of Genetics) Various High-level classifications of EA theory exist, here is one taxonomy: Schema Thoery Markov Models Vose Models Statistical Mechanics Perturbation Models There is No Free Lunch

No Free Lunch for Genetic Algorithms. Given any two optimization algorithms, their performance is exactly equal over the total space of all possible functions to be optimized. Not Possible to make statements like “my mousetrap is provably better then yours for all mice.” No Free Lunch

Markov Models of GAs One of the first descriptions was by Nix and Vose. Builds a probabalistic model of GA behavior. U is Z x Z matrix of where For 10 bits and 10 individuals that's 3 * Population at time t+1, p(t+1) is defined by p(t+1) = U * p(t)

Example Markov Model 2-bit mutation only GA, single genome, mu = 0.1 Single Individual

Dortmund Models A group of researchers from University of Dortmund are actively researching simple EAs and building Markov models of them. The (1+1) EA 1) Choose mutation rate p m ε (0, ½] 2) Choose x ε {0,1} n uniformly at random. 3) Create y by flipping each bit x independently with p m 4) If f(y) >= f(x), set x := y 5) Continue at line 3 The preceding slide is a (1,1) EA or Random Walk (Drunkards Walk)

Example (1+1) EA Model

Metropolis Selection This is a modified (1+1) EA with Metropolis selection The (1+1) Metropolis EA 1) Choose mutation rate p m ε (0, ½] 2) Choose α ε (1, ∞] 3) Choose x ε {0,1} n uniformly at random. 4) Create y by flipping each bit x independently with p m 5) With If f(y) >= f(x), set x := y 6) Else set x:=y with probability 1/ α f(x)-f(y) 7) Continue at line 3 This EA accepts 'worsenings' with some (usually small probability) If alpha is dependent on t (non-constant) it is a simulated annealing algorithm

Metropolis EA Model

(1+1) EA with Cyclic Mutation This is a modified (1+1) EA with Cyclic mutation operator The (1+1) EA 1) Choose mutation rate p m ε (0, n] 2) Choose x ε {0,1} n uniformly at random. 3) Create y by flipping each bit x independently with p m 4) If f(y) >= f(x), set x := y 5) p m := 2 * p m if p m > 1/2, set p m := 1/n 6) Continue at line 3 Note: This EA is a provably better performer on some fitness functions than the classic (1+1) EA.

(2+1) EA with Crossover This is a simple steady-state GA with crossover with the smallest possible population. The (2+1) EA w/ crossover 1) Choose mutation rate p m ε (0, ½] 2) Choose Population P :={x,y} where x,y ε {0,1} n uniformly at random. 3) Search With prob 1/3, z is created by mutate(x) With prob 1/3, z is created by mutate(y) With prob 1/3, z is created by mutate(crossover(x,y)) 4) Create P := {x,y,z} – {a} where a be the worst fitness individual. 5) Continue at line 3

Proofs Expected running time of (1+1) EA on binary functions is AT MOST n n Expected running time of (1+1) EA on ONEMAX O(n ln n) Expected running time of (1+1) EA on binary functions is O(4 n log n) The (2+1) Crossover-EA can outperform the (1+1) EA [or (2+1) EA] on some Royal Road functions.

Part 2 - Monday Monday: 1)2-Armed Bandit 2)Go over Schema Theorem 3)Talk about Royal Road Functions 4)Talk about Vose’s Infinite Population Model

Exploitation vs. Exploration John Holland’s invention of GAs: – Meant as implementation of a proposed general principle for adaptation in complex systems: Adaptation requires the correct balance between “exploitation” and “exploration” –“Exploitation” Adaptation consists of spreading useful traits once they are discovered –“Exploration” Adaptation also consists of “searching” for new useful traits

Two-Armed Bandit Problem You are given n quarters to play with, and don’t know the average payoffs of the respective arms. What is the optimal way to allocate your quarters between the two arms so as to maximize your earnings (or minimize your losses) over the n arm-pulls ? What is the relationship to schemas?

Two-Armed Bandit problem Slot machine has two arms, A 1 and A 2, with mean payoffs  1 and  2, with variances  1 and  2. Payoff processes stationary and independent Gambler given N coins. Goal: maximize payoff. Doesn’t know  s or  s. Must estimate by playing coins on arms. What is optimal strategy for allocating trials to arms? –Needs to both gather information and use it at the same time. –“On-line learning”: Payoff at each trial counts in performance.

In Holland’s theory, each arm roughly corresponds with a possible “strategy” to test. The question is, if one strategy (arm) seems good, how much time should you spend exploiting it, and how much time should you spend exploring other, possibly better, strategies? Holland claims the GA explores schemata via ‘implicit parallelism’ How does this relate to Schemas?

Chalkboard Discussion of Schema Theorem Example Schema Schema Theorem Simplified Version Interpretation Counter Example Building Block Hypothesis

Critiques Schema Theorem Muhelenbein: “..the Schema Theorem is almost a tautology, only describing proportional selection..” A bit unfair.. Nothing has appeared to challenge the mathematics.. only the assumptions are challenged. Vose showed that a tiny change in the mutation rate can cause a large change in the GAs trajectory. Butterfly effect – hallmark of non-linear dynamic systems Stochastic & dynamic nature of equation is ignored, pushing equation beyond sustainability See board Fundamentally NOT predictive of real GA behavior. Must track all schemata to make real predictions on trajectory

Royal Road Function

More Critiques Schema Theorem Rudolph showed 2-armed-bandit analogy fundamentally breaks down. Holland’s ‘optimal’ strategy is far outperformed by an alternative approach Macready & Wolpert use a Bayesian framework to argue that the ‘optimal strategy’ is no optimal. Even if we accept that the GA obeys the ‘exponentially increasing trials’ of the theorem, this is NOT the optimal way to solve a competition between schemata. The assumption that hyperplane/schemata competitions can be isolated and solved independently is false. Building Block Hypothesis: failed to predict performance on Royal Road functions.. Outperformed by (1+1) ie no crossover. Niether the S.T. or the BBH are recursive equations that can be iterated and solved like they have been. They are ‘expectations’.. Ie stochastic/random.

Vose's Simple Genetic Algorithm & Model 1.Randomly initialize population 2.Select two individuals from population via selection function 3.Combine individuals via crossover function 4.Mutate child via mutation function 5.Place mutated child into next generation's population 2 Until next population is full In the next few slides we will delete step 3 – no crossover

Vose Infinite Population Model Discrete Dynamical System (Called a Map) Map input population to output population Population represented as 'vector of proportions' 2-bit genome example: p = (0.1, 0.2, 0.5, 0.2) Size of vector is s = 2 d (d is length of binary chromosome string) Elements of vector are in [0,1] and sum to 1(Simplex Property) Fitness Vector is f = (f(x 0 ), f(x 1 ),..., f(x s-1 )) where f(x k ) is fitness of kth individual

Vose Infinite Population Model (2)

Vose Infinite Population Model (3) From Dynamic Systems and GA theory we know: US is a positive matrix: all entries are non-negative. Only one normalized eigenvector is in the simplex (via Perron-Frobenius) Eigenvalues of US are the average fitness of the population given by the corresponding eigenvector. The largest eigenvalue corresponds with the lone eigenvector inside the simplex. Output of G(p) is the 'expected' next population of a real GA with a very large population Fixed point is the 'expected' long term population of a real GA with very large number of generations

Vose Infinite Population Model (4) Example: f(00)=3 f(01)=2 f(10)=1 f(11)=4q=0.1 S = U = US = Fixed-points are population vectors such that:p = G(p) Eigenvectors of USEigenvalues of US ( )3.29 ( )2.48 ( )1.53 ( )0.78

Results of Markov Chain Model Nix & Vose, 1991 Nix and Vose used the theory of Markov chains to show: –For large n, trajectories of the Markov chain converge to iterates of G (infinite population model) with probability arbitrarily close to 1. –For large n, if G has a single fixed point, the GA asymptotically spends all of its time at that fixed point. Extended models (Vose, 1993; 2001): –Short-term GA behavior: dominated by initial population –Long-term GA behavior: determined only by structure of the GA surface

Problems with Exact Models In principle, can be used to predict every aspect of GA behavior. In practice? –Required matricies are intractably large –View is too microscopic –Need to reduce dimensionality