Evolutionary Algorithms Nicolas Kruchten 4 th Year Engineering Science Infrastructure Option

What are they? EAs are stochastic techniques used to search multidimensional spaces for points, according to some criteria. They operate by mimicking reproduction & selection processes More on this in a bit…

What are they good for? EAs are used to solve optimization problems of various types: Find the vector x such that f(x) is min or max A huge variety of engineering or science problems can be reduced to, or expressed as, optimization problems

A black-box example Say you have a black box, with n knobs, and a light bulb, what set of knob positions will make the bulb glow brightest?

What if… n=1? n=10000? knobs are continuous? discrete? mixed? some combinations are invalid? (i.e. knob 1 must be less than knob 2) an analytical expression exists?

Traditional methods Direct Calculus Gradient descent or hill climbing Frank Wolfe (!) Simplex Simulated Annealing SPSA Etc.

Unfortunately they… Sometimes simply dont apply Only work for specific problems Dont scale well (NP-Hard, NP-Complete) Get trapped in local optima

EAs… Dont have all of these problems Scale reasonably well Work for a huge variety of problems Dont require gradients but can use them

How EAs work 1)Pick a population of feasible points 2)Generate new points from the better ones 3)Throw out the worse ones 4)Thats one generation: rinse & repeat Lo and behold! The points in your population are getting better and better!

Back to our example Our feasible points were sets of knob positions, we call these chromosomes Individual knob positions are genes The brightness we call the fitness, were looking for chromosomes with good fitness

How to get new points This is where the genetic comes in Given some parent chromosomes, how do we mix them up and get a child? Various schemes are possible depending on how we represent the parents (binary, real, integer etc)

Mutation If we only ever blend or mix and match genes, we can only ever explore combinations of our initial gene pool To explore new parts and find new optima, we periodically change, or mutate, genes. Again, various processes are possible.

Which parents to use Various schemes exist for selection: –Pick the best ones –Pick random ones –Pick them based on relative fitness –Pick them based on relative rank –Pick them based on how different they are –etc

Which points to keep Various schemes exist to decide which children and/or parents to keep at each generation –Keep only the children –Select from a pool of children and parents Best Random Rank Fitness, etc

When to stop You can stop an EA whenever is appropriate, but typical convergence criteria include: –A number of generations –A given standard deviation of fitnesses –A period of time –Total convergence –Etc.

Putting it all together Let t = 0; Initialize P(t) Evaluate C(t) Let t = t + 1; Assemble P(t) from P(t-1) and C(t-1) Stop? no End yes Evaluate P(t) Generate C(t) from P(t)

EA Limitations EAs are extremely flexible, but they have their own problems: –Can be very slow, requiring many potentially long evaluations –Are tough to design and configure, many many parameters –No guarantee of global optimum

Some history You may have heard of GAs, which are a subset of EAs Much confusion and conflicting terminology exists A huge body of operators, combinations and approaches have been proposed

Evolutionary Programming Traditional: –Credited to Fogel, US, published in 66 –Single parents (so not genetic) –Basically pure mutation Since then: –EP approaches themselves mutated independently towards general EA practice

Evolutionary Strategies Traditional: –Credited to Rechenberg and Schwefel, Switzerland, published in 73 –Tiny populations (1 or 2) –Sophisticated adaptive mutation scheme Since then: –Adaptive mutation has gone mainstream

Genetic Algorithms Traditional: –Credited to Holland, US, published in 72 –Binary coding –Keep only the children Since then: –Has grown to be loosely interchangable with EA, with real-coded GA, crowding GA etc

State of the art No hard and fast rules about the best EA flavour for a given problem Some heuristics exist EAs have been successfully applied to many problems in various fields, but with few general results

Advanced EAs Parallelization Distribution –Master/Slave –Peered Demes (see parallelization) –Hybrid Hybridization

ITS Applications Model calibration is a great example of a challenging transportation-related optimization: Minimize an error term by varying some model parameters

ITS Example: GAID Genetic Adaptive Incident Detection Use of a GA to train a neural network: perfect example of ANN + EA = ITS The problem: automatically detecting incidents on the freeway, using traffic loop data

GAID: Basics GAID uses a probabilistic neural network (PNN) which takes 16d vectors and classifies them into two piles Traffic loop data -> incident/normal The PNN is defined by 16 parameters

GAID: Diagram PNN = f(parameters) Loop Data Incident Normal 16 Parameters EA

GAID: Fitness Fitness: correct classifications/number of vectors We quiz the PNN with known data to score it We are looking for the set of 16 real parameters which make GAID the most accurate

GAID: Initialization Initialization: –Pick population size: 50 –Pick 50 random sets of 16 real parameters between 0 and 1000 –Score the chromosomes

GAID: Generation Generation: –Pick the child population (generation) size: 50 –Select 2 parents at random –Each child gene is either mom or dads –Mutate every gene: add some uniformly distributed random number between -50 and 50 –Score the children

GAID: 2d view

One alternative

GAID: Assembly Assembly of children and parents: –Choose the 50 best from the combined pool of children and parents Convergence: repeat, stopping when the STD is less than 0.005

GAID: Summary Real representation, 16 genes Popsize = Gensize = 50 Random Initialization 2 parents, random selection real crossover, creep mutation on all genes Crowding assembly, best selection STD convergence