Population Based Incremental Learning Shumeet Baluja Presented by KC Tsui

Background Populations based search, such as GA –Create a probability matrix by counting the number of 1s and 0s in each gene position –Generate new population using the probability matrix –No information is carried from generation to generation! Supervised Competitive learning, e.g. LVQ –Winner-take-all reinforcement learning in ANN –Winner is a kind of prototype of the sample presented PBIL = GA + CL –Capture the trend from the best performer

Basic PBIL P  initialize probability vector (each position = 0.5) while (generations++ < limit) for each vector i do for each position j do generate V i (j) according to P(j) end-do evaluate f(V i ) end-do V max = max(f(V i )) update P according to V max if random(0,1] < P mutate mutate P end-if end-while

Update and Mutation Rules Update Rule Mutation Rule –P mutate = 0.02 –  = 0.05

Extensions Learning from M top scorer: Adapt the probability matrix –equally according to the M vectors, or –where there is consensus in, or most, of the M vectors –according to the rank of the M top vectors Learning also from negative sample –move away the worst vector –modify only those positions where the best and the worst disagree

Some applications Function optimization Job-shop scheduling TSP Bin-packing Knapsack Problem Neural Network weight training

Some Comparison PBILEDOEA populationFixed sizeVariable sizeFixed size Probability matrix OneOne per parentNone parametersfixedAdaptiveCan be adaptive learningPos + neg pos diversityhighVery highGradually decreasing Search scopeLocal Global & local

References Shumeet Baluja, Population-Based Incremental Learning: a method for integrating genetic search based function optimization and competitive learning, Technical report CMU-CS Shumeet Baluja & Rich Caruana, Removing the Genetics from the Standard Genetic Algorithm, ICML’95. Shumeet Baluja, An Empirical Comparison of Seven Iterative and Evolutionary Function Optimization Heuristics, Technical Report CMU-Cs , 1995.