Estimation of Distribution Algorithms (EDA) Siddhartha K. Shakya School of Computing. The Robert Gordon University Aberdeen, UK ss@comp.rgu.ac.uk

EDAs A novel paradigm in Evolutionary Algorithm Also known as Probabilistic model building Genetic Algorithms or Iterated density A probabilistic model based heuristic Motivated from the GA evolution More explicit evolution than the GA

Basic Concept of Solution and Fitness Graph colouring Problem: An Example Given a set of colours, GCP is to try and assign Colour to each nodes in such the way that neighbouring nodes will not have same colour

Basic concept of a solution and Fitness Given 2 colour Black = 0 White = 1 Representation of a solution as a chromosome Solution 1 a b d e f c 1 a b c d e f 1 fitness 1 a b d e f c 1 6

Chromosome and Fitness in GCP Chromosome: is a set of colours assigned to the nodes of graph. (there are other way of representing GCP in GA, such as order based representation). Fitness: is the number of correctly coloured nodes.

GA Iteration Initialisation of a “parent” population Evaluation Crossover Mutation Replace parent with “child” population and go to step 2 until termination criteria satisfies

GA Iteration 1 Selected Solution Initialization Evaluation 1 Selected Solution Initialization Evaluation 1 Parent population 2 4 3 fitness Crossover Selection After Crossover Repeat iteration 1 1 1 a b d e f c Given 2 colours (0,1) 1 After mutation 1 2 6 4 fitness 1 Mutation

GA evolution Selection drives evolution towards better solutions by giving a high pressure to the selection of high-quality solutions Crossover and mutation (Variation operator) together ensures the exploration of the possible space of the promising solutions. Maintains the variation in the population.

Variation in GA Evolution Has its limitation Can recombine fit solution to produce more fit solution Also can disrupt good solution and converge in local optimum

Estimation of Distribution Algorithm (EDA) To overcome the negative effective of the crossover and mutation approach of variation, a probabilistic approach of variation has been proposed. Algorithm using such approach is known as EDA (or PMBGA)

Probabilistic Model Building Sampling Child Population GA to EDA Simple GA framework Selection Crossover Mutation Evaluation Initial Population Initial Population Evaluation Selection Probabilistic Model Building Sampling Child Population EDA framework

General Notation EDA represents a solution as a set of value taken by a set of random variable. Chromosome is a set of value taken by set of random variables (Where each for bit representation) 1 Solution is a univariate marginal distribution is a conditional distribution is a joint probability distribution

Estimation of Probability distribution 1 Solution Usually it is not possible to calculate the joint probability distribution, so it is estimated. For example, assuming all are independent of each other, the joint probability distribution becomes the product of simple univariate marginal distribution.

Simple Univariate Estimation of Distribution Algorithm Initial Population 1 Solution Evaluation Selection Calculate univariate marginal probability and sample Child Population

Simple univariate EDA (UMDA) 1 Selected Solution Initialization Evaluation 1 Parent population 2 4 3 fitness Selection Repeat iteration Build model Estimation of Distribution 1 After mutation 1 2 6 4 fitness a b d e f c Given 2 colours (0,1) Calculate Distribution Sampling

Note It is not guaranteed that the above algorithm will give optimum solution for the graph colouring problem. The reason is obvious. The chromosome representation of GCP has dependency. i.e. node 1 taking black colour depends upon the colour of node 2. But univariate EDAs do not assume any dependency so it may fail. However, one could try

Complex Models To tackle problems where there is dependency between variables we need to consider more complex models. The extra model building step will be added to univariate EDA. Different algorithms has been purposed using different models They are categorised into three groups Univariate EDA Bivariate EDA Multivariate EDA

Univariate EDA Model x2 x1 x3 x5 x4 x7 x6 Graphical representation of probability model assuming no dependency among variables. (UMDA, PBIL, cGA)

Bivariate EDA Model a. Chain model (MMIC) b. Tree model (COMIT) c. Forest model (BMDA) Graphical representation of probability model assuming dependency of order two among variables.

Multivariate EDA Model Graphical representation of probability model considering multivariate dependency among variables. a. Marginal product model (ECGA) c. (BOA, EBNA) b. Triangular model (FDA)

Finding a probabilistic model Task of finding a good probabilistic model (finding the relationship between variable) is a optimization problem in itself. Most of the algorithm use Bayesian network to represent the probabilistic relationship. Two metric to measure the goodness of Bayesian Network. Bayesian Information Criterion (BIC) metric: Bayesian-Dirichlet (BD) metric: Use greedy heuristic to find a good model.

Summary EDA is an active area of research for GA community EDAs are reported to solve GA hard problems, and also hard optimization optimisation problems like MAX SAT. Success and failure of EDAs depends upon the accuracy of the used Probabilistic model.

Links http://cswww.essex.ac.uk/staff/zhang/MoldeBasedWeb/RGroup.htm (Research Groups working on EDAs) http://www.sc.ehu.es/ccwbayes/main.html (EDA homepage maintained by Intelligent system group). Books Larrañaga P., and Lozano J. A. (2001) Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. Kluwer Academic Publishers, 2001. Pelikan, M., (2002). Bayesian optimization algorithm: From single level to hierarchy. Ph.D. thesis, University of Illinois at Urbana-Champaign, Urbana, IL. Also IlliGAL Report No. 2002023.