1. FDA – A Scalable Evolutionary Algorithm for the Optimization of Additively Decomposed Functions
BISCuit EDA Seminar

2. Introduction Optimization of additively decomposed functions (ADFs)
How to decompose a given function? – a factorization problem
Bayesian network
Optimization by discovering distribution of good solutions.
Distribution model – Boltzmann distribution
Selecting good solutions to build model
Theoretical and empirical results

3. Factorization Theorem
Additively decomposed function (ADF)
Boltzmann distribution
The larger the function value f(x), the larger p(x).

4. Factorization Theorem
Given partition of variables, the joint distribution can be factorized into a product of marginal and conditional probabilities.

5. The Factorized Distribution Algorithm (FDA)
Sampling
Selection
Build Model

6. The Factorized Distribution Algorithm (FDA) – Factorization Algorithm
Can compute an exact factorization for simple structures (chains, trees).
Computes an approximation factorization for complex structures (rings, torus).

7. How to generate initial population?
The factorization is computed beforehand.
Conditional probabilities are computed using the local fitness functions only.
The steepness of the distribution (favor of better solutions) is adjusted so that

8. How to generate initial population?
Example of OneMax
Factorization
Span = 1, thus u = 10.
It might not give a Boltzmann distribution. Therefore, half of the population is generated in this way, and the other half is generated randomly.
For each site xi, 1 is more favored 10 times than 0.

9. Does FDA Converge?
Given the distribution of the selected individuals as
At each generation, the dist. of selected individuals follow Boltzmann dist.
In the end, the dist. of selected individuals converge to uniform dist. over optimal solutions.

10. Theoretical Analysis for Infinite Populations
Target functions
Factorization

11. Theoretical Analysis for Infinite Populations

12. Theoretical Analysis for Infinite Populations
Truncation
Boltzmann

13. For infinite populations, strongest selection is the best
For infinite populations, strongest selection is the best. However, the optimal annealing schedule is very difficult for finite populations.
For OneMax, FDA with truncation selection can generate a Boltzmann distribution.
Theorem 6, 7, 8 gives the maximum generation for convergence for Int problem.

14. Analysis of FDA for Finite Populations
Due to the heavy dependency of Boltzmann selection to annealing schedule, EDA with truncation selection will be considered.

15. Analysis of FDA for Finite Populations

16. Analysis of FDA for Finite Populations

17. Numerical Results Generations until convergence f2: order-3 OneMax
f3(1,1,1) = 10, Otherwise, 0.

18. Numerical Results

19. Numerical Results

20. LFDA – Computing a Bayes Factorization
Given a population of selected points, what is a good Bayes factorization fitting the data?
Minimum description length (MDL) score
Bayesian Information Criterion

21. LFDA – Computing a Bayes Factorization
Finding Bayesian network structure (greedy)
Due to the search for structure, LFDA is computationally very expensive!