Differential Evolution A/Prof. Xiaodong Li School of Computer Science and IT, RMIT University Melbourne, Australia Email: xiaodong.li@rmit.edu.au March 2015 ACISS'09, Melbourne
Outline Background Basics about DE DE variants Perturbation Contour matching Rotation invariance DE parameters No free lunch theorem Example questions after reading 25/06/2018
Background Proposed by Kenneth Price and Rainer Storn in 1995. It has become increasingly popular in the optimization field. A population-based stochastic method for global optimization. One key feature is the use of the differential between two randomly chosen vectors. Many DE variants have been developed. 25/06/2018
DE basics 25/06/2018
DE basics 25/06/2018
How to generate a mutant vector? 25/06/2018
DE basics 25/06/2018
The rule of thumb values F is in [0.5, 1.0]; Cr is in [0.8, 1.0]; Np = 10 x D. Adaptive schemes for these parameters have also been developed. 25/06/2018
Perturbation 25/06/2018
Basin to basin transfer The vector population adapts such that promising regions of the objective function surface are investigated automatically once they are detected. 25/06/2018
Contour matching 25/06/2018
Contour matching 25/06/2018
Contour matching 25/06/2018
Rotation invariant Quadratic Function 25/06/2018
Rotation invariant Coordinate rotation causes the improvement interval to shrink. This is the rotated version of function in the previous slide. Point A on the level curves represent the same point before and after rotation. Note that in this figure, the global optimum is outside of the improvement window, which makes it much harder for an algorithm to locate the global optimum. 25/06/2018
Crossover destroys contour matching 25/06/2018
Crossover in different coordinate systems 25/06/2018
Dithering 25/06/2018
Jittering 25/06/2018
Problem domain characteristics 25/06/2018
Challenges in combinatorial problems How to find a discrete operator that corresponds to the “difference vector” in the continuous domain? The combination of a base vector and a difference vector (or recombination vector) yields a new valid vector. In the travelling salesman problem (TSP), distances between every two cities may be utilized by DE. The self-adaptivity of the vector difference distribution may be severely disturbed because a converged population still might exhibit large difference vectors. Additional problem is that we face the heavy constraints inherent in the TSP. 25/06/2018
No Free Lunch Theorem No free lunch theorem states that for certain types of mathematical problems, the computational cost of finding a solution, averaged over all problems in the class, is the same for any solution method. No solution therefore offers a 'short cut'. Introduced by David Wolpert and William G. Macready. This condition does not hold precisely in practice. 25/06/2018
Readings on DE Rainer Storn and Kenneth Price (1997), "Differential Evolution - A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces", Journal of Global Optimization, 11:341-359, 1997. Storn, R. (2008). "Differential Evolution Research – Trends and Open Questions". Advances in Differential Evolution, SCI 143, pp. 1–31, 2008. 25/06/2018
Questions on DE Example questions to ask, after reading the following paper: Storn, R. (2008). "Differential Evolution Research – Trends and Open Questions". Advances in Differential Evolution, SCI 143, pp. 1–31, 2008. What are the 5 constituents that define the original version of DE? Discuss the different DE perturbation techniques and the effects they have. What cost can a high level of crossover have on DE? In DE what difference does using one-array or two-arrays make? Discuss why DE has trouble solving combinatorial problems, refer to the travelling salesman problem. 25/06/2018