1. Differential Evolution
A/Prof. Xiaodong Li
School of Computer Science and IT, RMIT University
Melbourne, Australia
March 2015
ACISS'09, Melbourne

2. Outline Background Basics about DE DE variants Perturbation
Contour matching
Rotation invariance
DE parameters
No free lunch theorem
Example questions after reading
25/06/2018

3. Background
Proposed by Kenneth Price and Rainer Storn in It has become increasingly popular in the optimization field.
A population-based stochastic method for global optimization.
One key feature is the use of the differential between two randomly chosen vectors.
Many DE variants have been developed.
25/06/2018

4. DE basics
25/06/2018

5. DE basics
25/06/2018

6. How to generate a mutant vector?
25/06/2018

7. DE basics
25/06/2018

8. The rule of thumb values
F is in [0.5, 1.0];
Cr is in [0.8, 1.0];
Np = 10 x D.
Adaptive schemes for these parameters have also been developed.
25/06/2018

9. Perturbation
25/06/2018

10. Basin to basin transfer
The vector population adapts such that promising regions of the objective function surface are investigated automatically once they are detected.
25/06/2018

11. Contour matching
25/06/2018

12. Contour matching
25/06/2018

13. Contour matching
25/06/2018

14. Rotation invariant
Quadratic Function
25/06/2018

15. Rotation invariant
Coordinate rotation causes the improvement interval to shrink. This is the rotated version of function in the previous slide. Point A on the level curves represent the same point before and after rotation. Note that in this figure, the global optimum is outside of the improvement window, which makes it much harder for an algorithm to locate the global optimum.
25/06/2018

16. Crossover destroys contour matching
25/06/2018

17. Crossover in different coordinate systems
25/06/2018

18. Dithering
25/06/2018

19. Jittering
25/06/2018

20. Problem domain characteristics
25/06/2018

21. Challenges in combinatorial problems
How to find a discrete operator that corresponds to the “difference vector” in the continuous domain?
The combination of a base vector and a difference vector (or recombination vector) yields a new valid vector.
In the travelling salesman problem (TSP), distances between every two cities may be utilized by DE.
The self-adaptivity of the vector difference distribution may be severely disturbed because a converged population still might exhibit large difference vectors.
Additional problem is that we face the heavy constraints inherent in the TSP.
25/06/2018

22. No Free Lunch Theorem
No free lunch theorem states that for certain types of mathematical problems, the computational cost of finding a solution, averaged over all problems in the class, is the same for any solution method. No solution therefore offers a 'short cut'.
Introduced by David Wolpert and William G. Macready.
This condition does not hold precisely in practice.
25/06/2018

23. Readings on DE
Rainer Storn and Kenneth Price (1997), "Differential Evolution - A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces", Journal of Global Optimization, 11: , 1997.
Storn, R. (2008). "Differential Evolution Research – Trends and Open Questions". Advances in Differential Evolution, SCI 143, pp. 1–31, 2008.
25/06/2018

24. Questions on DE
Example questions to ask, after reading the following paper:
Storn, R. (2008). "Differential Evolution Research – Trends and Open Questions". Advances in Differential Evolution, SCI 143, pp. 1–31, 2008.
What are the 5 constituents that define the original version of DE?
Discuss the different DE perturbation techniques and the effects they have.
What cost can a high level of crossover have on DE?
In DE what difference does using one-array or two-arrays make?
Discuss why DE has trouble solving combinatorial problems, refer to the travelling salesman problem.
25/06/2018