A study of simulated annealing variants Ana Pereira Polytechnic Institute of Braganca, Portugal Edite Fernandes University of Minho, Braga, Portugal SEIO’04 Cádiz, October 25-29, 2004

2 A study of simulated annealing variants Outline Motivation Simulated annealing algorithm Simulated annealing variants Computational results – Characterization of the presented variants Computational results – Comparison of the presented variants Simulated annealing Conclusions

3 A study of simulated annealing variants Motivation We aim to find the global solution of the nonlinear optimization problem Numerical methods  Deterministic methods  Stochastic methods Examples: Multistart, clustering, genetic algorithms, simulated annealing ….

4 A study of simulated annealing variants Simulated annealing In 1953, Metropolis proposed an algorithm to simulate the behavior of physical systems in the presence of a heat bath. In 1983, based on ideas of Metropolis algorithm, Kirkpatrick, Gelatt and Vecchi, and in 1985 Cerny, proposed the simulated annealing (SA) algorithm to solve combinatorial optimization problems. In 1986, Bohachevsky, Johnson and Stein applied the SA algorithm to solve continuous optimization problems. Since then, the SA algorithm has been subject to various modifications and has been applied in many areas such as graph coloring, circuit design, data analysis, image reconstruction, biology, …

5 A study of simulated annealing variants Simulated annealing Advantages: Easily implemented. Can be applied to any optimization problem. Does not use derivative information. Does not require specific conditions on the objective function. Asymptotically converges to a global maximum. Disadvantages: Requires a great number of function evaluations. Many authors have been proposing variants of the SA algorithm.

6 A study of simulated annealing variants Given an initial approximation, a control parameter and the number of iterations with the same control parameter while stopping criterion is not reached do Generate a new candidate point y Analyze the acceptance criterion end Update Reduce the control parameter end Simulated annealing algorithm

7 A study of simulated annealing variants Simulated annealing algorithm Generation of a new candidate point The new point is found using the current approximation,, and the generating probability density function,. Acceptance criterion is the acceptance function and it represents the probability of accepting the point y when is the current point. The acceptance criterion has the following form The most used acceptance function is

8 A study of simulated annealing variants Simulated annealing algorithm Reduction of the control parameter The function is called the control parameter and must be a decreasing function that verifies Stopping criterion The stopping criterion is based on the idea that the algorithm should terminate when no changes occur. We propose that the algorithm stops when successive approximations to a global maximum are similar, i.e, the algorithm stops if the following condition is verified for successive iterations where represents the previous approximation to an optimum value.

9 A study of simulated annealing variants Simulated annealing variants Standard SA variant (SSA) Generation of a new point Reduction of the control parameter Length of the chain: constant

10 A study of simulated annealing variants Simulated annealing variants Corana SA variant (CSA) Generation of a new point Reduction of the control parameter Length of the chain: constant

11 A study of simulated annealing variants ASA variant Main difference : There are two control parameters: one associated with the generation of the new points and another associated with the acceptance function. And it is possible the redefinition of the control parameters. Simulated annealing variants

12 A study of simulated annealing variants ASA variant Generation of a new point Redefinition of the control parameters Reduction of the control parameters Length of the chain: 1 Simulated annealing variants

13 A study of simulated annealing variants SALO variant Similar to ASA algorithm except on generation of a new point. Generation of a new point Simulated annealing variants

14 A study of simulated annealing variants ASALO variant Based on ASA and SALO algorithms. Generation of a new point Like ASA algorithm. If y is infeasible, applies the reflection technique proposed by Romeijn and Smith If the new point y is accepted Simulated annealing variants

15 A study of simulated annealing variants SSA CSA Computational results – Characterization of the presented variants Crucial parameters:

16 A study of simulated annealing variants ASA SALO ASALO Computational results – Characterization of the presented variants Crucial parameters:

17 A study of simulated annealing variants Computational results – Comparison of the presented variants Test functionsnN FE N AP B GP R2R R4R Me H3H Ra ASA algorithm ASALO algorithm Test functionsnN FE N AP B GP R2R R4R Me H3H Ra

18 A study of simulated annealing variants Conclusions SSA variant requires a large number of function evaluations. SSA and CSA variants recognize more than one solution, when the problem has more than one global maximum. CSA is the variant that has more accepted points. ASA variant requires a reduced number of function evaluations.

19 A study of simulated annealing variants Conclusions ASA variant did not converge to a global maximum in some runs of some problems. CSA and ASALO variants provide better approximations to the global maximum. SALO and ASALO variants have a small number of accepted points. CSA and ASALO variants did not converge to a global maximum only in one run of a problem.

20 A study of simulated annealing variants Ana Pereira Polytechnic Institute of Braganca, Portugal Edite Fernandes University of Minho, Braga, Portugal