1. Scientific Research Group in Egypt (SRGE)
Meta-heuristics techniques (II)
Simulated annealing
Scientific Research Group in Egypt (SRGE)
Dr. Ahmed Fouad Ali
Suez Canal University,
Dept. of Computer Science, Faculty of Computers and informatics
Member of the Scientific Research Group in Egypt

2. Scientific Research Group in Egypt

3. Meta-heuristics techniques

4. Outline 1. What is annealing? 2. Motivation
3. Simulated annealing (SA)(Background)
4. SA (main concepts)
5. SA algorithm
6. SA control parameters
7. SA example
8. SA applications

5. What is annealing? Annealing is a physical process used
to harden metals, glass, etc.
Starts at a high temperature and
cools slowly.
Lower temperature of the glass slowly so that at each temperature the atoms can move just enough to begin adopting the most stable orientation.

6. What is annealing?
The temperature determines how much mobility the atoms have.
How slowly you cool glass is critical.
If the glass is cooled slowly enough, the atoms are able to "relax" into the most stable orientation.
For example, the glass surrounding a hockey rink.

7. Motivation ? barrier to local search starting point descend direction
local minima
global minima

8. Simulated annealing (SA)(Background)
The method was independently described by Scott Kirkpatrick, C. Daniel Gelatt and Mario P. Vecchi in 1983, and by Vlado Černý in 1985.
SA is probably the most widely used meta-heuristic in combinatorial optimization problem.
It was motivated by the analogy between the physical annealing of metals and the process of searching for the optimal solution in a combinatorial optimization problem.
The main objective of SA method is to escape from local optima and so to delay the convergence.

9. Simulated annealing (main concepts)
SA proceeds in several iterations from an initial solution x0.
At each iteration, a random neighbor solution x' is generated.
The neighbor solution that improves the cost function is always accepted.
Otherwise, the neighbor solution is selected with a given probability that depends on the current temperature T and the amount of degradation E of the objective function.
E represents the difference in the objective value between the current solution x and the generated neighboring solution x'.

10. Simulated annealing (main concepts)
The probability follows, in general, the Boltzmann distribution as shown in the following equation
Many trial solutions are generated as an exploration process at a particular level of temperature.
The temperature is updated until stopping criteria satisfied.

11. Simulated annealing algorithm
Outer
loop
Inner
loop

12. SA control parameters Initial temperature
Choosing too high temperature will cost computation time expensively.
Too low temperature will exclude the possibility of ascent steps, thus losing the global optimization feature of the method.
We have to balance between these two extreme procedures.

13. SA control parameters Choice of the temperature reduction strategy
If the temperature is decreased slowly, better solutions are obtained but with a more significant computation time.
A fast decrement rule causes increasing the probability of being trapped in a local minimum.

14. SA control parameters Equilibrium State.
In order to reach an equilibrium state at each temperature, a number of sufficient transitions (moves) must be applied.
The number of iterations must be set according to the size of the problem instance and particularly proportional to the neighborhood size.

15. SA control parameters Stopping criterion
Concerning the stopping condition, theory suggests a final temperature equal to 0.
In practice, one can stop the search when the probability of accepting a move is negligible, or reaching a final temperature TF.

16. 1||SwjTj
SA examples
Jobs 1 2 3 4 wj 5 pj 12 8 15 9 dj 16 26 25 27

17. Iteration 1 SA examples Step 1. S0=S1=(1,3,2,4). G(S1)=136.
Let T=10 and a=0.9 => b1=9
Step 2. Select randomly which jobs to swap, suppose a Uniform(0,1) random number is V1= 0.24 => swap first two jobs
Sc=(3,1,2,4), G(Sc)=174, P(Sk,Sc)=1.5%
U1=0.91 => Reject Sc
Step 3. Let k=2

18. SA examples
Iteration 2
Step 2. Select randomly which jobs to swap, suppose a Uniform(0,1) random number is V2= 0.46 => swap 2nd and 3rd jobs
Sc=(1,2,3,4), G(Sc)=115
S3=S0=Sc
Step 3. Let k=3

19. SA examples
Iteration 3
Step 2. V3= 0.88 => swap jobs in 3rd and 4th position
Sc=(1,2,4,3), G(Sc)=67
S4=S0=Sc
Step 3. Let k=4

20. SA examples
Iteration 4
Step 2. V4= 0.49 => swap jobs in 2nd and 3rd position
Sc=(1,4,2,3), G(Sc)=72, b4=10(0.9)4=6.6
P(Sk,Sc)=47%, U4=.90 => S5=S4
Step 3. Let k=5

21. Iteration 5 SA examples Step 2. V5= 0.11 => swap 1st and 2nd jobs
Sc=(2,1,4,3), G(Sc)=83, b5=10(0.9)5=5.9
P(Sk,Sc)=7%, U5=0.61 => S6=S5
Step 3. Let k=6

22. SA Applications Scheduling Quadratic assignment Frequency assignment
Car pooling
Capacitated p-median,
Resource constrained project scheduling (RCPSP)
Vehicle routing problems
Graph coloring
Retrieval Layout Problem
Maximum Clique Problem,
Traveling Salesman Problems
Database systems
Nurse Rostering Problem
Neural Nets
Grammatical inference,
Knapsack problems
SAT Constraint Satisfaction Problems
Network design
Telecomunication Network
Global Optimization

23. References
Metaheuristics From design to implementation, El-Ghazali Talbi, University of Lille – CNRS – INRIA.
S. Kirkpatrick, C. Gelatt, M. Vecchi, Optimization by simulated annealing, Science 220 (1983),

24. Thank you