1. School of Computer Science & Engineering
Artificial Intelligence
Escaping Local Optima:
Simulated Annealing
Dae-Won Kim
School of Computer Science & Engineering
Chung-Ang University

2. We’ve discussed a few traditional problem-solving strategies

3. Some guarantee discovering the global solution, but too expensive

4. No chance to speed up algorithms that find the global solution

5. No chance of finding polynomial-time algorithms for real problems

6. Simple local search algorithms have a tendency of getting stuck in local optima
Pros: Ease of Implementation
Cons: Get stuck in poor local optima

7. They are NP-hard, thus remaining option is …

8. Designing algorithms that are capable of escaping local optima
Random restart
Increase the size of neighbors
Probabilistic approach

9. One possibility: iterative hill-climber

10. After reaching a local optimum, search is restarted with new starting

11. Today we’ll discuss two approaches:

12. (1) Simulated annealing: based on additional parameter (called temperature) that changes the probability of moving from one point of the search space to another

13. (2) Tabu search: based on memory, which forces the algorithm to explore new areas of the space

14. Key: newly generated neighbor doesn’t have to be a better solution.

15. Accept the new solution with some probability that depends on the relative merit of two solutions

16. This leads to stochastic hill-climber

17. procedure stochastic hill-climber
begin
t←0
select a current point vc at random
evaluate vc
repeat
create the neighbor vn from the neighborhood of vc
select vn with probability
t←t+1
until t=MAX
end

18. It has only one loop. We don’t have to repeat its iterations
Newly selected solution is accepted with probability p.

19. Moving from the current to the new neighbor is probabilistic

20. It’s possible for the new one to be worse than the current

21. The acceptance probability of the next point depends on two things.

22. Difference in merit: eval(vc)-eval(vn)
Constant parameter: T

23. Q: What is the role of p and T?

24. e.g. 1) for a maximization problem, assume eval(vc)=107, eval(vn)=120

25. In case of the hill-climber, we simply accept the new vn

26. In case of the stochastic hill-climber, it depends on p and T.

27. Let us consider some cases

28. T p 1
1.00 5
0.93 10
0.78 20
0.66 50
0.56
Infinite
0.50

29. The greater the value of T, the smaller the importance “eval(vc) - eval(vn)”

30. If T is huge, <p> approaches 0.5. The search becomes random.

31. If T is very small (e.g., T=1), it reverts to an ordinary hill-climber!

32. We have to find an appropriate T for a particular problem.

33. e.g. 2) suppose T=10, eval(vc)=107,
examine p as a function of eval(vn)

34. The behavior of p is clear
Eval(vn)
Difference p 80 27
0.06
100 7
0.33
107
0.50
120
-13
0.78
150
-43
0.99
The behavior of p is clear

35. What’s the difference between the stochastic hill-climber and simulated annealing?

36. Def: Annealing?

37. The SA changes T during the run

38. It starts with high values of T (like a random search), then gradually decreases T.

39. Towards the end of the run, T are quite small (like an ordinary hill-climber)

40. It escape local optima by allowing some bad moves, but gradually decrease their frequencies

41. Procedure simulated annealing
begin
t←0, initialize T
select a current point vc at random
evaluate vc
repeat
select the neighbor vn from the neighborhood of vc
if vn is better than vc then vc ← vn
else if random[0,1) < e^(Diff/T) then vc ← vn
until (termination-condition)
T ← g(T,t)
t ← t+1
end

42. SA is known as Monte Carlo annealing, statistical cooling, probabilistic hill-climbing, stochastic relaxation, and probabilistic exchange algorithm

43. SA raises additional questions:

44. How to determine the initial temperature T, the cooling ratio g(T,t)?
How to determine the termination conditions?

45. Tabu search

46. Tabu search uses a memory to force the search to explore new areas of the search space.

47. Memorize some solutions that have been examined recently,

48. These become tabu (forbidden) solutions to be avoided when selecting the next solutions.

49. Tabu seach is deterministic heuristic search.
Stochastic hill-climber and SA are probabilistic search techniques, thus the result could change at each execution

50. The best non-tabu solution is accepted for the next iteration

51. Tabu Search procedure Tabu-Search begin Initial tabu list
Generate randomly initial solution vc
evaluate vc
repeat
Generate all neighborhood solutions of the vc
Find best solution vn in the neighborhood
If vn is not tabu solution then vc=vn
else if ‘aspiration criteria’ is fulfilled then vc=vn
else find best not tabu solution in the neighborhood vn, and vc=vn
Update tabu list
until (terminate-condition)
end

52. Tabu Search Selection of solutions
The acceptance of solution for next iteration depends not only from its quality
The memory has also the impact in the selection process

53. Tabu Search Selection of solutions
Solution are classified in tabu and not tabu solutions
Usually the best non tabu solution is accepted for the next iteration

54. Tabu Search Selection of solutions Aspiration criteria
Tabu solution may be accepted if it fulfills
some conditions
Example: The tabu solution is the best solution so far

55. Tabu Search Tabu list Length of tabu list
For how many iteration should the solution be made
a tabu, which avoids searching cycles
Usually depends from size of problem
The length could also change during the search

56. Tabu Search Tabu list Length of tabu list
For how many iteration should the solution be made
a tabu, which avoids searching cycles
Usually depends from size of problem
The length could also change during the search

57. Two types of memories:
Recency-based memory
Frequency-based memory

58. Recency-based memory records some actions of the last few steps

59. Frequency-based memory shows the distribution of moves during the last steps

60. Example: Tabu search for an eight-city TSP
Example: Tabu search for an eight-city TSP. Consider moves that swap two cities in a particular solution
( )

61. We have 28 neighbors to swap

62. Memory is a matrix structure where the swap of cities i and j is recorded in the i-th row and j-th column.

63. Recency-based memory with five states can be given as:

64. city 1 2 3 4 5 6 7 8

65. What’s the meaning of M(2,6)=5?

66. The most recent swap was made for cities 2 and 6
The most recent swap was made for cities 2 and 6. The previous solution was ( ). Thus, swapping 2 and 6 is tabu for the next five iterations.

67. The swap between 1 and 4 is the oldest: it happened five steps ago.

68. Note that only 5 swaps (out of 28 possible swaps) are forbidden (tabu)

69. Frequency-based memory provides additional statistics of the search.

70. city 1 2 3 4 5 6 7 8

71. Swapping [7,8] was the most frequent (6 times in the last swaps)

72. Some pairs of cities (like 3 and 8) weren’t swapped: preferred candidates

73. Question for Tabu search:

74. Which information should be stored to possibly avoid the cycles or to better escape the local optima?

75. No doubt you can imagine many possibilities of the tabu design.

76. The more sophisticated the method, the more you have to use your judgment (parameters) as to how it should be utilized