An Introduction to Simulated Annealing Kevin Cannons November 24, 2005

Cannons1 Presentation Goals ●Motivate the importance of simulated annealing (SA) ●Explain the simulated annealing algorithm ●Provide an example of solving a real-world problem with SA

Cannons2 Motivation ●Optimization problems are a very important and practical class of problems ●The best known algorithms for many optimization problems run in exponential time ●Simulated annealing can provide decent solutions to numerous difficult problems in reasonable time ► Travelling salesman problem ► Graph coloring problem ► And many more

Cannons3 Optimization Problems Background Cost Solution

Cannons4 Potential Solution Techniques ●Analyze every potential solution? ► Often an exponential number of solutions Cost Solution

Cannons5 Potential Solution Techniques ●Hill Climbing/Iterative Improvement? ► Algorithm: ■Start with a random solution ■Analyze nearby solutions and accept a solution in the neighbourhood if it has a lower cost than the current solution ■Stop when the current solution has no neighbours with a lower cost Cost Solution

Cannons6 Potential Solution Techniques ●Hill Climbing/Iterative Improvement? ► Algorithm: ■Start with a random solution ■Analyze nearby solutions and accept a solution in the neighbourhood if it has a lower cost than the current solution ■Stop when the current solution has no neighbours with a lower cost Cost Solution Initial Solution

Cannons7 Potential Solution Techniques ●Hill Climbing/Iterative Improvement? ► Algorithm: ■Start with a random solution ■Analyze nearby solutions and accept a solution in the neighbourhood if it has a lower cost than the current solution ■Stop when the current solution has no neighbours with a lower cost Cost Solution

Cannons8 Potential Solution Techniques ●Hill Climbing/Iterative Improvement? ► Algorithm: ■Start with a random solution ■Analyze nearby solutions and accept a solution in the neighbourhood if it has a lower cost than the current solution ■Stop when the current solution has no neighbours with a lower cost Cost Solution

Cannons9 Potential Solution Techniques ●Hill Climbing/Iterative Improvement? ► Algorithm: ■Start with a random solution ■Analyze nearby solutions and accept a solution in the neighbourhood if it has a lower cost than the current solution ■Stop when the current solution has no neighbours with a lower cost Cost Solution

Cannons10 Potential Solution Techniques ●Hill Climbing/Iterative Improvement? ► Algorithm: ■Start with a random solution ■Analyze nearby solutions and accept a solution in the neighbourhood if it has a lower cost than the current solution ■Stop when the current solution has no neighbours with a lower cost Cost Solution Final Solution

Cannons11 Potential Solution Techniques ●Hill Climbing/Iterative Improvement? ► Algorithm: ■Start with a random solution ■Analyze nearby solutions and accept a solution in the neighbourhood if it has a lower cost than the current solution ■Stop when the current solution has no neighbours with a lower cost Cost Solution Initial Solution

Cannons12 Potential Solution Techniques ●Hill Climbing/Iterative Improvement? ► Algorithm: ■Start with a random solution ■Analyze nearby solutions and accept a solution in the neighbourhood if it has a lower cost than the current solution ■Stop when the current solution has no neighbours with a lower cost Cost Solution

Cannons13 Optimization Problems ●Hill Climbing/Iterative Improvement? ► Algorithm: ■Start with a random solution ■Analyze nearby solutions and accept a solution in the neighbourhood if it has a lower cost than the current solution ■Stop when the current solution has no neighbours with a lower cost Cost Solution Final Solution

Cannons14 Optimization Problems ●Hill Climbing/Iterative Improvement? ► Disadvantages: ■Gets trapped in local minima ■Identified local minimum depends on the initial solution ► Improvements? ■Run the algorithm several times with different starting points ■Occasionally allow transitions to solutions which increase the cost Cost Solution Second Minimum Found First Minimum Found

Cannons15 Optimization Problems ●Hill Climbing/Iterative Improvement? ► Disadvantages: ■Gets trapped in local minima ■Identified local minimum depends on the initial solution ► Improvements? ■Run the algorithm several times with different starting points ■Occasionally allow transitions to solutions which increase the cost Cost Solution

Cannons16 Optimization Problems ●Hill Climbing/Iterative Improvement? ► Disadvantages: ■Gets trapped in local minima ■Identified local minimum depends on the initial solution ► Improvements? ■Run the algorithm several times with different starting points ■Occasionally allow transitions to solutions which increase the cost

Cannons17 What is Simulated Annealing? ●General purpose optimization technique ► Introduced by Kirkpatrick et al in 1983 ●Converges asymptotically to a global minimum ► Not as impressive as it sounds ■With infinite time, a global minimum could be found with a hill climbing method ► Does work well in practise for many problems ■Final solution typically has a cost close to the true minimum ●Inspired by the physical annealing of solids ► Temperature increased to a level where the solid melts ► Temperature is then slowly decreased until the substance reaches a low energy state

Cannons18 Metropolis Algorithm ●At the heart of the SA technique ●Developed by Metropolis et al in 1953 ●Models the movement of atoms as they evolve towards equilibrium at a particular temperature

Cannons19 Metropolis Algorithm ●Ingredients: ► i = current solution ► n = proposed solution ► E k = energy (or cost) of solution k ►  E = E n - E i ► T = current temperature

Cannons20 Metropolis Algorithm ●Simple, three-step algorithm: 1. Generate solution n by making a small change to solution i 2. Compute  E = E n - E i 3. if  E < 0 then P accept = 1 else P accept = ●Above steps are performed Q times ► If Q = , system will reach equilibrium Ingredients: i = current solution n = proposed solution E k = energy (or cost) of solution k  E = E n - E i T = current temperature

Cannons21 Metropolis Algorithm ●What does the P accept = function look like?

Cannons22 Metropolis Algorithm ●What does the P accept = function look like?

Cannons23 Metropolis Algorithm ●What does the P accept = function look like?

Cannons24 Metropolis Algorithm ●What does the P accept = function look like?

Cannons25 Metropolis Algorithm ●What does the P accept = function look like?

Cannons26 Simulated Annealing ●Simulated annealing is just repeated executions of the Metropolis algorithm at slowly decreasing temperatures ●Four-step algorithm: 1.Select an initial solution, i, and temperature, T 2.while (the system hasn’t “frozen”) 3. i = MetropolisAlg(i,T) 4. T = LowerTemp(T) end

Cannons27 Example: Integrated Circuit Layout ●Common example problem (Rutenbar 1989) ●Given a logic circuit: ► A set of components ► A list of connections between the components ●Goal: ► Place the components on an integrated circuit (IC) board ► Minimize the wire needed to connect the components A B C D E F G H A B C D E F G H Connected Logic CircuitPlacement on IC board

Cannons28 Example: Integrated Circuit Layout ●When using SA (to solve any problem), four ingredients must be determined: 1. Solutions: The properties that any valid solution must have 2. Move Set: The allowable alterations that can be made to the current solution to generate the next potential solution 3. Cost Function: Measures the “goodness” of a solution 4. Cooling Schedule: Specifies how the temperature is changed ■Initial temperature ■When to lower the temperature ■Amount by which to decrease the temperature ■When annealing should conclude

Cannons29 Example: Integrated Circuit Layout 1. Solutions ► Simplify by restricting components on the IC to a grid ► A valid solution is an assignment of components to the grid such that: ■There is no more than one component per grid location ■No component is repeated ■All components in the logic circuit are present A B C D E F G H A B C D E F G H Connected Logic CircuitPlacement on IC board

Cannons30 Example: Integrated Circuit Layout 1. Solutions ► Simplify by restricting components on the IC to a grid ► A valid solution is an assignment of components to the grid such that: ■There is no more than one component per grid location ■No component is repeated ■All components in the logic circuit are present A B C D E F G H H F B D E G A C Connected Logic CircuitPlacement on IC board

Cannons31 Example: Integrated Circuit Layout 2. Move Set ► Generate a new solution by swapping the position of two components A B C D E F G H A B C D E F G H Connected Logic CircuitPlacement on IC board

Cannons32 Example: Integrated Circuit Layout 2. Move Set ► Generate a new solution by swapping the position of two components A B C D E F G H A B C D E F G H Connected Logic CircuitPlacement on IC board

Cannons33 Example: Integrated Circuit Layout 2. Move Set ► Generate a new solution by swapping the position of two components A B C D E F G H A C B D E F G H Connected Logic CircuitPlacement on IC board

Cannons34 Example: Integrated Circuit Layout 2. Move Set ► Generate a new solution by swapping the position of two components A B C D E F G H A C B D E F G H Connected Logic CircuitPlacement on IC board

Cannons35 Example: Integrated Circuit Layout 3. Cost Function ► Estimate of the total wire length on the board ■Consider each pair of components that are connected in the logic circuit ■Draw a box around the components on the board ■The amount of wire needed to connect those two components is 0.5  (Box Perimeter) ■Total wire length is the sum of the pairwise lengths A B C D E F G H A B C D E F G H Connected Logic CircuitPlacement on IC board

Cannons36 Example: Integrated Circuit Layout 3. Cost Function ► Estimate of the total wire length on the board ■Consider each pair of components that are connected in the logic circuit ■Draw a box around the components on the board ■The amount of wire needed to connect those two components is 0.5  (Box Perimeter) ■Total wire length is the sum of the pairwise lengths A B C D E F G H A B C D E F G H Connected Logic CircuitPlacement on IC board

Cannons37 Example: Integrated Circuit Layout 4. Cooling Schedule ► T init is set so that P accept = 95% ► Metropolis algorithm performs 100M iterations at each temperature ■M = the number of components ► T new =  T old,  < 1 ► SA halts when the results returned by the Metropolis algorithm improve by less than 1% over three consecutive temperatures

Cannons38 Example: Integrated Circuit Layout ●Results (Rutenbar 1989): ► SA algorithm described was implemented and run on an instance of the problem ■Approximately 800 components to place on the IC board Cost Temperature

Cannons39 Mathematically Modelling SA ●Ideas that underlie SA are mathematically rigorous (Trosset 2001) ●Typically model SA algorithm as a number of Markov chains ► One Markov chain for each temperature ●Allows for interesting mathematical proofs: ► Guaranteed asymptotic convergence to a global minimum ► Maximum rate of cooling which allows equilibrium to be reached

Cannons40 Summary Motivated the importance of simulated annealing ► Can find decent solutions to challenging problems in a reasonable amount of time

Cannons41 Summary Motivated the importance of simulated annealing ► Can find decent solutions to challenging problems in a reasonable amount of time Explained the simulated annealing algorithm ► Repeated executions of the Metropolis algorithm for decreasing temperature levels

Cannons42 Summary Motivated the importance of simulated annealing ► Can find decent solutions to challenging problems in a reasonable amount of time Explained the simulated annealing algorithm ► Repeated executions of the Metropolis algorithm for decreasing temperature levels Provided an example of solving a real-world problem with SA ► Integrated circuit board layout

Cannons43 References L. Ingber. Simulated annealing: practice versus theory. Mathl. Comput. Modelling, 18(11):29-57, S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by simulated annealing. Science, 220: , P. J. M. van Laarhoven and E. H. L. Aarts. Simulated Annealing: Theory and Applications. D. Reidel Publishing Company, N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, and A. H. Teller. Equation of state calculations by fast computing machines. The Journal of Chemical Physics, 21(6): , R. A. Rutenbar. Simulated annealing: an overview. IEEE Circuits and Devices Magazine, 5(1):19-26, M. W. Trosset. What is simulated annealing? Optimization and Engineering, 2: , 2001.

Cannons44 Questions?

Cannons45

Cannons46 Mathematically Modelling SA T = T init T = T q T = T final … … Stat e 1 Stat e 2 Stat e 3 Stat e S … Stat e 1 Stat e 2 Stat e 3 Stat e S … Stat e 1 Stat e 2 Stat e 3 Stat e S …

Cannons47 Optimization Problems ●Share a common set of ingredients (Edmonds 2005) ► Instances: The set of potential inputs for the problem ► Solutions: Each instance has an exponential number ► Cost of a Solution: Each solution has a value associated with it ●Goal: ► Find a valid solution to an instance which minimizes (or maximizes) the cost function

Cannons48 Disadvantages ●Long run-time ► Must decrease the temperature carefully and slowly to ensure convergence ■Slower the cooling, the better the chance of finding an optimal solution ●Not always trivial to fine tune for specific problems ► Ex. Designing the move set ●Some problems don’t have solution spaces that are well-suited for annealing ●Suffered from over-hype

Cannons49 Summary ●During this presentation Motivated the importance of simulated annealing ■Can find decent solutions to challenging problems in a reasonable amount of time Explained the simulated annealing algorithm ■Repeated execution of the Metropolis algorithm for decreasing temperature levels

Cannons50 Potential Solution Techniques ●Hill Climbing/Iterative Improvement? ► Algorithm: ■Start with a random solution ■Take small steps in the direction that decreases the solution cost ■Stop taking steps when they can no longer provide a solution with a lower cost Cost Solution

Cannons51 Example: Integrated Circuit Layout 1. Solutions ► Simplify by restricting components on the IC to a grid ► A valid solution is an assignment of components to the grid such that: ■There is no more than one component per grid location ■No component is repeated ■All components in the logic circuit are present A B C D E F G H A B C D E F G A Connected Logic CircuitPlacement on IC board

Cannons52 Example: Integrated Circuit Layout 2. Move Set ► Generate a new solution by swapping the position of two components A B C D E F G H A C B D E F G H Connected Logic CircuitPlacement on IC board

Cannons53 Example: Integrated Circuit Layout 2. Move Set ► Generate a new solution by swapping the position of two components A B C D E F G H A C B DF G H Connected Logic CircuitPlacement on IC board E

Cannons54 Example: Integrated Circuit Layout 2. Move Set ► Generate a new solution by swapping the position of two components A B C D E F G H A C B DF G H Connected Logic CircuitPlacement on IC board E

Cannons55 Example: Integrated Circuit Layout 3. Cost Function ► Example: ■Wire needed to connect Component A to Component C  Wire AC = 0.5  ( ) = 3 A B C D E F G H A B C D E F G H Connected Logic CircuitPlacement on IC board 1 2

Cannons56 Example: Integrated Circuit Layout 3. Cost Function ► Example: ■Wire needed to connect Component A to Component C  Wire AC = 0.5  ( ) = 3 ■Wire needed to connect Component A to Component D  Wire AD = 0.5  ( ) = 4 A B C D E F G H A B C D E F G H Connected Logic CircuitPlacement on IC board 2 2

Cannons57 Example: Integrated Circuit Layout 3. Cost Function ► Example: ■Wire needed to connect Component A to Component C  Wire AC = 0.5  ( ) = 3 ■Wire needed to connect Component A to Component D  Wire AD = 0.5  ( ) = 4 ■Etc. ■Total Wire Length = …. A B C D E F G H A B C D E F G H Connected Logic GatesPlacement on IC board