Heuristic Optimization Athens 2004 Department of Architecture and Technology Universidad Politécnica de Madrid Víctor Robles

Teachers  Universidad Politécnica de Madrid Víctor Robles (coordinator) María S. Pérez Vanessa Herves  Universidad del País Vasco Pedro Larrañaga

Course outline and Class hours  Day 1 / 10:00-14:00 / Víctor Introduction to optimization Some optimization problems About heuristics Greedy algorithms Hill climbing Simulated Annealing

Course outline and Class hours  Day 2 / 9:30-13:30 / Víctor, María Learn by practice: Simulated Annealing Genetic Algorithms  Day 3 / 9:30-13:30 / Vanessa Learn by practice: Genetic Algorihtms  Day 4 / 10:00-13:30 / Pedro Estimation of Distribution Algorithms

Introduce yourself  Name  University / Country  Optimization experience  Expectations of the course

Optimization  A optimization problem is a par being the search space (all the possible solutions), and a function,  The solution is optime if,  Combinatorial optimization  Systematic search

State space landscape  Objective function defines state space landscape

Some optimization problems  TSP – Travel Salesman Problem  The assignment problem  SAT – Satistiability problem  The 0-1 knapsack problem Important tasks  Find a representation of possible solutions  To be able to evaluate each of the possible solutions  “fitness function” or “objective function”

TSP  A salesman has to find a route which visits each of n cities, and which minimizes the total distance travelled  Given an integer and a n x n matrix where each is a nonnegative integer. Which cyclic permutation of integers from 1 to n minimizes the sum ?

TSP representations  Binary representation Each city is encoded as a string of [log 2 n] bits. Example 8 cities  3 bits  Path representation A list is represented as a list of n cities. If city i is the j-th element of the list, city i is the j-th city to be visited  Adjancecy representation City j is listed in position i if the tour leads from city i to city j ( )  tour:

The assignment problem  A set of n resources is available to carry out n tasks. If resource i is assigned to task j, it cost units. Find an assignment that minimizes  Solution: permutition of the numbers

SAT  The satisfiability problem consists on finding a truth assignment that satisfied a well-formed Boolean expression  Many applications: VLSI test and verification, consistency maintenance, fault diagnosis, etc  MAX-SAT: Find an assignment which satisfied the maximum number of clauses

SAT  Data sets in conjunctive normal form (cnf)  Example Literals: Clauses:  Representation? Fitness function?...

The 0-1 knapsack problem  A set of n items is available to be packed into a knapsack with capacity C units. Item i has value v i and uses up c i units of capacity. Determine the subset of items which should be packed to maximize the total value without exceding the capacity  Representation? Fitness function?

Heuristics  Faster than mathematical optimization (branch & bound, simplex, etc)  Well developed  good solutions for some problems  Special heuristics: Greedy algorithms – systematic Hill-climbing (based on neighbourhood search) – randomized

Greedy algorithms  Step-by-step algorithms  Sometimes works well for optimization problems  A greedy algorithm works in phases. At each phase: You take the best you can get right now, without regard for future consequences You hope that by choosing a local optimum at each step, you will end up at a global optimum

Example: Counting money  Suppose you want to count out a certain amount of money, using the fewest possible bills and coins  A greedy algorithm would do this would be: At each step, take the largest possible bill or coin that does not overshoot Example: To make $6.39, you can choose: a $5 bill a $1 bill, to make $6 a 25¢ coin, to make $6.25 A 10¢ coin, to make $6.35 four 1¢ coins, to make $6.39  For US money, the greedy algorithm always gives the optimum solution

A failure of the greedy algorithm  In some (fictional) monetary system, “krons” come in 1 kron, 7 kron, and 10 kron coins  Using a greedy algorithm to count out 15 krons, you would get A 10 kron piece Five 1 kron pieces, for a total of 15 krons This requires six coins  A better solution would be to use two 7 kron pieces and one 1 kron piece This only requires three coins  The greedy algorithm results in a solution, but not in an optimal solution

Practice  Develop a greedy algorithm for the knapsack problem. Groups of 2 persons

Local search algorithms  Based on neighbourhood system  Neighbourhood system: being X the search space, we define the neighbourhood system N in X  Examples: TSP (2-opt), SAT, assignment and knap

Local search algorithms  Basic principles: Keep only a single (complete) state in memory Generate only the neighbours of that state Keep one of the neighbours and discard others  Key features: No search paths Neither systematic nor incremental  Key advantages: Use very little memory (constant amount) Find solutions in search spaces too large for systematic algorithms

TSP – 2 opt A B C D New distance = Old dist – dist(A-D) – dist(B-C) + dist(A-B) + dist (C-D)

Neighbourhood search (Reeves93) 1.(Initialization) i. Select a starting solution ii. Current best and 2.(Choice and termination) i. Choice. If choice criteria cannot be satisfied or if other termination criteria apply, then the method stops 3.(Update) i. Re-set, and if, perform step 1.ii. Return to Step 2

Hill climbing  Diferent procedures depending on choice criteria and termination criteria  Hill climbing: only permit moves to neighbours that improve the current 2.(Choice and termination) i.Choose such that and terminate if no such can be found

Hill-climbing: 8-Queens problem  Complete state formulation: All 8 queens on the board,one per column Neighbourhood: move one queen to a different place in the same column Fitness function: number of pairs of queens that are attacking each other

8-Queens problem: fitness values of neighbourhood

8-Queens problem: Local minimun

Problematic landscapes  Local maximum: a peek that is higher than all its neighbours, but not a global maximum  Plateau: an area where the elevation is constant Local maximum Shoulder  Ridge: a long, narrow, almost plateau-like landscape

Random-Restart Hill-Climbing  Method: Conduct a series of hill-climbing searches from randomly generated initial states Stop when a goal is found  Analysis: Requires 1/p restarts where p is the probability of success (1 success + 1/p failures)

Hill-Climbing: Performance on the 8-Queen Problem  From randomly generated start state  Success rate: 86% - gets stuck 14% - solves problem  Average cost: 4 steps to success 3 steps to get stuck

Hill-Climbing with Sideways Moves  Sideways moves: moves at same fitness  Must limit number of sideways moves!  Performance on the 8-Queen problem: Success rate: 6% - get stucks 94% - solves problem Average cost: 21 steps to succeed 64 steps to get stuck

Hill-climbing: Further variants  Stochastic hill-climbing: Choose at random from among uphill moves  First-choice hill-climbing: Generate neighbourhood in random order Move to first generated that represents an uphill move

Practice  Develop a hill-climbing algorithm for the knapsack problem. Groups of 2 persons

Shape of State Space Landscape  Success of hill-climbing depends on shape of landscape  Shape of landscape depends on problem formulation and fitness function  Landscapes for realistic problems often look like a worst-case scenario  NP-hard problems typically have exponential number of local-maxima  Despite the above, hill-climbers tend to have good performance

Simulated annealing  Failing on neighbourhood search Propensity to deliver solutions which are only local optima Solutions depend on the initial solution  Reduce the chance of getting stuck in a local optimum by allowing moves to inferior solutions  Developed by Kirkpatrick ’83: Simulation of the cooling of material in a heat bath could be used to search the feasible solutions of an optimization problem

Simulated annealing  If a move from one solution to another neighbouring but inferior solution results in a change in value, the move to is still accepted if T (temperature) – control parameter – uniform random number

Simulated annealing: Intuition  Minimization problem; imagine a state space landscape on table  Let ping-pong ball from random point  local minimum  Shake table  ball tends to find different minimum  Shake hard at first (high temperature) but gradually reduce intensity (lower temperature)

Simulated annealing: Algorithm current = problem.initialSate for t=1 to T = schedule(t) if T=0 then return = a random neighbour of if then else with probability

Simulated annealing: Simple example  Maximize x coded as a 5-bit binary integer in [0,31] maximum (01010)  f=4100  With ‘greedy’ we can find 3 local maxima

Simulated annealing: Simple example The temperature is not high enough to move out of this local optimum

Simulated annealing: Simple example Optimum found!!!

Simulated annealing: Generic decisions  Initial temperature Should be ‘suitable high’. Most of the initial moves must be accepted (> 60%)  Cooling schedule Temperature is reduced after every move Two methods:  close to 1  close to 0

Simulated annealing: Generic decisions  Number of iterations  Other factors: Reannealing Restricted neighbourhoods Order of searching