EMIS 8373: Integer Programming Primal Heuristics: Greedy Solutions and Local Search updated 8 February 2005

Combinatorial Optimization Problems Input A finite set N = {1, 2, …, n} Weights (costs) cj for all j  N Cost(SN) = A set F of feasible subsets of N Optimization Problem Find a minimum-weight feasible subset

Generic Greedy Heuristic for COP Start with an “empty” solution Let S0 =  and t = 1. Choose j in N \ St-1 such that the cost of the resulting solution St-1  {j} is minimized. St = St-1  {j} If St-1 is feasible and Cost(St)  Cost(St-1  {j}) stop and return St-1. If t = n then Stop. Otherwise, let t = t + 1 and go to step 2.

A Greedy Heuristic for UFL UFL can be modeled as a COP where N is the set of depots and we assume that for any given set of depots S  N all 100% of each client’s demand is satisfied by the nearest depot.

A Greedy Heuristic for UFL: Initialization and First Iteration S0 =  and S0 is infeasible. Cost({1}) = 10+ 1+2+3+5+6+8 =35 Cost({2}) =11+ 9+5+4+3+2+1=35 Cost({3}) = 34 Cost({4}) = 50 j = 3 S1 = {3}, Cost({S1}) = 34

A Greedy Heuristic for UFL: Iteration 2 Cost({1,3}) = 10 + 16 + 11 = 37 1 c1 d1 10 2 c2 3 d2 c3 2 c4 d3 16 2 c5 1 d4 c6

A Greedy Heuristic for UFL: Iteration 2 Cost({2,3}) = 16 + 11 + 16 = 43 3 c1 d1 4 c2 d2 11 4 c3 2 c4 d3 16 2 c5 1 d4 c6

A Greedy Heuristic for UFL: Iteration 2 Cost({3,4}) = 16 + 10 + 15= 41 3 c1 d1 4 c2 3 d2 c3 2 c4 d3 16 2 c5 1 d4 10 c6

A Greedy Heuristic for UFL: Second Iteration S1 = {3}, Cost({S1}) = 34 S2 = {1,3} Cost({S2}) = 37 > 34 Stop. Return S1.

Generic Local Search for COP For each S  F define a neighborhood N(S)  F\{S}. Select an initial solution S  F If Cost(T)  Cost(S) for all T in N(S) then stop and return S. Otherwise select a minimum cost neighbor T  N(S), let S = T, and goto step 3.

Local Search The initial solution S could be generated in a variety of ways such as: A randomly generated solution A solution returned by a greedy heuristic A solution returned by solving an LP relaxation and rounding It is up to the user to define the neighborhood The neighborhoods of a solution is the set of solutions that are “close” to it. “Good” neighborhoods are usually ones that are “easy” to evaluate

A Local Search Heuristic for UFL Neighborhood defined by two “moves” T = S  {j} where j in N\S (i.e., add another depot to the current solution) T = S \ {j} where j in S (i.e., shut down one of the depots in the current solution) Let N(S) be the set of all T  F such that |(S  T) \ (S  T)| = 1. Evaluating the neighborhood requires solving |N| transportation problems.

UFL Local Search Example 1 {3, 4} Cost = 41 {1, 3, 4} Cost = 47 {2, 3, 4} Cost = 52 {4} Cost = 50 { 3} Cost = 34 {1, 3} Cost = 37 {2, 3} Cost = 43 {3, 4} Cost = 41 Stop. Return {3}

UFL Local Search Example 2 {1, 3, 4} Cost = 47 {3, 4} Cost = 41 {1, 2, 3, 4} Cost = 58 {1, 4} Cost = 42 {1, 3} Cost = 37 {3} Cost = 34 {1, 2, 3} Cost = 48 {1} Cost = 35 {1, 3, 4} Cost = 47 Stop. Return {3} {1, 3} Cost = 37 {2, 3} Cost = 43 {3, 4} Cost = 41

UFL Local Search Example 3 {1, 2, 3, 4} Cost =58 {2, 3, 4} Cost = 52 {1, 3, 4} Cost = 47 {1, 2, 4} Cost = 43 {1, 2, 3} Cost = 48 {2, 4} Cost = 39 {1, 4} Cost = 42 {1, 2, 3, 4} Cost = 58 {1, 2} Cost =33 {2} Cost =35 {1} Cost = 35 {1, 2, 3} Cost = 48 {1, 2, 4} Cost = 43

Solution Space for UFL Example

Local Minima Depending on where we start in the solution space, the local search heuristic may converge to different local minima. A solution S  F is a local minima if Cost(S)  Cost(T) for all T  N(S) Usually there is no a priori guarantee that a local search will converge to a global minimum. There are some exceptions to this such as The Simplex Method for LP & MST Metaheuristics such as Simulated Annealing and Tabu Search attempt to overcome this drawback of local search.

Symmetric TSP 9 2 3 10 6 2 5 1 1 4 3 6 5 4 2 tij Table Graph Representation 9 2 3 10 6 2 5 1 4 1 3 6 5 4 2

Greedy Heuristic Nearest Neighbor Start at city 1 Let N = N \ {1} Let i = 1 Move to city j where j = argminN d[i,j] Let N = N \ {j} If |N| > 0 then let i = j and goto set 4

Nearest Neighbor Example 9 10 1 3 2 5 4 6 2 3 10 2 1 1 5 5 4 Cost = 20 2

Local Search Heuristic: Pairwise Exchange

First Pairwise Exchange 10 1 2 4 5 3 Cost = 20 (4, 3, 1, 5, 2) 1 3 2 5 4 6 Cost = 17 (1, 3, 4, 5, 2)

Pairwise Exchange Example: Second Pass

Second Pairwise Exchange 2 2 3 6 1 4 1 3 (4, 3, 1, 5, 2) 5 4 1 3 2 5 4 6 Cost = 17 Cost = 16 (5, 3, 1, 4, 2)

Pairwise Exchange Example: Third Pass Stop: Every pairwise exchange leads to a worse solution.