1. EMIS 8373: Integer Programming
Primal Heuristics: Greedy Solutions and Local Search
updated 8 February 2005

2. Combinatorial Optimization Problems
Input
A finite set N = {1, 2, …, n}
Weights (costs) cj for all j  N
Cost(SN) =
A set F of feasible subsets of N
Optimization Problem
Find a minimum-weight feasible subset

3. Generic Greedy Heuristic for COP
Start with an “empty” solution
Let S0 =  and t = 1.
Choose j in N \ St-1 such that the cost of the resulting solution St-1  {j} is minimized.
St = St-1  {j}
If St-1 is feasible and Cost(St)  Cost(St-1  {j}) stop and return St-1.
If t = n then Stop. Otherwise, let t = t + 1 and go to step 2.

4. A Greedy Heuristic for UFL
UFL can be modeled as a COP where N is the set of depots and we assume that for any given set of depots S  N all 100% of each client’s demand is satisfied by the nearest depot.

5. A Greedy Heuristic for UFL: Initialization and First Iteration
S0 =  and S0 is infeasible.
Cost({1}) = =35
Cost({2}) = =35
Cost({3}) = 34
Cost({4}) = 50
j = 3
S1 = {3}, Cost({S1}) = 34

6. A Greedy Heuristic for UFL: Iteration 2
Cost({1,3}) = = 37 1 c1 d1 10 2 c2 3 d2 c3 2 c4 d3 16 2 c5 1 d4 c6

7. A Greedy Heuristic for UFL: Iteration 2
Cost({2,3}) = = 43 3 c1 d1 4 c2 d2 11 4 c3 2 c4 d3 16 2 c5 1 d4 c6

8. A Greedy Heuristic for UFL: Iteration 2
Cost({3,4}) = = 41 3 c1 d1 4 c2 3 d2 c3 2 c4 d3 16 2 c5 1 d4 10 c6

9. A Greedy Heuristic for UFL: Second Iteration
S1 = {3}, Cost({S1}) = 34
S2 = {1,3}
Cost({S2}) = 37 > 34
Stop. Return S1.

10. Generic Local Search for COP
For each S  F define a neighborhood N(S)  F\{S}.
Select an initial solution S  F
If Cost(T)  Cost(S) for all T in N(S) then stop and return S.
Otherwise select a minimum cost neighbor T  N(S), let S = T, and goto step 3.

11. Local Search
The initial solution S could be generated in a variety of ways such as:
A randomly generated solution
A solution returned by a greedy heuristic
A solution returned by solving an LP relaxation and rounding
It is up to the user to define the neighborhood
The neighborhoods of a solution is the set of solutions that are “close” to it.
“Good” neighborhoods are usually ones that are “easy” to evaluate

12. A Local Search Heuristic for UFL
Neighborhood defined by two “moves”
T = S  {j} where j in N\S (i.e., add another depot to the current solution)
T = S \ {j} where j in S (i.e., shut down one of the depots in the current solution)
Let N(S) be the set of all T  F such that |(S  T) \ (S  T)| = 1.
Evaluating the neighborhood requires solving |N| transportation problems.

13. UFL Local Search Example 1
{3, 4}
Cost = 41
{1, 3, 4}
Cost = 47
{2, 3, 4}
Cost = 52
{4}
Cost = 50
{ 3}
Cost = 34
{1, 3}
Cost = 37
{2, 3}
Cost = 43
{3, 4}
Cost = 41
Stop.
Return {3}

14. UFL Local Search Example 2
{1, 3, 4}
Cost = 47
{3, 4}
Cost = 41
{1, 2, 3, 4}
Cost = 58
{1, 4}
Cost = 42
{1, 3}
Cost = 37
{3}
Cost = 34
{1, 2, 3}
Cost = 48
{1}
Cost = 35
{1, 3, 4}
Cost = 47
Stop.
Return {3}
{1, 3}
Cost = 37
{2, 3}
Cost = 43
{3, 4}
Cost = 41

15. UFL Local Search Example 3
{1, 2, 3, 4}
Cost =58
{2, 3, 4}
Cost = 52
{1, 3, 4}
Cost = 47
{1, 2, 4}
Cost = 43
{1, 2, 3}
Cost = 48
{2, 4}
Cost = 39
{1, 4}
Cost = 42
{1, 2, 3, 4}
Cost = 58
{1, 2}
Cost =33
{2}
Cost =35
{1}
Cost = 35
{1, 2, 3}
Cost = 48
{1, 2, 4}
Cost = 43

16. Solution Space for UFL Example

17. Local Minima
Depending on where we start in the solution space, the local search heuristic may converge to different local minima.
A solution S  F is a local minima if Cost(S)  Cost(T) for all T  N(S)
Usually there is no a priori guarantee that a local search will converge to a global minimum.
There are some exceptions to this such as
The Simplex Method for LP & MST
Metaheuristics such as Simulated Annealing and Tabu Search attempt to overcome this drawback of local search.

18. Symmetric TSP 9 2 3 10 6 2 5 1 1 4 3 6 5 4 2 tij Table
Graph Representation 9 2 3 10 6 2 5 1 4 1 3 6 5 4 2

19. Greedy Heuristic Nearest Neighbor
Start at city 1
Let N = N \ {1}
Let i = 1
Move to city j where j = argminN d[i,j]
Let N = N \ {j}
If |N| > 0 then let i = j and goto set 4

20. Nearest Neighbor Example9 10 1 3 2 5 4 6 2 3 10 2 1 1 5 5 4
Cost = 20 2

21. Local Search Heuristic: Pairwise Exchange

22. First Pairwise Exchange10 1 2 4 5 3
Cost = 20
(4, 3, 1, 5, 2) 1 3 2 5 4 6
Cost = 17
(1, 3, 4, 5, 2)

23. Pairwise Exchange Example: Second Pass

24. Second Pairwise Exchange2 2 3 6 1 4 1 3
(4, 3, 1, 5, 2) 5 4 1 3 2 5 4 6
Cost = 17
Cost = 16
(5, 3, 1, 4, 2)

25. Pairwise Exchange Example: Third Pass
Stop: Every pairwise exchange leads to a worse solution.