Greedy Algorithm Enyue (Annie) Lu

Greedy Algorithm It makes the choice that looks best at the moment and adds it to the current subsolution. It makes a locally optimal choice in the hope that this choice will lead to a globally optimal solution. Examples Prim/Kruskal’s MST algorithms Dijkstra’s shortest path algorithm Optimal Storage on Tapes

Knapsack Problem (fractional) The problem: Given a knapsack with a certain capacity M, n items, are to be put into the knapsack, each has a weight and a profit if put in the knapsack . The objective: find where s.t. is maximized and   Note: All items can break into small pieces or xi can be any fraction between 0 and 1.

Example: Greedy Strategy#1: Profits are ordered in nonincreasing order (1,2,3) Greedy Strategy#2: Weights are ordered in nondecreasing order (3,2,1)

Example: Optimal solution Greedy Strategy#3: p/w are ordered in nonincreasing order (2,3,1) Optimal solution

Algorithm O(n) O(nlogn) O(n) O(nlogn) 1. Calculate vi = pi / wi for i = 1, 2, …, n 2. Sort items by nonincreasing vi. (all wi are also reordered correspondingly ) 3. Let M' be the current weight limit (Initially, M' = M and xi=0 ).In each iteration, choose item i from the head of the unselected list. If M' >= wi , set xi=1, and M' = M'-wi If M' < wi , set xi=M'/wi and the algorithm is finished. O(nlogn) O(n) O(nlogn)

Proof Proof by contradiction: Given some knapsack instance Suppose the items are ordered s.t. let the greedy solution be Show that this ordering is optimal Case1: it’s optimal Case2: s.t. where

Proof Assume X is not optimal, and then there exists s.t. and Y is optimal examine X and Y, let yk be the 1st one in Y that yk  xk. yk  xk  yk < xk same Since we sort x_k in nondecresing order Now we increase yk to xk and decrease as many of as necessary, so that the capacity is still M.

Proof Let this new solution be where

Proof So, if else (Repeat the same process. At the end, Y can be transformed into X.  X is also optimal. Contradiction! )

Greedy Strategy Obtain an optimal solution by making a sequence of choices. The choice that seems best at the moment is chosen. This strategy does not always produces an optimal solution. Then how can one tell if a greedy algorithm will solve a particular optimization problem?? There is no way in general. But there are 2 ingredients exhibited by most greedy problems: Greedy Choice Property Optimal Sub Structure

Greedy Choice Property A globally optimal solution can be arrived at by making a locally optimal (Greedy) choice. We make whatever choice seems best at the moment and then solve the sub problems arising after the choice is made. The choice made by a greedy algorithm may depend on choices so far, by it cannot depend on any future choices or on the solutions to sub problems. Thus, a greedy strategy usually progresses in a top-down fashion, making one greedy choice after another, iteratively reducing each given problem instance to a smaller one.

Optimal Sub Structure A problem exhibits optimal substructure if an optimal solution to the problem contains (within it) optimal solution to sub problems. knapsack problem: an optimal solution X begins with the item with maximum pi/wi, then the X-xi is an optimal solution to the knapsack problem with remaining n-1 items and capacity M-wi

0-1 Knapsack Problem (xi can be 0 or 1) Knapsack capacity: 50 80 + 100 60 =240 + 100 60 =160 + 120 60 =180 + 120 100 =220 i 1 2 3 wi 10 20 30 pi 60 100 120 Does 0-1 Knapsack has a greedy solution?