1. December 14, 2015 Design and Analysis of Computer Algorithm Pradondet Nilagupta Department of Computer Engineering

2. Design and Analysis of Computer Algorithm 2December 14, 2015 Greedy Method

3. Design and Analysis of Computer Algorithm 3December 14, 2015 Greedy Method: Definition An algorithm which always takes the best immediate, or local, solution while finding an answer. Greedy algorithms will always find the overall, or globally, optimal solution for some optimization problems, but may find less-than-optimal solutions for some instances of other problems. optimal solutionoptimization problems

4. Design and Analysis of Computer Algorithm 4December 14, 2015 Example of Greedy Method (1/4) Prim's algorithm and Kruskal's algorithm are greedy algorithms which find the globally optimal solution, a minimum spanning tree. In contrast, any known greedy algorithm to find an Euler cycle might not find the shortest path, that is, a solution to the traveling salesman problem. Prim's algorithmKruskal's algorithm minimum spanning treeEuler cycletraveling salesman Dijkstra's algorithm for finding shortest paths is another example of a greedy algorithm which finds an optimal solution. Dijkstra's algorithmshortest paths

5. Design and Analysis of Computer Algorithm 5December 14, 2015 Example of Greedy Method (2/4) If there is no greedy algorithm which always finds the optimal solution for a problem, one may have to search (exponentially) many possible solutions to find the optimum. Greedy algorithms are usually quicker, since they don't consider possible alternatives.

6. Design and Analysis of Computer Algorithm 6December 14, 2015 Example of Greedy Method (3/4) Consider the problem of making change: Coins of values 25c, 10c, 5c and 1c Return 63c in change –Which coins? Use greedy strategy: –Select largest coin whose value was no greater than 63c –Subtract value (25c) from 63 getting 38 –Find largest coin … until done

7. Design and Analysis of Computer Algorithm 7December 14, 2015 Example of Greedy Method (4/4) At any individual stage, select that option which is “locally optimal” in some particular sense Greedy strategy for making change works because of special property of coins If coins were 1c, 5c and 11c and we need to make change of 15c? –Greedy strategy would select 11c coin followed by 4 1c coins –Better: 3 5c coins

8. Design and Analysis of Computer Algorithm 8December 14, 2015 Problem Make a change of a given amount using the smallest possible number of coins. MAKE-CHANGE (n) C ← {100, 25, 10, 5, 1} // constant. Sol ← {}; // set that will hold the solution set. Sum ← 0 sum of item in solution set WHILE sum not = n x = largest item in set C such that sum + x ≤ n IF no such item THEN RETURN "No Solution" S ← S {value of x} sum ← sum + x RETURN S

9. Design and Analysis of Computer Algorithm 9December 14, 2015 Greedy Algorithm Start with a solution to a small subproblem Build up to a solution to the whole problem Make choices that look good in the short term Disadvantage: Greedy algorithms don’t always work ( Short term solutions can be diastrous in the long term). Hard to prove correct Advantage: Greedy algorithm work fast when they work. Simple algorithm, easy to implement

10. Design and Analysis of Computer Algorithm 10December 14, 2015 Greedy Algorithm Procedure GREEDY(A,n) // A(1:n) contains the n inputs// solution   //initialize the solution to empty// for i  1 to n do x  SELECT(A) if FEASIBLE(solution,x) then solution  UNION(solution,x) endif repeat return(solution) end GREEDY

11. Design and Analysis of Computer Algorithm 11December 14, 2015 Activity-Selection Problem The problem is to select a maximum-size set of mutally compatible activities. Example We have a set S = { 1,2,…, n} of n proposed activities that wish to use a resource, such as a lecture hall, which can be used by only one activities at a time.

12. Design and Analysis of Computer Algorithm 12December 14, 2015 Example is i f i 106 235 314 4213 538 61214 7811 8812 9610 1057 1159  

13. Design and Analysis of Computer Algorithm 13December 14, 2015 Brute Force Try every all possible solution Choose the largest subset which is feasible Ineffcient  (2 n ) choices

14. Design and Analysis of Computer Algorithm 14December 14, 2015 Greedy Approach   Sort by finish time

15. Design and Analysis of Computer Algorithm 15December 14, 2015 Activity-Selection Problem Pseudo code Greedy_Activity_Selector(s,f) 1n <- length[s] 2A <- {1} 3j <- 1 4for i <- 2 to n 5 do if s i > f j 6then A <- A U {i} 7 j <- i 8return A It can schdule a set S of n activities in  (n) time, assuming that the activities were already sorted

16. Design and Analysis of Computer Algorithm 16December 14, 2015 Proving the greedy algorithm correct We assume that the input activities are in order by increasing finishing time f 1 < f 2 < … < f n Activities #1 has the earliest finish time then it must be in an optimal solution.   k 1 1 possible solution Activitiy 1

17. Design and Analysis of Computer Algorithm 17December 14, 2015 Proving (cont.)  k 1 Eliminate the activities which has a start time early than the finish time of activity 1

18. Design and Analysis of Computer Algorithm 18December 14, 2015 Proving (cont.)  1 Greedy algorithm produces an optimal solution

19. Design and Analysis of Computer Algorithm 19December 14, 2015 Element of the Greedy Strategy Question? How can one tell if a greedy algorithm will solve a particular optimization problem? No general way to tell!!! There are 2 ingredients that exhibited by most problems that lend themselves to a greedy strategy –The Greedy Choice Property –Optimal Substructure

20. Design and Analysis of Computer Algorithm 20December 14, 2015 The Greedy Choice Property A globally optimal solution can be arrived at by making a locally optimal (greedy) choice. Make whatever choice seems best at the moment. May depend on choice so far, but not depend on any future choices or on the solutions to subproblems

21. Design and Analysis of Computer Algorithm 21December 14, 2015 Optimal Substructure An optimal solution to the problem contains within it optimal solutions to subproblems

22. Design and Analysis of Computer Algorithm 22December 14, 2015 Knapsack Problem We are given n objects and a knapsack. Object i has a weight w i and the knapsack has a capacity M. If a fraction x i, 0  x i  1, of object I is placed into the knapsack the a profit of p i x i is earned. The objective is to obtain a filling of the knapsack that maximizes the total weight of all chosen objects to be at most M maximize subject to and 0  xi  1, 1  I  n

23. Design and Analysis of Computer Algorithm 23December 14, 2015 Example 10 20 30 50 $60$100$120 Item 1 Item 2 Item 3 knapsack

24. Design and Analysis of Computer Algorithm 24December 14, 2015 Knapsack 0/1 30 20 $120 $100 Total=$220 20 10 $100 $60 =$160 30 10 $120 $60 =$180

25. Design and Analysis of Computer Algorithm 25December 14, 2015 Fractional Knapsack Taking the items in order of greatest value per pound yields an optimal solution 20 10 $100 $60 =$240Total 20 30 $80

26. Design and Analysis of Computer Algorithm 26December 14, 2015 Optimal Substructure Both fractional knapsack and 0/1 knapsack have an optimal substructure.

27. Design and Analysis of Computer Algorithm 27December 14, 2015 Example Fractional Knapsack (cont.) There are 5 objects that have a price and weight list below, the knapsack can contain at most 100 Lbs. Method 1 choose the least weight first –Total Weight = 10 + 20 + 30 + 40 = 100 –Total Price = 20 + 30 + 66 + 40 = 156

28. Design and Analysis of Computer Algorithm 28December 14, 2015 Example Fractional Knapsack (cont.) Method 2 choose the most expensive first –Total Weight = 30 + 50 + 20 = 100 –Total Price = 66 + 60 + 20 = 146 half

29. Design and Analysis of Computer Algorithm 29December 14, 2015 Example Fractional Knapsack (cont.) Method 3 choose the most price/ weight first –Total weight = 30 + 10 + 20 + 40 = 100 –Total Price = 66 + 20 + 30 + 48 = 164 

30. Design and Analysis of Computer Algorithm 30December 14, 2015 More Example on fractional knapsac Consider the following instance of the knapsack problem: n = 3, M = 20, (p 1,p 2,p 3 ) = 25,24,15 and (w 1,w 2,w 3 ) = (18,15,10) (x1,x2,x3) 1) (1/2,1/3,1/4)16.524.25 2) (1,2/15,0)2028.2 3) ( 0,2/3,1)2031 4) ( 0,1,1/2)2031.5

31. Design and Analysis of Computer Algorithm 31December 14, 2015 The Greedy Solution Define the density of object Ai to be w i /s i. Use as much of low density objects as possible. That is, process each in increasing order of density. If the whole thing t s, use all of it. If not, fill the remaining space with a fraction of the current object,and discard the rest. First, sort the objects in nondecreasing density, so that w i /s i  w i+1 /s i+1 for 1  i < n. Then do the following

32. Design and Analysis of Computer Algorithm 32December 14, 2015 PseudoCode Procedure GREEDY_KNAPSACK(P,W,M,X,n) //P(1:n) and W(1:n) contain the profits and weights respectively of the n objects ordered so that P(I)/W(I) > P(I+1)/W(I+1). M is the knapsack size and X(1:n) is the solution vector// real P(1:n), W(1:n), X(1:n), M, cu; integer I,n; x  ; //initialize solution to zero // cu  M; // cu = remaining knapsack capacity // for i  to n do if W(i) > cu then exit endif X(I)  1; cu c - W(i); repeat if I < n then X(I)  cu/W(I) endif End GREEDY_KNAPSACK