1. UMass Lowell Computer Science 91.404 Analysis of Algorithms Prof. Karen Daniels Fall, 2002 Monday, 12/2/02 Design Patterns for Optimization Problems Greedy Algorithms

2. Algorithmic Paradigm Context

3. Greedy Algorithms

4. What is a Greedy Algorithm? ä Solves an optimization problem ä Optimal Substructure: ä optimal solution contains in it optimal solutions to subproblems ä Greedy Strategy: ä At each decision point, do what looks best “locally” ä Choice does not depend on evaluating potential future choices or solving subproblems ä Top-down algorithmic structure ä With each step, reduce problem to a smaller problem ä Greedy Choice Property: ä “locally best” = globally best

5. Examples ä Minimum Spanning Tree ä Dijkstra Shortest Path ä Huffman Codes ä Fractional Knapsack ä Activity Selection

6. Minimum Spanning Tree

7. A B C D E F G 2 2 1 1 3 4 4 5 6 6 8 7 source: 91.503 textbook Cormen et al. for Undirected, Connected, Weighted Graph G=(V,E) Produces minimum weight tree of edges that includes every vertex. Invariant: Minimum weight spanning forest Becomes single tree at end Invariant: Minimum weight tree Spans all vertices at end Time: O(|E|lg|E|) given fast FIND-SET, UNION Time: O(|E|lg|V|) = O(|E|lg|E|) slightly faster with fast priority queue

8. Dijkstra Shortest Path

9. Single Source Shortest Paths: Dijkstra’s Algorithm source: 91.503 textbook Cormen et al. for (nonnegative) weighted, directed graph G=(V,E) A B C D E F G 2 2 1 1 3 4 4 5 6 6 8 7

10. Huffman Codes

12. Fractional Knapsack

13. 50 10 20 30 item1item2item3 “knapsack” Value: $60 $100$120

14. Activity Selection

15. Example Optimization Problem: Activity Selection ä Problem Instance: ä Set S = {1,2,...,n} of n activities ä Each activity i has: ä start time: s i ä finish time : f i ä Activities i, j are compatible iff non-overlapping: ä Objective: ä select a maximum-sized set of mutually compatible activities source: 91.503 textbook Cormen, et al.

17. Algorithmic Progression ä “Brute-Force” ä Dynamic Programming #1 ä Exponential number of subproblems ä Dynamic Programming #2 ä Quadratic number of subproblems ä Greedy Algorithm

18. Running time? Greedy Algorithm ä Algorithm: ä S’ = presort activities in S by nondecreasing finish time ä and renumber ä GREEDY-ACTIVITY-SELECTOR(S’) ä n length[S’] ä A {1} ä j1 ä for i 2 to n ä do if ä then ä j i ä return A source: 91.503 textbook Cormen, et al.

19. Greedy Heuristic ä If optimization problem does not have “greedy choice property”, greedy approach may still be useful as a heuristic in bounding the optimal solution ä Example: minimization problem Optimal (unknown value) Upper Bound (heuristic) Lower Bound Solution Values