1. 1 Greedy 2 Jose Rolim University of Geneva

2. Algorithmique Greedy 2Jose Rolim2 Examples Greedy  Minimum Spanning Trees  Shortest Paths Dijkstra

3. Algorithmique Greedy 2Jose Rolim3 Definition MST  Given a connected graph G = (V, E), with weight function w : E --> R  Min-weight connected subgraph  Spanning tree T:  A tree that includes all nodes from V  T = (V, E’), where E’  E  Weight of T: W( T ) =  w(e)  Minimum spanning tree (MST):  A tree with minimum weight among all spanning trees

4. Algorithmique Greedy 2Jose Rolim4 Example:

5. Algorithmique Greedy 2Jose Rolim5 MST:  MST for given G may not be unique  Since MST is a spanning tree:  # edges : |V| - 1  If the graph is unweighted:  All spanning trees have same weight

6. Algorithmique Greedy 2Jose Rolim6 Kruskal’s algorithm (Greedy)  Start with A empty, and each vertex being its own connected component  Repeatedly merge two components by connecting them with a light edge crossing them  Two issues:  Maintain sets of components  Choose light edges

7. Algorithmique Greedy 2Jose Rolim7 Pseudo-code

8. Algorithmique Greedy 2Jose Rolim8 The Greedy Algorithm in Action 1 2 3 4 5 6 7 35 10 30 15 25 40 20 17 8 15 11 21 1 2 3 4 5 6 7

9. Algorithmique Greedy 2Jose Rolim9 The Greedy Algorithm in Action 1 2 3 35 10 4 5 30 15 25 40 20 6 7 17 8 15 11 21 35 10 30 15 25 40 20 17 8 15 11 21 1 2 3 4 5 6 7

10. Algorithmique Greedy 2Jose Rolim10 The Greedy Algorithm in Action 1 2 3 35 10 4 5 30 15 25 40 20 6 7 17 8 15 11 21 35 10 30 15 25 40 20 17 8 15 11 21 1 2 3 4 5 6 7

11. Algorithmique Greedy 2Jose Rolim11 The Greedy Algorithm in Action 1 2 3 35 10 4 5 30 15 25 40 20 6 7 17 8 15 11 21 35 10 30 15 25 40 20 17 8 15 11 21 1 2 3 4 5 6 7

12. Algorithmique Greedy 2Jose Rolim12 The Greedy Algorithm in Action 1 2 3 35 10 4 5 30 15 25 40 20 6 7 17 8 15 11 21 35 10 30 15 25 40 20 17 8 15 11 21 1 2 3 4 5 6 7

13. Algorithmique Greedy 2Jose Rolim13 The Greedy Algorithm in Action 1 2 3 35 10 4 5 30 15 25 40 20 6 7 17 8 15 11 21 35 10 30 15 25 40 20 17 8 15 11 21 1 2 3 4 5 6 7

14. Algorithmique Greedy 2Jose Rolim14 The Greedy Algorithm in Action 1 2 3 35 10 4 5 30 15 25 40 20 6 7 17 8 15 11 21 35 10 30 15 25 40 20 17 8 15 11 21 1 2 3 4 5 6 7

15. Algorithmique Greedy 2Jose Rolim15 The Greedy Algorithm in Action 1 2 3 35 10 4 5 30 15 25 40 20 6 7 17 8 15 11 21 35 10 30 15 25 40 20 17 8 15 11 21 1 2 3 4 5 6 7

16. Algorithmique Greedy 2Jose Rolim16 The Greedy Algorithm in Action 1 2 3 35 10 4 5 30 15 25 40 20 6 7 17 8 15 11 21 35 10 30 15 25 40 20 17 8 15 11 21 1 2 3 4 5 6 7

17. Algorithmique Greedy 2Jose Rolim17 The Greedy Algorithm in Action 1 2 3 35 10 4 5 30 15 25 40 20 6 7 17 8 15 11 21 35 10 30 15 25 40 20 17 8 15 11 21 1 2 3 4 5 6 7

18. Algorithmique Greedy 2Jose Rolim18 The Greedy Algorithm in Action 1 2 3 35 10 4 5 30 15 25 40 20 6 7 17 8 15 11 21 35 10 30 15 25 40 20 17 8 15 11 21 1 2 3 4 5 6 7

19. Algorithmique Greedy 2Jose Rolim19 The Greedy Algorithm in Action 1 2 3 35 10 4 5 30 15 25 40 20 6 7 17 8 15 11 21 35 10 30 15 25 40 20 17 8 15 11 21 1 2 3 4 5 6 7

20. Algorithmique Greedy 2Jose Rolim20 The Greedy Algorithm in Action 1 2 3 35 10 4 5 30 15 25 40 20 6 7 17 8 15 11 21 35 10 30 15 25 40 20 17 8 15 11 21 1 2 3 4 5 6 7

21. The Greedy Algorithm in Action 1 2 3 35 10 4 5 30 15 25 40 20 6 7 17 8 15 11 21 Node 1 2 3 4 5 6 7 First 1 2 3 4 5 4 7 35 10 30 15 25 40 20 17 8 15 11 21 1 2 357 root node 46 6 21

22. The Greedy Algorithm in Action 1 2 3 35 10 4 5 30 15 25 40 20 6 7 17 8 15 11 21 Node 1 2 3 4 5 6 7 First 1 4 3 4 5 4 7 35 10 30 15 25 40 20 17 8 15 11 21 1 2 3 4 5 6 7 2 22

23. The Greedy Algorithm in Action 1 2 3 35 10 4 5 30 15 25 40 20 6 7 17 8 15 11 21 Node 1 2 3 4 5 6 7 First 1 4 3 4 5 4 5 35 10 30 15 25 40 20 17 8 15 11 21 1 2 3 4 5 6 7 7 23

24. The Greedy Algorithm in Action 1 2 3 35 10 4 5 30 15 25 40 20 6 7 17 8 15 11 21 Node 1 2 3 4 5 6 7 First 1 4 5 4 5 4 5 35 10 30 15 25 40 20 17 8 15 11 21 1 2 3 4 5 6 7 73 24

25. The Greedy Algorithm in Action 1 2 3 35 10 4 5 30 15 25 40 20 6 7 17 8 15 11 21 Node 1 2 3 4 5 6 7 First 1 4 4 4 4 4 4 35 10 30 15 25 40 20 17 8 15 11 21 1 2 3 4 5 6 7 5 73 573 25

26. The Greedy Algorithm in Action 1 2 3 35 10 4 5 30 15 25 40 20 6 7 17 8 15 11 21 Node 1 2 3 4 5 6 7 First 4 4 4 4 4 4 4 35 10 30 15 25 40 20 17 8 15 11 21 1 2 3 4 5 6 7 57 3 1 26

27. Algorithmique Greedy 2Jose Rolim27 Complexity  Time complexity:  #make-set, find-set and union operations: O(|V| + |E|) O( (|V| + |E|)  (|V| + |E|))  Sorting: O(|E| log |E|) = O(|E| log |V|)  Total: O(|E| log |V|)

28. Algorithmique Greedy 2Jose Rolim28 Prim’s algorithm (also Greedy)  Start with an arbitrary node from V  Instead of maintaining a forest, grow a MST  At any time, maintain a MST for V’  V  At any moment, find a light edge connecting V’ with (V-V’)  I.e., the edge with smallest weight connecting some vertex in V’ with some vertex in V-V’ !

29. Algorithmique Greedy 2Jose Rolim29 Issues:  Again two issues:  Maintain the tree already build at any moment Easy: simply a tree rooted at r : the starting node  Find the next light edge efficiently For v  V - V’, define key(v) = the min distance between v and some node from V’ At any moment, find the node with min key.

30. Algorithmique Greedy 2Jose Rolim30 Pseudo-code

31. Algorithmique Greedy 2Jose Rolim31 Prim’s Algorithm in Action 1 2 3 35 10 4 5 30 15 25 40 20 6 7 17 8 15 11 21 4 5 6 7 1 2 3 The minimum cost arc from yellow nodes to green nodes can be found by placing arc values in a priority queue.

32. Algorithmique Greedy 2Jose Rolim32 Prim’s Algorithm in Action 1 3 35 4 5 30 15 25 40 20 6 7 17 8 15 11 21 4 5 6 7 1 35 2 2 10 25 10 2 3

33. Algorithmique Greedy 2Jose Rolim33 20 Prim’s Algorithm in Action 1 3 35 4 5 15 25 40 6 7 17 15 11 1 35 2 2 10 25 10 24 8 21 30 8 20 30 21 5 6 73 4

34. Algorithmique Greedy 2Jose Rolim34 20 Prim’s Algorithm in Action 1 3 35 4 5 15 25 40 6 7 17 15 11 1 35 2 2 10 25 10 24 8 21 30 8 20 30 21 6 8 17 15 573 64

35. Algorithmique Greedy 2Jose Rolim35 20 Prim’s Algorithm in Action 1 3 35 4 5 15 25 40 6 7 17 15 11 1 35 2 2 10 25 10 24 8 21 30 8 20 30 21 6 8 17 15 64 5 11 735

36. Algorithmique Greedy 2Jose Rolim36 20 Prim’s Algorithm in Action 1 3 35 4 5 15 25 40 6 7 17 15 11 1 35 2 2 10 25 10 24 8 21 30 8 20 30 21 6 8 17 15 64 5 11 735

37. Algorithmique Greedy 2Jose Rolim37 20 Prim’s Algorithm in Action 1 3 35 4 5 15 25 40 6 7 17 15 11 1 35 2 2 10 25 10 24 8 21 30 8 20 30 21 6 8 17 15 64 5 11 375 7

38. Algorithmique Greedy 2Jose Rolim38 20 Prim’s Algorithm in Action 1 3 35 4 5 15 25 40 6 7 17 15 11 1 35 2 2 10 25 10 24 8 21 30 8 20 30 21 6 8 17 15 64 5 11 7 735 15 3

39. Algorithmique Greedy 2Jose Rolim39 20 Prim’s Algorithm in Action 1 3 35 4 5 15 25 40 6 7 17 15 11 1 35 2 2 10 25 10 24 8 21 30 8 20 30 21 6 8 17 15 64 5 11 7 735 15 3

40. Algorithmique Greedy 2Jose Rolim40 20 Prim’s Algorithm in Action 1 3 35 4 5 15 25 40 6 7 17 15 11 1 35 2 2 10 25 10 24 8 21 30 8 20 30 21 6 8 17 15 64 5 11 7 735 15 3

41. Algorithmique Greedy 2Jose Rolim41 Complexity  Time complexity  # insert: O(|V|)  # Decrease-Key: O( |E|)  # Extract-Min O( |V| )  Using heap for priority queue:  Each operation is O ( log |V| )  Total time complexity: O (|E| log |V|)

42. Algorithmique Greedy 2Jose Rolim42 Single-Source Shortest Paths  Problem Definition  Shortest paths  Dijkstra’s algorithm (can be viewed as a greedy algorithm)

43. Algorithmique Greedy 2Jose Rolim43 Problem Definition:  Real problem: A motorist wishes to find the shortest possible route from Chicago to Boston.Given a road map of the United States on which the distance between each pair of adjacent intersections is marked, how can we determine this shortest route?  Formal definition: Given a graph G=(V, E, W), where each edge has a weight, find a shortest path from s to v for some interesting vertices s and v.  s—source  v—destination.

44. Algorithmique Greedy 2Jose Rolim44 Shortest path:  The weight of path p= is the sum of the weights of its constituent edges: The cost of the shortest path from s to v is denoted as  (s, v).

45. Algorithmique Greedy 2Jose Rolim45 Representing shortest paths:  we maintain for each vertex v  V, a predecessor [ v] that is the vertex in the shortest path right before v.  With the values of, a backtracking process can give the shortest path. (We will discuss that after the algorithm is given)

46. Algorithmique Greedy 2Jose Rolim46 Dijkstra’s Algorithm:  Dijkstra’s algorithm assumes that w(e)  0 for each e in the graph.  maintain a set S of vertices such that  Every vertex v  S, d[v]=  (s, v), i.e., the shortest-path from s to v has been found. (Intial values: S=empty, d[s]=0 and d[v]=  )  (a) select the vertex u  V-S such that d[u]=min {d[x]|x  V-S}. Set S=S  {u} (b) for each node v adjacent to u do RELAX(u, v, w).  Repeat step (a) and (b) until S=V.

47. Algorithmique Greedy 2Jose Rolim47 Implementation:  a priority queue Q stores vertices in V-S, keyed by their d[] values.  the graph G is represented by adjacency lists.

48. Algorithmique Greedy 2Jose Rolim48 0 10 5 2 1 34 2 6 9 7 s uv x y 88 88 (a)

49. Algorithmique Greedy 2Jose Rolim49 0 5/s 10/s 10 5 2 1 34 2 6 9 7 s uv x y 8 8 (b) (s,x) is the shortest path using one edge. It is also the shortest path from s to x.

50. Algorithmique Greedy 2Jose Rolim50 0 7/x 14/x 5/s 8/x 10 5 2 1 34 2 6 9 7 s uv x y (c)

51. Algorithmique Greedy 2Jose Rolim51 0 7/x 13/y 5/s 8/x 10 5 2 1 34 2 6 9 7 s uv x y (d)

52. Algorithmique Greedy 2Jose Rolim52 0 7/x 9/u 5/s 8/x 10 5 2 1 34 2 6 9 7 s uv x y (e)

53. Algorithmique Greedy 2Jose Rolim53 Time complexity of Dijkstra’s Algorithm:  Time complexity:  #Extract-min: O(|V|)  #Decrease-key = #Relax: O(|E|)  Use binary heap for priority heap: Time complexity: O((|V| + |E|) log |V|)  Use Fibonacci heap for priority heap: Amortized time complexity: O(|V| log |V| + |E|)  Greedy algorithm:  Any moment, select the lightest vertex in V-S to add to set S