1. Chapter 7: Greedy Algorithms 7.4 Finding the Shortest Path Dijkstra’s Algorithm pp. 295-301

2. Finding Shortest Paths Many real world situations can be represented using Graphs. Many real world applications and problems require finding the shortest path in a Graph. Consider the following “real world” example...

3. “Real World” Example 1 Ninja’s often find themselves in precarious situations where they must escape. Like when they are being chased by a Pirate disguised as Johnny Depp. A good Ninja always knows the shortest path to the escape route

4. Problem 1 Location of Ninja being chased by Pirate The only way out crazy Pirate maze. PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1

5. PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1 A  B  C  D  E  F  G  I  J  L  N  O  P  T  U  A  B  C  D  E  F  G  I  J  L  N  O  P  T  U   Keeps track of the shortest distance to each vertex and the path to get there. Initially, the distance is set to infinite and all the paths are null.

6. PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1 A  B  C  D  E  F 0 G  I  J  L  N  O  P  T  U  A  B  C  D  E  F 0 G  I  J  L  N  O  P  T  U   Initialization: Set the start vertex to zero. Represents that you can get to F with zero cost.

7. PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1 A    B    C    D    E    F  0  G    I    J    L  1 F N  14 F O    P    T    U    Step 1: Find the unmarked vertex with minimum distance (v). Mark it as visited Iterate over v’s adjacency list, and update the distances. Where can you go from F.

8. PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1 A  14 L B    C    D  13 L E    F  0  G    I  3 L J    L  1 F N  14 F O  7 L P    T    U    Step 2: Find the unmarked vertex with minimum distance (v). Mark it as visited Iterate over v’s adjacency list, and update the distances. Where can you go from F and L.

9. PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1 A  14 L B    C    D  13 L E    F  0  G    I  3 L J    L  1 F N  14 F O  7 L P  4 I T    U    Step 3: Find the unmarked vertex with minimum distance (v). Mark it as visited Iterate over v’s adjacency list, and update the distances. Where can you go from F, L, and I

10. PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1 A  14 L B    C    D  13 L E    F  0  G    I  3 L J    L  1 F N  14 F O  6 P P  4 I T    U  8 P Step 4: Find the unmarked vertex with minimum distance (v). Mark it as visited Iterate over v’s adjacency list, and update the distances. Where can you go from F, L, I and P

11. PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1 A  14 L B    C    D  13 L E    F  0  G    I  3 L J    L  1 F N  14 F O  6 P P  4 I T    U  7 O Step 5: Find the unmarked vertex with minimum distance (v). Mark it as visited Iterate over v’s adjacency list, and update the distances. Where can you go from F, L, I, P, and O.

12. PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1 A  14 L B    C    D  13 L E    F  0  G    I  3 L J  8 U L  1 F N  14 F O  6 P P  4 I T  9 U U  7 O Step 6: Find the unmarked vertex with minimum distance (v). Mark it as visited Iterate over v’s adjacency list, and update the distances.

13. PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1 A  14 L B    C    D  11 J E    F  0  G    I  3 L J  8 U L  1 F N  14 F O  6 P P  4 I T  9 U U  7 O Step 7: Find the unmarked vertex with minimum distance (v). Mark it as visited Iterate over v’s adjacency list, and update the distances.

14. PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1 A  14 L B    C    D  10 T E    F  0  G  12 T I  3 L J  8 U L  1 F N  14 F O  6 P P  4 I T  9 U U  7 O Step 8: Find the unmarked vertex with minimum distance (v). Mark it as visited Iterate over v’s adjacency list, and update the distances.

15. PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1 A  12 D B  15 D C  17 D D  10 T E    F  0  G  12 T I  3 L J  8 U L  1 F N  14 F O  6 P P  4 I T  9 U U  7 O Step 9: Find the unmarked vertex with minimum distance (v). Mark it as visited Iterate over v’s adjacency list, and update the distances.

16. PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1 A  12 D B  15 D C  17 D D  10 T E    F  0  G  12 T I  3 L J  8 U L  1 F N  14 F O  6 P P  4 I T  9 U U  7 O Step 10: Find the unmarked vertex with minimum distance (v). Mark it as visited Iterate over v’s adjacency list, and update the distances.

17. PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1 A  12 D B  15 D C  17 D D  10 T E    F  0  G  12 T I  3 L J  8 U L  1 F N  13 A O  6 P P  4 I T  9 U U  7 O Step 11: Find the unmarked vertex with minimum distance (v). Mark it as visited Iterate over v’s adjacency list, and update the distances.

18. PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1 A  12 D B  15 D C  15 N D  10 T E    F  0  G  12 T I  3 L J  8 U L  1 F N  13 A O  6 P P  4 I T  9 U U  7 O Step 12: Find the unmarked vertex with minimum distance (v). Mark it as visited Iterate over v’s adjacency list, and update the distances.

19. PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1 A  12 D B  15 D C  15 N D  10 T E  16 C F  0  G  12 T I  3 L J  8 U L  1 F N  13 A O  6 P P  4 I T  9 U U  7 O Step 13: Find the unmarked vertex with minimum distance (v). Mark it as visited Iterate over v’s adjacency list, and update the distances.

20. PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1 A  12 D B  15 D C  15 N D  10 T E  16 C F  0  G  12 T I  3 L J  8 U L  1 F N  13 A O  6 P P  4 I T  9 U U  7 O Step 14: Find the unmarked vertex with minimum distance (v). Mark it as visited Iterate over v’s adjacency list, and update the distances.

21. PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1 A  12 D B  15 D C  15 N D  10 T E  16 C F  0  G  12 T I  3 L J  8 U L  1 F N  13 A O  6 P P  4 I T  9 U U  7 O Step 15: Find the unmarked vertex with minimum distance (v). Mark it as visited Iterate over v’s adjacency list, and update the distances.

22. PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1 A  12 D B  15 D C  15 N D  10 T E  16 C F  0  G  12 T I  3 L J  8 U L  1 F N  13 A O  6 P P  4 I T  9 U U  7 O Final Step: This data structure can be used to compute the shortest path from F to all other vertices.

23. ABCDEFGIJLNOPTUABCDEFGIJLNOPTU L 14   13   3  1F 14 7    I 14   13   3L  14 7 4   P 14   13    14 6 4I  8 PILF OAN UJD C TGBE 121 4 2 4 6 12 13 14 1 2 1 2 7 4 54 3 2 1 4 135 3 5 1 F      0     1 14     O 14   13    14 6P  7 U 14   13   8 14 9 7O J 14   11   8U 14 9 T 14   10  12 14 9U D 12 15 17 10T  12 14 G 12 15 17  12T 14 A 12D 15 17  13 N 15 15  13A C 15 15N 16 B 15D 16 B 16C

24. “Real World” Lesson Learned Just do the Flip Out Dance to escape from the crazy Ninja maze with minimum damage.

25. Serious Lessons Learned Dijkstra’s algorithm is greedy. It always chooses the vertex with minimum distance from the source vertex. Optimization after each step Dijkstra’s algorithm updates the distances Shorter paths are discovered as minimum distances get updated.

26. Worst Case Finding the shortest path from a Start Vertex to a End Vertex could require marking every vertex. The Crazy Pirate Maze example showed us this fact. However, could we have stopped the algorithm after we marked the End Vertex as visited?

27. Example 2 PILF OAN UJD C TGBE 191 4 2 4 9 12 13 5 1 5 1 2 1 4 54 3 2 1 4 435 3 5 1 F  0 

28. Example 2 PILF OAN UJD C TGBE 191 4 2 4 9 12 13 5 1 5 1 2 1 4 54 3 2 1 4 435 3 5 1 F  0  L  1 F N  5 F

29. Example 2 PILF OAN UJD C TGBE 191 4 2 4 9 12 13 5 1 5 1 2 1 4 54 3 2 1 4 435 3 5 1 F  0  L  1 F N  5 F I  10 L O  10 L D  13 L A  14 L

30. Example 2 PILF OAN UJD C TGBE 191 4 2 4 9 12 13 5 1 5 1 2 1 4 54 3 2 1 4 435 3 5 1 F  0  L  1 F N  5 F A  6 N I  10 L O  10 L C  10 N D  13 L

31. Example 2 PILF OAN UJD C TGBE 191 4 2 4 9 12 13 5 1 5 1 2 1 4 54 3 2 1 4 435 3 5 1 F  0  L  1 F N  5 F A  6 N D  8 A I  10 L O  10 L C  10 N

32. Example 2 PILF OAN UJD C TGBE 191 4 2 4 9 12 13 5 1 5 1 2 1 4 54 3 2 1 4 435 3 5 1 F  0  L  1 F N  5 F A  6 N D  8 L C  9 D I  10 L O  10 L J  11 D G  11 D T  12 D B  13 D

33. Question: Is 9 the minimum distance from F to C? PILF OAN UJD C TGBE 191 4 2 4 9 12 13 5 1 5 1 2 1 4 54 3 2 1 4 435 3 5 1 F  0  L  1 F N  5 F A  6 N D  8 L C  9 D I  10 L I  10 L J  11 D G  11 D E  11 C T  12 D B  13 D

34. Question: How do we know there isn’t a shorter path? PILF OAN UJD C TGBE 191 4 2 4 9 12 13 5 1 5 1 2 1 4 54 3 2 1 4 435 3 5 1 F  0  L  1 F N  5 F A  6 N D  8 L C  9 D I  10 L I  10 L J  11 D G  11 D E  11 C T  12 D B  13 D

35. Technicality The textbook’s version of Dijkstra’s algorithm stops when all vertices are marked. which computes the shortest distance from a Start Vertex to all the other vertices. But, if you are only interested in the shortest path from A to B, you can stop the algorithm as soon as B is marked.

36. Worst Case Nonetheless, there are A to B problems that require all vertices to be marked. Thus, Dijkstra’s Algorithm takes O(N) steps. Now, in the worst case, how much work does one step take?

37. Worst Case Step Dijkstra’s table will have N entries, one for each vertex. You have to pick the unmarked entry with minimum distance. Given an array of N numbers, how long does it take to find the minimum? What would be the overall worst case running time of Dijkstra’s algorithm consider it has to find a minimum N times? ABCDEFGIJLNOPTUABCDEFGIJLNOPTU 14   13  3  2  1 14 6 4I  8 Find minimum

38. Worst Case Step N Steps Find minimum for each step O(N 2 ) so far... ABCDEFGIJLNOPTUABCDEFGIJLNOPTU 14   13  3  2  1 14 6 4I  8 Find minimum

39. Worst Case Step Worse yet, once we find the closest vertex, we must update all the distances. It is possible that the closest vertex could be adjacent to all vertices. Thus, there could be up to N updates per step. Does this impact the Big-O?

40. F D A BC 900 E 440 Worst Case Example A  0 B  440 C  900 D  900 E  900 F  900

41. F D A BC 900 E 440 Worst Case Example 440 210 A  0 B  440 C  630 D  880 E  880 F  880

42. F D A BC 900 E 440 Worst Case Example 440 210 100 A  0 B  440 C  630 D  670 E  730 F  730 100 40

43. F D A BC 900 E 440 Worst Case Example 440 210 100 A  0 B  440 C  630 D  670 E  680 F  690 100 40 20 10

44. F D A BC 900 E 440 Worst Case Example 440 210 100 A  0 B  440 C  630 D  670 E  680 F  685 100 40 20 10 5

45. D Worst Case Example A  0 B  440 C  630 D  670 E  680 F  685 F A BC 900 E 44 0 21 0 10 0 40 20 10 5 Step 1: N-1 updates Step 2: N-2 updates Step 3: N-3 updates... Step i: N-i updates... Step N-1: 1 update Step N: 0 updates Worst case on number of updates: The Problem: An update requires finding a vertex in the table. Find is O(N) Each of the N 2 updates required N operations.

46. D Worst Case Example A  0 B  440 C  630 D  670 E  680 F  685 F A BC 900 E 44 0 21 0 10 0 40 20 10 5 Step 1: N-1 updates Step 2: N-2 updates Step 3: N-3 updates... Step i: N-i updates... Step N-1: 1 update Step N: 0 updates Worst case on number of updates: Note: The worst case can only arise in a complete graph with N(N-1)/2 edges. The number of updates is really O(M), where M is the number of edges, which is bounded by O(N 2 )

47. Worst Case Algorithm while there are no un-marked vertices { find the vertex v with minimum weight mark it as visited for all of v’s edges { update the minimum distance and path to each vertex adjacent to v } N vertices * Find min: O(N) N-1 edges * 1 update: requires a find: O(N) N * (N + (N-1)*N) = O(N 3 )

48. Data Structures to the Rescue Is there a data structure we can use instead of an array. Goals: Find a vertex in constant time Update a distance in constant time Find minimum as efficiently as possible

49. Sorting & Binary Search: Sort the array by Distance while there are no un-marked vertices { find the vertex v with minimum weight mark it as visited for all of v’s edges { update the minimum distance and path to each vertex adjacent to v } re-sort the vertices by distance } N * (1 + (N-1)*N + N*log N) = O(N 3 ) N vertices * Find min: O(1) N-1 edges * 1 update: requires a find: O(N) Sort: O(N log N)

50. Sorting & Binary Search: Sort the array by Vertex Label while there are no un-marked vertices { find the vertex v with minimum weight mark it as visited for all of v’s edges { update the minimum distance and path to each vertex adjacent to v } N * (N + (N-1)*log N) = O(N 2 log N) N vertices * Find min: O(N) N-1 edges * 1 update: find: O(log N)

51. Hash Table: Constant time vertex lookup while there are no un-marked vertices { find the vertex v with minimum weight mark it as visited for all of v’s edges { update the minimum distance and path to each vertex adjacent to v } N * (N + (N-1)*1) = O(N 2 ) N vertices * Find min: O(N) N-1 edges * 1 update: find: O(1)

52. Binary Heap with Hash Index: Constant time vertex lookup Find min: O(log N) Updates: O(1) while there are no un-marked vertices { find the vertex v with minimum weight mark it as visited for all of v’s edges { update the minimum distance and path to each vertex adjacent to v } N * log N + M) = O(N log N + M) N vertices * Find min: O(log N) N-1 edges * 1 update: find: O(1)

53. Is there a difference?

54. PILF OAN UJD C TGBE 823 4 2 4 1 12 2 4 1 2 8 6 2 5 14 5 2 8 4 872 3 9 6