1. Algorithms: Design and Analysis
, Semester 2,
9. Shortest Paths
Objectives
Look at two types of shortest path problem:
SSSP (single source shortest path):
BFS, Dijkstra, Bellman-Ford
APSP (all pairs shortest path):
Floyd-Warshall

2. 1. Shortest Paths
The length of a path = the sum of the weights of the edges in the path.
The shortest path between two verticies = the path having the minimum length.
849
PVD
1843
ORD
142
SFO
1743
LGA
1205
337
802
1387
HNL
2555
1099
LAX
1233
DFW
1120
MIA

3. Shortest Path Tree (SPT)
A shortest path tree spans all the nodes in the graph, using edges that give the shortest paths from the starting node to all the other vertices.
start
node
is 0
SPT

4. Types of Shortest Path Problems
SSSP: single source shortest path
find the shortest path from a given vertex to all the other nodes
for undirected, unweighted graphs use BFS
for directed, weighted graph use:
Dijkstra's algoritm, Bellman-Ford
APSP: all pairs shortest path
find the shortest path between all pairs of nodes in the graph
Floyd-Warshall for small graphs

5. 2. Dijkstra’s Algorithm O(|E| * log |V|) Similar to Prim's algorithm
Grow a tree gradually,taking vertices from a priority queue ordered by "distance from the start node"
Instead of a MST, we are building a SPT
the algorthm is greedy: once a node has been added to the tree, the choice is fixed
A vertex's "distance from the start node" is calculated using relaxing

6. Pseudocode
void Dijkstra(start, Graph) { for each vertex v in Graph { distTo[v] = infinity prev[v] = null } distTo[start] = 0 // distance from start to start == 0 Q = priority queue of all nodes in Graph while Q is not empty { v = node in Q with smallest distTo[] remove v from Q; // no need to add v to a SPT for each neighbor n of v { d = distTo[v] + weight(v, n) if d < distTo[n] { distTo[n] = d prev[n] = v
relaxing

7. Algorithm in Words Each vertex has a entry in a distTo[] array
distTo[v] == v's distance from the start node
Initialize dist[start] to 0 and all other distTo[] entries to infinity ()
At each iteration, the SPT is grown by adding the vertex v with the smallest distTo[] value.
The nodes adjacent to v have their distTo[] values reduced; this is called relaxing.

8. What is Relaxing? b b 5 5 v v 4 4 c c relax nodes linked to v 8 8
distTo[b] = 13
distTo[b] = 8 v v 4 4 c
distTo[a] = 3
distTo[a] = 3 c
relax
nodes
linked
to v
distTo[c] = 7
distTo[c] = 13 8 8
v is added
to SPT d d
distTo[d] = 10
distTo[d] = 10
for each neighbor n of v
d = distTo[v] + weight(v, n)
if d < distTo[n]
distTo[n] = d

9. Data Structures Adjacency list for a directed, weighted graph.
distTo[] holding distances from the starting vertex to the other vertices
A priority queue of (distTo values, vertex) pairs sorted based on increasing distances
i.e. smallest distance is first
prev[v] contains v's parent in the SPT
used to calculate the path from start node to v

10. We dont need to maintain a tree data structure for the SPT.
Instead prev[] is used to reconstruct the paths through the SPT.

11. Example 1 1 2 2 2 4 1 2 3 7 3 4 4 3 7 1 6
Start Vertex: 0 5 6 5
continued

12. Initialisation 2 2 4 1 2 3 4 3 7 1 6 5 = changing distTo Inf 1 2 Inf
prev[]
0 :
1 :
2 :
3 :
4 :
5 :
6 :
7 : 2 4 1 2 3
Inf 7
Inf 3 4
Inf 4 3 7 1 6 5 6
Inf 5
Inf
continued

13. First Iteration 2 2 4 1 2 3 4 3 7 1 6 5 = final distTo 2 1 2 Inf
prev[]
0 :
1 :
2 :
3 :
4 :
5 :
6 :
7 : 2 4 1 2 3
Inf 7
Inf 3 4
Inf 4 3 7 1 6 5 6
Inf 5 1
continued

14. Second Iteration 2 2 4 1 2 3 4 3 7 1 6 5 2 1 2 Inf prev[] 0 : 1 : 2 :
3 :
4 :
5 : 0
6 :
7 : 2 4 1 2 3
Inf
Inf 7
Inf 3 4 4 3 7 1 6 5 6 6 5 1
continued

15. Third Iteration 2 2 4 1 2 3 4 3 7 1 6 5 2 1 2 4 prev[] 0 : 1 : 0 2 :
3 :
4 :
5 : 0
6 :
7 : 2 4 1 2 3 4 6 7
Inf 3 4 4 3 7 1 6 5 6 6 5 1
continued

16. Fourth Iteration 2 2 4 1 2 3 4 3 7 1 6 5 2 4 1 2 prev[] 0 : 1 : 0
2 : 1
3 :
4 :
5 : 0
6 :
7 : 2 4 1 2 3 4 6 7 5 3 4 4 3 7 1 6 5 6 6 5 1
Could have chosen
‘node 3’ instead.
continued

17. Fifth Iteration 2 2 4 1 2 3 4 3 7 1 6 5 2 4 1 2 prev[] 0 : 1 : 0 2 : 1
3 : 1
4 :
5 : 0
6 :
7 : 2 4 1 2 3 4 6 7 5 3 4 4 3 7 1 6 5 5 6 6 1
continued

18. Sixth Iteration 2 2 4 1 2 3 4 3 7 1 6 5 2 4 1 2 prev[] 0 : 1 : 0 2 : 1
3 : 1
4 :
5 : 0
6 :
7 : 2 2 4 1 2 3 4 7 5 3 4 6 4 3 7 1 6
Shortest path from
node 0 to node 7 is
0 -> 1 -> 2 -> 7,
distance 5. 5 6 6 5 1

19. 7th Iteration 2 2 4 1 2 3 4 3 7 1 6 5 2 4 1 2 prev[] 0 : 1 : 0 2 : 1
3 : 1
4 : 1
5 : 0
6 :
7 : 2 2 4 1 2 3 4 7 5 3 4 6 4 3 7 1 6 5 6 5 6 1
Could have chosen
‘node 6’ instead.

20. 8th Iteration & finish 2 2 4 1 2 3 4 3 7 1 6 5 2 4 1 2 prev[] 0 :
1 : 0
2 : 1
3 : 1
4 : 1
5 : 0
6 : 5
7 : 2 2 4 1 2 3 4 7 5 3 4 6 4 3 7 1 6 5 6 5 6 1

21. Example 2

23. Example 3
SPT
from 0 37
prev[] 5 7 4 3 2

24. Running Time
Dijkstra's original algorithm did not use a priority queue and so ran in O(|V|2)
it used an array to store the vertices
The priority queue implemented using a minimum heap runs in O(|E| * log |V|)

25. Why O(E log V)? in a dense graph E ≈ V2, so is bigger than V
dijkstra
O(V)
for
while
look at every vertex
O(E*log V)
look at
all edges
= E iters
relax
for
look at every edge from a vertex
queue ops
O(log V)
due to minheap impl.
of the priority queue
the queue contains
at most V vertices

26. Why O(log V)? minimum binary heap Fibonacci Heap (multiple trees)
PQ Operation
Array
Binary heap
d-ary Heap
Fib heap †
remove() V
log V
log V
log V
change() 1
log V
log V 1
Total V2
E log V
E log E/V V
V log V + E
† Individual ops are amortized analysis
minimum
binary
heap
Fibonacci Heap
(multiple trees)

27. Why Doesn’t Dijkstra Work for Negative Weight Edges?
Adding a negative edge can change the distTo[] values for vertices already in the SPT.
This breaks the algorithm since distTo[] is assumed to not change after its node is selected A 4
SPT
vertices 5 1 4 C D -8 5 F 9
distTo[c] will change from 5 to = 1

28. 3. Bellman-Ford
Bellman-Ford can be used on graphs with negative edge weights
as long as the graph contains no negative cycles reachable from the start node
Slower than Dijkstra's algorithm in the worst case
O(|V| * |E|)
Better than Dijkstra in the average case
O(|V| + |E|)

29. Negative Cycles Problem
In graphs with negative weight cycles, some shortest paths will not exist: -4
What is distTo[x]
starting from s? w v 2 1 s t u x 2 1 1

30. Example
This example
can be processed by Bellman-Ford since there are no negative cycles. 29 32 28 39 52 34 28 35 35 26 37
-120 38
-140
-125

31. SPT Using Bellman-Ford
prev[] 5 7 6 4 3 2 32 39 34 52 35 26
-125
distTo[4] = – 125 = 26

32. Algorithm in Words Bellman-Ford relaxes all the edges|V | − 1 times.
The biggest path in the SPT is at most |V|-1 edges long. So |V|-1 loops is certain to process it
assuming there are no negative cycles
Worse case running time is O(|V|*|E|)
The order in which edges are processed affects how quickly the algorithm finishes

33. Example The start node is s t x s y z5
distTo[]
prev[] t x 6 s t x y z  - -2 -3 8 s -4 7 7 2 y z 9
after each relax for-loop iteration 5
distTo[]
prev[] t x 6 s t x y z 6  7 - s -2 -3 8 s -4 7 7 2 y z 9

34. t x s y z t x s y z t x s y z Finished 5 distTo[] prev[] 6 s t x y z 66 4 7 2 - s y t -2 -3 8 s -4 7 7 2 y z 9 5 t x 6 s t x y z 2 4 7 - x y s t -2 -3 8 s -4 7 7 2 y z 9 t x 6 s t x y z 2 4 7 -2 - x y s t -2 -3 8 s -4 7 7 2 y z 9
Finished

35. Data Structures Adjacency list for a directed, weighted graph.
distTo[] array of distances from the starting vertex to the other vertices
Unlike Dijkstra, there is no need for a priority queue of (distances, vertex) pairs
this makes the code simpler
prev[v] contains v's parent in the SPT
used to calculate the shortest path from start node to v

36. 4. Floyd-Warshall Used for all pairs shortest path (APSP) problems.
If you can get from A to B at cost c1, and you can get from B to C with cost c2, then you can get from A to C with at most cost c1+c2
As the algorithm proceeds, if it finds a lower cost for A to C then it uses that instead.
Running time: O(|V|3) B c1 c2 A C
at most c1 + c2

37. Example to 1 2 3 4 8 13 - 1 6 12 9 7 11 1
from 2 3 4
D0 = (dij )
dij = shortest distance from i to j through no nodes
dij = shortest distance from i to j through nodes {0, …, k} k

38. dij = shortest distance from i to j through {0, …, k}
min(
k-1
k-1
k-1
dij ,
dik )
dkj
Before
After 1 2 3 4 1 2 3 4 8 13 - 1 6 12 9 7 11 8 13 - 1 1 1 - - 6 12 2 2 - 9 - - 3 3 7 15 8 4 4 - - - 11
D0 = (dij )
D1 = (dij ) 1
dij = shortest distance from i to j through {0, …, k} k

39. D4 = (dij ) D2 = (dij ) D3 = (dij ) D5 = (dij ) Finished 8 13 14 1 88 13 14 1 8 13 14 1 1 - - 6 12 1 13 6 6 12 2 - 9 15 21 2 22 9 15 21 3 7 15 8 3 7 9 8 4 - - - 11 4 18 20 11 11 4
D2 = (dij ) 2
D4 = (dij ) 8 13 14 1 8 12 12 1 1 - - 6 12 1 13 6 6 12 2 - 9 15 21 2 22 9 15 21 3 7 9 8 3 7 9 8 4 - - - 11 4 18 20 11 11
D3 = (dij ) 3
D5 = (dij ) 5
Finished

40. Result

41. Data Structures A 2D matrix for holding the distances.
There's no need for a priority queue (as in Dijkstra) or a distTo[] array (as in Dijkstra and Bellman-Ford).
To calculate paths, another 2D matrix can hold the v's previous vertices for different start nodes

42. Pseudocode dij = min( dij , dik + ) dkj
fill dist|V| × |V| = ∞ for each vertex v dist[v][v] = 0 for each edge (u,v) dist[u][v] = weight(u,v) for k from 1 to |V| for i from 1 to |V| for j from 1 to |V| d = dist[i][k] + dist[k][j] if dist[i][j] > d dist[i][j] = d
dij = k
min(
k-1
k-1
k-1
dij ,
dik )
dkj

43. Comparison with Dijkstra & Bellman-Ford
Use Dijkstra or Bellman-Ford by calling them repeatedly with every vertex as the start node
Floyd-Warshall running time is O(|V|3):
Dijkstra running time: |V| * O(|E| log |V|) = O(|V|3 log|V|)
Bellman-Ford running time: |V| * O(|V|*|E|) = O(|V|4)
Floyd-Warshall is easy to implement, but memory will become a problem for large matricies (|V| > 400)