1. CS330 Discussion 6

2. Shortest path In lecture, the following problem was discussed:
Input: A graph 𝐺(𝑉,𝐸), with edge weights 𝑐 𝑒 , and 𝑠, 𝑡∈𝑉
Output: The shortest path (length) from 𝑠 to 𝑡 in 𝐺
and Dijkstra’s algorithm was found to be the optimal algorithm.
If instead we want to find the shortest path from 𝑠 to all vertices, we can still use Dijkstra’s.
However, if we want to find the shortest path from all vertices to all vertices, the problem becomes much more difficult.

3. All pairs’ shortest path (APSP)
The problem is defined as follows:
Input: A graph 𝐺(𝑉,𝐸), with edge weights 𝑐 𝑒
Output: For all 𝑠, 𝑡 pairs, the shortest path (length) from 𝑠 to 𝑡 in 𝐺
To simplify the algorithm, we will also require that no negative cycles exist in the graph.
To solve this, we’ll use dynamic programming.

4. Breaking down the problem
Suppose we know the shortest paths from all 𝑖∈𝑉 to all 𝑗∈𝑉, which don’t use 𝑘 as an intermediate vertex.
That is, if the shortest path from 𝑖 to 𝑗 uses 𝑘 we don’t know that path, but if the second shortest path doesn’t use 𝑘, we know that path.
We also know the shortest path from any vertex to 𝑘 and from 𝑘 to any vertex, since in neither case is 𝑘 is an intermediate vertex.

5. Breaking down the problem
Suppose we know the shortest paths from all 𝑖∈𝑉 to all 𝑗∈𝑉, which don’t use 𝑘 as an intermediate vertex.
There’s two cases to consider for each 𝑖, 𝑗 pair:
𝑘 is on the shortest path from 𝑖 to 𝑗. What is the shortest 𝑖, 𝑗 path length here?
𝑘 isn’t on the shortest path from 𝑖 to 𝑗. What is the shortest 𝑖, 𝑗 path length here?
Once we know these two path lengths, which do we pick?

6. Generalization
Let’s generalize the previous problem – suppose we know all shortest paths for all 𝑖, 𝑗 pairs using only intermediate vertices in 𝑆, a subset of 𝑉, and we want to know all the shortest paths using only intermediate vertices in 𝑆∪{𝑘}.
Again, there are two cases:
The shortest path using only 𝑆∪ 𝑘 contains 𝑘. What is the shortest 𝑖, 𝑗 path length using only 𝑆∪ 𝑘 in this case?
The shortest path using only 𝑆∪{𝑘} doesn’t contain 𝑘. What is the shortest 𝑖, 𝑗 path length using only 𝑆∪ 𝑘 in this case?

7. Recursive formulation
Let us define an arbitrary order on the vertices, i.e. let 𝑉= {1, 2, 3, …𝑛}.
Let 𝑓(𝑖, 𝑗, 𝑘) be the shortest path length from 𝑖 to 𝑗 using only intermediate vertices {1, 2, 3, …𝑘}.
Then:
If 𝑘 is on the shortest path, 𝑓 𝑖, 𝑗, 𝑘 =𝑓 𝑖, 𝑘, 𝑘−1 +𝑓(𝑘, 𝑗, 𝑘−1)
Else, 𝑓 𝑖, 𝑗, 𝑘 =𝑓(𝑖, 𝑗, 𝑘−1)
Equivalently, 𝑓 𝑖, 𝑗, 𝑘 = min 𝑓 𝑖,𝑗,𝑘−1 , 𝑓 𝑖, 𝑘, 𝑘−1 +𝑓 𝑘, 𝑗, 𝑘−1
In what order should we solve the subproblems? (i.e. how should we iterate over 𝑖,𝑗,𝑘?

8. Base cases
For the base cases – if we can’t use any intermediate vertices, then what is the shortest path from 𝑖 to 𝑗?
What’s the shortest path from a vertex to itself?

9. First version
def APSP(G): SP = 3-D array indexed 1 to n, 1 to n, 0 to n For each vertex i: SP[i][i][0] = 0 For each edge (i, j): SP[i][j][0] = e(i,j) For k from 1 to n: For i from 1 to n: For j from 1 to n: SP[i][j][k] = min(SP[i][j][k-1], SP[i][k][k-1]+SP[k][j][k-1]) Return SP[1:n][1:n][n]

10. Analysis
Our DP table is clearly size 𝑂( 𝑛 3 ), and each update takes 𝑂(1) time, so the runtime is simply 𝑂( 𝑛 3 ).
This runtime is pretty good – there are various improvements, but they come with nuances.
However, we can very easily improve on the spatial complexity.

11. Improving the algorithm
One nice property of this algorithm: If we compute in increasing 𝑘 order, once we compute 𝑓(𝑖, 𝑗,𝑘), we don’t need to store 𝑓(𝑖, 𝑗, 𝑘− 1).
We won’t use 𝑓(𝑖, 𝑗, 𝑘−1) to compute any 𝑓 𝑢, 𝑣, 𝑘+1 , 𝑓 𝑢, 𝑣, 𝑘+2 … because the difference in the third argument is 2 or greater.
We won’t use 𝑓(𝑖, 𝑗, 𝑘−1) to compute any 𝑓(𝑢, 𝑣, 𝑘) unless 𝑖=𝑘 or 𝑗=𝑘, in which case 𝑓(𝑖,𝑗,𝑘) and 𝑓(𝑖, 𝑗, 𝑘−1) are the same.
Thus, at any stage in the algorithm, we only need to store 𝑓(𝑖, 𝑗, 𝑘) for the most recent 𝑘 value.

12. Second version
def APSP(G): SP = 2-D array indexed 1 to n, 1 to n For each vertex i: SP[i][i] = 0 For each edge (i, j): SP[i][j] = e(i,j) For k from 1 to n: For i from 1 to n: For j from 1 to n: SP[i][j]= min(SP[i][j], SP[i][k]+SP[k][j]) Return SP

13. Improvements
Now, we only require 𝑂( 𝑛 2 ) space at no cost to correctness or runtime.
This is the Floyd-Warshall algorithm. The algorithm was developed in 1962, and until 2014, all improvements on it required the graph to hold specific properties.