1. Graph Algorithms: Part 1
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Spring, 2010
Lecture 4
Tuesday, 2/23/10
Graph Algorithms: Part 1
Shortest Paths
Chapters 24-25

2. 91.404 Graph Review Elementary Graph Algorithms: Chapter 22
Minimum Spanning Trees: Chapter 23

3. Introductory Graph Concepts
G= (V,E)
Vertex Degree
Self-Loops
Undirected Graph
No Self-Loops
Adjacency is symmetric
Directed Graph (digraph)
Degree: in/out
Self-Loops allowed B E C F D A B E C F D A
This treatment follows textbook Cormen et al. Some definitions differ slightly from other graph literature.

4. Introductory Graph Concepts: Representations
Undirected Graph
Directed Graph (digraph) B E C F D A B E C F D A
A B C D E F A B C D E F
A B C D E F A B C D E F
A BC
B ACEF
C AB
D E
E BDF
F BE
A BC
B CEF C
D D
E BD
F E
Adjacency Matrix
Adjacency List
Adjacency List
Adjacency Matrix
This treatment follows textbook Cormen et al. Some definitions may differ slightly from other graph literature.

5. Introductory Graph Concepts: Paths, Cycles
length: number of edges
simple: all vertices distinct
Cycle:
Directed Graph:
<v0,v1,...,vk > forms cycle if v0=vk and k>=1
simple cycle: v1,v2..,vk also distinct
self-loop is cycle of length 1
Undirected Graph:
<v0,v1,...,vk > forms (simple) cycle if v0=vk and k>=3 B E C F D A
path <A,B,F> B E C F D A
simple cycle <E,B,F,E>
most of our cycle work will be for directed graphs
simple cycle <A,B,C,A>= <B,C,A,B> B E C F D A
This treatment follows textbook Cormen et al. Some definitions may differ slightly from other graph literature.

6. Introductory Graph Concepts: Connectivity
connected B E C F D A
Undirected Graph: connected
every pair of vertices is connected by a path
one connected component
connected components:
equivalence classes under “is reachable from” relation
Directed Graph: strongly connected
every pair of vertices is reachable from each other
one strongly connected component
strongly connected components:
equivalence classes under “mutually reachable” relation A B C
2 connected components D F E B E C F D A
not strongly connected
strongly connected component B E C F D A
This treatment follows textbook Cormen et al. Some definitions may differ slightly from other graph literature.

7. Elementary Graph Algorithms: SEARCHING: DFS, BFS
for unweighted directed or undirected graph G=(V,E)
Time: O(|V| + |E|) adj list O(|V|2) adj matrix
predecessor subgraph = forest of spanning trees
Breadth-First-Search (BFS):
Shortest Path Distance
From source to each reachable vertex
Record during traversal
Foundation of many “shortest path” algorithms
Depth-First-Search (DFS):
Encountering, finishing times
“well-formed” nested (( )( ) ) structure
Every edge of undirected G is either a tree edge or a back edge
EdgeColor of vertex when first tested determines edge type
Vertex color shows status:
not yet encountered
encountered, but not yet finished
finished
See DFS/BFS slide show
See DFS, BFS Handout for PseudoCode

8. Elementary Graph Algorithms: DFS, BFS
Review problem: TRUE or FALSE?
The tree shown below on the right can be a DFS tree for some adjacency list representation of the graph shown below on the left. A C B E D F
Tree Edge
Cross Edge
Back
Edge B E C F D A

9. Elementary Graph Algorithms: Topological Sort
for Directed, Acyclic Graph (DAG)
G=(V,E)
TOPOLOGICAL-SORT(G)
1 DFS(G) computes “finishing times” for each vertex
2 as each vertex is finished, insert it onto front of list
3 return list
Produces linear ordering of vertices.
For edge (u,v), u is ordered before v.
See also DFS/BFS slide show
source: textbook Cormen et al.

10. Minimum Spanning Tree: Greedy Algorithms
Invariant: Minimum weight spanning forest
Becomes single tree at end
Time: O(|E|lg|E|) given fast FIND-SET, UNION A B C D E F G 2 1 3 4 5 6 8 7
Produces minimum weight tree of edges that includes every vertex.
Time: O(|E|lg|V|) = O(|E|lg|E|) slightly faster with fast priority queue
Invariant: Minimum weight tree
Spans all vertices at end
for Undirected, Connected, Weighted Graph
G=(V,E)
source: textbook Cormen et al.

11. Minimum Spanning Trees
Review problem:
For the undirected, weighted graph below, show 2 different Minimum Spanning Trees. Draw each using one of the 2 graph copies below. Thicken an edge to make it part of a spanning tree. What is the sum of the edge weights for each of your Minimum Spanning Trees? A B C D E F G 2 1 3 4 5 6 8 7

12. Shortest Paths Chapters 24 & 25
Chapter 24: Single-Source Shortest Paths
(but not Section 24.4 on Difference Constraints)
Chapter 25: All-Pairs Shortest Paths
(but not Section 25.3 on Johnson’s Algorithms for Sparse Graphs)

13. BFS as a Basis for Some Shortest Path Algorithms
for unweighted, undirected graph G=(V,E)
Time: O(|V| + |E|) adj list
O(|V|2) adj matrix
Source/Sink Shortest Path
Problem: Given 2 vertices u, v, find the shortest path in G from u to v.
Solution: BFS starting at u. Stop at v.
Single-Source Shortest Paths
Problem: Given a vertex u, find the shortest path in G from u to each vertex.
Solution: BFS starting at u. Full BFS tree.
All-Pairs Shortest Paths
Problem: Find the shortest path in G from each vertex u to each vertex v.
Solution: For each u: BFS starting at u; full BFS tree.
Time: O(|V|(|V| + |E|)) adj list
O(|V|3) adj matrix
but for weighted, directed graphs…
source: based on Sedgewick, Graph Algorithms

14. Shortest Path Applications
edge weights
for weighted, directed graph G=(V,E)
adjacency linked list
adjacency matrix
Weight ~ Cost ~ Distance
Road maps
Airline routes
Telecommunications network routing
VLSI design routing
source: based on Sedgewick, Graph Algorithms

15. Shortest Path Trees
source: Sedgewick, Graph Algorithms
Example from previous slide:
New example with same vertices and edges but different edge weights:
different edge weights
Shortest Path Tree gives shortest path from root to each other vertex
Shortest Path Tree is a spanning tree.
Shortest path need not be unique, but shortest path total weight is unique.
Here assume 0 is root.
.99
.51
.38
.45
.83 .1
.41
.83 .1
.45
.21 .1
.51
.38
.36
.41 .5

16. Shortest Path Principles: Optimal Substructure
Lemma: Any subpath of a shortest path is a shortest path.
Proof: Cut-and-paste argument.
Let p be a shortest path from u to v. u x y v
pux
pxy
pyv p:
Suppose there existed a shorter path p’xy from x to y.
Then:
forming a path from u to v with smaller weight than path p contradicts the assumption that p is a shortest path. u x y v
pux
p’xy
pyv
source: textbook Cormen et al.

17. Shortest Path Principles: Relaxation
“Relax” a constraint to try to improve solution
Relaxation of an edge (u,v):
test if shortest path to v [found so far] can be improved by going through u A B C D E F G 2 1 3 4 5 6 8 7
Example:
source: textbook Cormen et al.

18. Shortest Path Principles (cont.)
for weighted, directed graph G=(V,E) with no negative-weight cycles
shortest path weight A B C D E F G 2 1 3 4 5 6 8 7
shortest path weight estimate
proof uses upper-bound property
proof uses convergence property
source: textbook Cormen et al.

19. Single-Source Shortest Paths
Chapters 24 & 25
Chapter 24: Single-Source Shortest Paths
(but not Section 24.4 on Difference Constraints)

20. Single Source Shortest Paths Bellman-Ford
for (negative) weighted, directed graph G=(V,E) with no negative-weight cycles
weights
source
why this upper bound?
Time is in O(|V||E|)
update d(v) if d(u)+w(u,v) < d(v)
detect negative-weight cycle
source: textbook Cormen et al.

21. with no negative-weight cycles
Bellman-Ford
for (negative) weighted, directed graph G=(V,E)
with no negative-weight cycles
Board work: Example of negative weight cycle detection.
Note: Edges are relaxed in lexicographic order ; other orders are possible.
source: textbook Cormen et al.

22. Single-Source Shortest Paths in DAG
Useful application: PERT charts for scheduling activities
source: materials to accompany textbook Cormen et al.

23. Single Source Shortest Paths: Dijkstra’s Algorithm
Dijkstra’s algorithm solves problem efficiently for the case in which all weights are nonnegative (as in the example graph).
Dijkstra’s algorithm maintains a set S of vertices whose final shortest path weights have already been determined. 1 2 3 4 6 5 10 8
It also maintains, for each vertex v not in S, an upper bound d[v] on the weight of a shortest path from source s to v.
The algorithm repeatedly selects the vertex
u e V – S with minimum bound d[u], inserts u into S, and relaxes all edges leaving u (determines if passing through u makes it “faster” to get to a vertex adjacent to u).

24. Single Source Shortest Paths: Dijkstra’s Algorithm
for (nonnegative) weighted, directed graph G=(V,E)
Similar to weighted version of BFS.
implicit DECREASE-KEY
Loop invariant: At start of each iteration of while loop, d [v] = d (s,v) for all v in S.
Greedy strategy: where is greedy choice being made?
source: textbook Cormen et al.

25. Single Source Shortest Paths Dijkstra’s Algorithm
Review problem:
For the directed, weighted graph below, find the shortest path that begins at vertex A and ends at vertex F. List the vertices in the order that they appear on that path. What is the sum of the edge weights of that path? A B C D E F G 2 1 3 4 5 6 8 7
Why can’t Dijkstra’s algorithm handle negative-weight edges?

26. Single Source Shortest Paths Dijkstra’s Algorithm
for (nonnegative) weighted, directed graph G=(V,E)
Fibonacci Heap (Ch. 20) O(VlgV + E) amortized analysis
PFS = Priority-First Search = generalize graph search with priority queue to determine next step.
Fringe = set of edges that are candidates for being added next to shortest path tree.
sources: Sedgewick, Graph Algorithms & textbook Cormen et al.

27. All-Pairs Shortest Paths
Chapter 25
Chapter 25: All-Pairs Shortest Paths
(but not Section 25.3 on Johnson’s Algorithms for Sparse Graphs)

28. Transitive Closure (Matrix): Unweighted, Directed Graph
Transitive Closure concepts will be useful for All-Pairs Shortest Path calculation in directed, weighted graphs G
“self-loops” added for algorithmic purposes
Transitive Closure Graph contains edge (u,v) if there exists a directed path in G from u to v.
source: Sedgewick, Graph Algorithms

29. Transitive Closure (Matrix)G G2
“self-loops” added for algorithmic purposes G G2
Boolean Matrix Product: and, or replace *,+
source: Sedgewick, Graph Algorithms

30. Transitive Closure (Matrix)G
why this upper limit?
Algorithm 1: Find G, G2 , G3 ,..., G|V-1|
Time: O(|V|4) G2
Algorithm 2: Find G, G2 , G4 ,..., G|V|
Time: O(|V|3lg|V|)
Algorithm 3: [Warshall]
for i to |V|-1
for s 0 to |V|-1
for t 0 to |V|-1
if G[s][i] and G[i][t]
then G[s][t] 1
Time: O(|V|3) G3 G4
source: Sedgewick, Graph Algorithms

31. Transitive Closure (Matrix)
Warshall
good for dense graphs (why?)
Consider 0 as maximum intermediate vertex label.
Consider 3 as maximum intermediate vertex label.
Consider 1 as maximum intermediate vertex label.
Consider 4 as maximum intermediate vertex label.
Consider 5 as maximum intermediate vertex label.
Consider 2 as maximum intermediate vertex label.
source: Sedgewick, Graph Algorithms

32. Transitive Closure (Matrix)
Warshall
Correctness by Induction on i:
Inductive Hypothesis: ith iteration of loop sets
G[s][t] to 1 iff there’s a directed path from s to t
with (internal) indices at most i.
Inductive Step for i+1 (sketch): 2 cases for path <s…t>
internal indices at most i
- covered by inductive hypothesis in prior iteration so
G[s][t] already set to 1
an internal index exceeds i (= i+1)
- G[s][i+1], G[i+1][t] set in a prior iteration so
G[s][t] set to 1 in current iteration
source: Sedgewick, Graph Algorithms

33. All Shortest Paths source: Sedgewick, Graph Algorithms
.41
.29
.29
.45
.21
.51
.32
.38
.36
.32
.50
total path weight from 1 to 0
source: Sedgewick, Graph Algorithms

34. All Shortest Paths (Compact)
Total distance of shortest path
Shortest Paths
.41
.29
.29
.45
.21
.32
.51
.38
.36
.32
.50
Entry s,t gives next vertex on shortest path from s to t (our textbook uses predecessor).
source: Sedgewick, Graph Algorithms

35. Slides courtesy of Prof. Pecelli
Algorithms for ALL shortest path pairs.
If all the edge weights were non-negative, we could use Dijkstra’s algorithm: O(|V|(|V| lg |V| + |E|)), which, for a dense graph, could be O(|V|3).
If we allow negative edge weights, we must use Bellman-Ford: O(|V|•|E|) from each vertex, which could be O(|V|2•|E|) = O(|V|4 ).
Can we do better? Is there some advantage in looking for algorithms that would compute ALL the path lengths simultaneously? We separate the problem into two parts: dense graphs and sparse graphs.
1/18/2019

36. Slides courtesy of Prof. Pecelli
Dense Graphs – the Floyd-Warshall Algorithm. The family of algorithms normally presented in this context is based on analogs of matrix multiplication - more precisely dynamic programming: graphs are represented as adjacency matrices rather than adjacency lists. This will lead us to the Floyd-Warshall algorithm. Johnson’s algorithm, used for sparse graphs and discussed in textbook (Section 25.3), uses adjacency lists.
Note: Social Networks (the rage today) seem to be based on “sparse graphs” (|E| = O(|V|*log|V|)) even though many nodes have large numbers of edges.
1/18/2019

37. Slides courtesy of Prof. Pecelli
Some notation.
G = (V, E); |V| = n;
W = (wij);
wij = weight of edge (i, j); wii = 0, wij = ∞ if (i, j) not in E.
Negative edges allowed; no negative cycles.
D = (dij) = weight matrix for shortest paths: dij = d(i, j).
P = (pij) = predecessor matrix on shortest paths: predecessor of j in some shortest path from i to j; pij = NIL if i = j or (i, j) not in E.
ith row of P = shortest path tree with root i.
Gp = predecessor graph; Gp,i = (Vp,i, Ep.i) = predecessor subgraph for i; Vp,i = {j in V: pij ≠ NIL}U{i}; Ep,j = {(pij, j): j in Vp,j – {j}}.
1/18/2019

38. Slides courtesy of Prof. Pecelli
Shortest Paths & Matrix Multiplication.
We need to first characterize the structure of the solutions – we try a dynamic programming approach, which will also give us a way to carry the program to success.
Characterize the structure of an optimal solution
Recursively define the value of an optimal solution
Compute the value of an optimal solution in a bottom up fashion.
Construct the optimal solution from computed information.
1/18/2019

39. Slides courtesy of Prof. Pecelli
Structure of a shortest path.
By Lemma 24.1 we know that subpaths of shortest paths are shortest paths. Let W = (wij) be the adjacency matrix of the graph. Let p be a path from i to j, p with at most m edges (no negative cycles implies p is finite).
if i = j, p has weight 0 and no edges
if i ≠ j, p can be decomposed into i –p’ k j, where p’ has at most m-1 edges. Lemma 24.1 gives p’ as a shortest path from i to k, d(i,j) = d(i, k) + wkj.
We are now ready to set up a recursive Dynamic Programming algorithm to compute the solution.
1/18/2019

40. Slides courtesy of Prof. Pecelli
Recursive Solution.
Let lij(m) denote the min weight of any path from i to j using no more than m edges. The recursion gives:
The actual shortest path lengths eventually stabilize:
d(i, j) = lij(n-1) = lij(n) = lij(n+1) …
(25.2)
1/18/2019

41. Slides courtesy of Prof. Pecelli
Shortest Paths & Matrix Multiplication.
We can translate this idea from (25.2) into the pseudo-code:
Where the input matrices are W and L(m) and the output is L(m+1).
1/18/2019

42. Slides courtesy of Prof. Pecelli
If we look at the operations min and +, and replace them by + and *, we see a correspondence with matrix multiplication: Each iteration of the algorithm has cost Q(n3). Because of the previous argument on termination, the total cost is Q(n4).
Substitutions:
1/18/2019
similar to Transitive Closure Algorithm 1

43. Running Time is in Q(n4). similar to Transitive Closure Algorithm 1
1/18/2019 43

44. Slides courtesy of Prof. Pecelli
We can speed up the running time to Q(n3log(n)) by using a well-know trick in matrix power computations We’ll see this same trick used to good effect in the implementation of RSA encryption.
similar to Transitive Closure Algorithm 2
1/18/2019

45. Slides courtesy of Prof. Pecelli
Floyd-Warshall: can we find a better dynamic programming algorithm (e.g., Q(|V|3))? – still with no negative-weight cycles.
The only option open to us (short of finding a much better matrix multiplication algorithm) is to find a more efficient characterization of the structure of a shortest path – which is what allows us to use the dynamic programming framework.
Def.: an intermediate vertex of a simple path p = <v1, v2, …, vl> is any vertex of p other than v1 or vl.
1/18/2019

46. Slides courtesy of Prof. Pecelli
Observation: if V = {v1, v2, …, vn} = {1, …, n}, consider a subset {v1, v2, …, vk} for some k. For any pair of vertices i, j in V, consider all the paths from i to j whose intermediate vertices are all drawn from {1, 2, …, k}. Let p be a (simple) minimum-weight path among them.
If k is not an intermediate vertex of p, then all intermediate vertices of p are in {1, 2, …, k-1}. Thus a shortest path from i to j with all intermediate vertices in {1, 2, …, k-1} is a shortest path from i to j with all intermediate vertices in {1, 2, …, k}.
If k is an intermediate vertex of p, then decompose p = i –p1-> k –p2-> j. By Lemma 24.1, p1 is a shortest path from i to k with all intermediate vertices in {1, 2, …, k}. Since k is not an intermediate vertex of p1, all the intermediate vertices are in {1, 2, …, k-1}: p1 is a shortest path from i to k with intermediate vertices in {1, 2, …, k-1}; similarly p2 from k to j.
1/18/2019

47. Slides courtesy of Prof. Pecelli
A picture that might help:
1/18/2019

48. Slides courtesy of Prof. Pecelli
A recursive solution. Let dij(k) be the weight of a shortest path from i to j with intermediate vertices in {1, …, k}. When k = 0, the path can have no intermediate vertices, so dij(0) = wij. From the discussion on the previous slides: Since for any path all intermediate vertices are in {1, …, n}, dij(n) = d(i, j).
1/18/2019

49. Slides courtesy of Prof. Pecelli
Running Time is in Q(n3).
similar to Transitive Closure Algorithm 3 [Warshall]
1/18/2019

50. Shortest Path Algorithms
source: Sedgewick, Graph Algorithms

51. Connected Dominating Set Literature Paper
“On Calculating Connected Dominating Set for Efficient Routing in Ad Hoc Wireless Networks” by Wu and Li, DIAL M 1999.

52. Goal & Approach of Paper
Goal: Efficient routing for mobile hosts
All hosts have same wireless transmission range.
Approach:
Represent network by undirected, unweighted graph G = (V, E).
2 nodes connected if both hosts are within transmission range
Find small, connected, dominating set of gateway nodes
In addition, this set must include all intermediate nodes on every shortest path between 2 nodes in dominating set.
All-pairs shortest paths
Maintain routing table for gateway nodes.
Use heuristic, due to suspected intractability of minimum dominating set problem. 1 2 3 4 5 6 7 8 9
source: “On Calculating Connected Dominating Set for Efficient Routing in Ad-Hoc Wireless Networks”

53. Dominating Set 1 2 3 4 5 6 7 8 9
source: Garey & Johnson

54. Marking Heuristic Initialize each node’s marker to F (false)
Each vertex v exchanges its open neighbor set N(v) with each of its neighbors
Each vertex changes its marker to T (true) if it can determine that it has 2 (directly) unconnected neighbors.
Resulting V ’ induces reduced graph G ’ = G [V ’ ].
THEOREM 1 (reworded): If G is (not completely) connected, then V ’ is a dominating set of G.
THEOREM 2 (reworded): G ’ is connected.
THEOREM 3 (reworded): Shortest path between any 2 nodes of G ’ does not include any non-gateway node as an intermediate node. 1 2 3 4 5 6 7 8 9
source: “On Calculating Connected Dominating Set for Efficient Routing in Ad-Hoc Wireless Networks”

55. Enhanced Marking Heuristic
RULES 1 & 2 try to reduce number of gateway nodes
assuming ordering on vertex id’s in V ’
define closed neighbor set of v in V ’:
RULE 1: Consider 2 vertices v and u in G ’ . If in G and id(v) < id(u), change marker of v to F if node v is marked.
RULE 2: Given 2 marked neighbors u and w of marked vertex v in G ’, if
in G and then change marker of v to F.
Paper also discusses dynamic network with hosts turning on and off.
Experimental simulation uses random graphs.
RULE 1
RULE 2 v u v u u v w
id=1
id=2
source: “On Calculating Connected Dominating Set for Efficient Routing in Ad-Hoc Wireless Networks”