1. 15-211 Fundamental Data Structures and Algorithms
Graph Algorithms
Fundamental Data Structures and Algorithms
Aleks Nanevski
March 18, 2004

2. Announcements Homework Assignment #6 due April 5
Building a Web Search Engine
Web Crawler (due March 25)
Web Reader (due March 25)
Web Search (due April 5)
Reading: Chapter 14

3. Recap

4. Graphs — an overview Vertices (aka nodes) BOS DTW SFO PIT JFK LAX
618
DTW
SFO
2273
211
190
PIT
318
JFK
344
1987
2145
2462
LAX
Edges
Weights
(Undirected)

5. Definitions Graph G = (V,E) Weighted graph Directed graph (digraph)
Set V of vertices (nodes)
Set E of edges
Elements of E are pair (v,w) where v,w  V.
An edge (v,v) is a self-loop. (Usually assume no self-loops.)
Weighted graph
Elements of E are ((v,w),x) where x is a weight.
Directed graph (digraph)
The edge pairs are ordered
Undirected graph
The edge pairs are unordered
E is a symmetric relation
(v,w)  E implies (w,v)  E
In an undirected graph (v,w) and (w,v) are the same edge

6. Representing graphs Adjacency matrix Adjacency lists1 2 3 4 5 6 7 x
Adjacency matrix
2-dimensional array
For each edge (u,v), set A[u][v] to true; otherwise false
Adjacency lists
For each vertex, keep a list of adjacent vertices 1 2 3 4 5 6 7 3 4 5 6 7 1 4 7 6 3 5 2
Q: How to represent weights?

7. Representing graphs We will assume adjacency list representation
For each vertex v, denote the adjacency list of v as adj(v)

8. Graph Traversal

9. Graph Traversals Explore all vertices reachable from some vertex s.
One of the fundamental operations in graphs. Find things such as
Count the total edges
Output the content in each vertex
Identify connected components

10. Graph Traversals Before/during the tour - mark vertices as
Completed (v and adj(v) are visited)
Frontier (v visited, but adj(v) not)
Not visited
Data structures
C: list of completed nodes
F: list representing frontier

11. Graph Traversals
Both take time: O(|V|+|E|)

12. Depth-first Search C = [] // empty list F = [s] // singleton list
while (F <> []) {
x = head(F); F = tail(F);
C = x, C; // mark x as completed
for each y in adj(x) do
if (y not in C) then F = y, F }

13. Depth-first Search
Completed
empty
Frontier s f c s b e a g d

14. Depth-first Search
Completed s
Frontier c b a f c s b e a g d

15. Depth-first Search
Completed s c
Frontier f b a f c s b e a g d

16. Depth-first Search
Completed s c f
Frontier e b a f c s b e a g d

17. Depth-first Search
Completed s c f e
Frontier b a f c s b e a g d

18. Depth-first Search
Completed s c f e b
Frontier d b a f c s b e a g d

19. Depth-first Search
Completed s c f e b d
Frontier a b f c s b e a g d

20. Depth-first Search f c s b e a g d Completed s c f e b d a Frontier
empty f c s b e a g d

21. Depth-first Search Notice: Frontier list is used like a stack.
Simpler implementation using recursion.
dfs( vertex x ) {
mark x as completed; // put x in C
forall y in adj(x) do
if( y not marked )
dfs( y ); // explore edge }

22. Depth-first Search Complexity: Overall cost: O(|E|+|V|)
each node marked on C only once for a total of O(|V|) operations.
each node put on F once for each of its incoming edges, for a total of O(|E|) operations.
Overall cost: O(|E|+|V|)

23. Breadth-first Search C = [] // empty list F = [s] // singleton list
while (F <> []) {
F’=[];
for each x in F {
for each y in adj(x) do
if (y not in C and y not in F) F = F,y }
C = C, F // mark all of F as completed
F = F’

24. Breadth-first Search
Completed
empty
Frontier s f c s b e a g d

25. Breadth-first Search
Completed s
Frontier c b a f c s b e a g d

26. Breadth-first Search
Completed s c
Frontier b a f f c s b e a g d

27. Breadth-first Search
Completed s c b
Frontier a f e d f c s b e a g d

28. Breadth-first Search
Completed s c b a
Frontier f e d f c s b e a g d

29. Breadth-first Search
Completed s c b a f
Frontier e d f c s b e a g d

30. Breadth-first Search
Completed s c b a f e
Frontier d f c s b e a g d

31. Breadth-first Search f c s b e a g d Completed s c b a f e d Frontier
empty f c s b e a g d

32. Breadth-first Search Notice: frontier list used like a queue.
Implementation using a queue:
F.enqueue( s );
mark s; // put x into C
while (!F.empty()) {
x = F.dequeue();
forall y in adj(x) do
if ( y not marked ) {
F.enqueue(y);
mark y; // put y into C }

33. Breadth-first Search Complexity: Overall cost: O(|E|+|V|)
each node enqued on F only once for a total of O(|V|) operations.
each edge touched at most twice (once for each of its end nodes) for a total of O(|E|) operations.
Overall cost: O(|E|+|V|)

34. Single-Source Shortest Paths

35. Airline routes BOS ORD PVD SFO JFK BWI LAX DFW MIA 2704 867 1846 187
849
PVD
SFO
740
JFK
144
802
1464
337
621
1258
184
BWI
1391
LAX
DFW
1090
1235
946
1121
MIA
2342

36. Single-source shortest path
Suppose we live in Baltimore (BWI) and want the shortest path to San Francisco (SFO).
One way to solve this is to solve the single-source shortest path problem:
Find the shortest path from BWI to every city.

37. Single Source Shortest Path Problem
Given: a directed graph G = (V,E)
weight(u,v)  0 for all edges (u,v)  E
Find: length of the shortest path from start vertex s to every other vertex in G s f a b d e c g 4 2 5 1
Example: shortest path from s to e has length 6 (it’s the path s-b-e)
Question: how to adapt for unweighted graphs?

38. Single-source shortest path
While we may be interested only in BWI-to-SFO, there are no known algorithms that are asymptotically faster than solving the single-source problem for BWI-to-every-city.

39. Many applications
Shortest paths model many useful real-world problems.
Minimization of latency in the Internet.
Minimization of cost in power delivery.
Job and resource scheduling.
Route planning.

40. Dijkstra’s algorithm Initialization a. Set D(s) = 0
(see Weiss, Section 14.3)
Initialization
a. Set D(s) = 0
b. For all vertices v  V, v  s, set D(v) = 
c. Insert all vertices into priority queue P,
using distances as the keys s f a b d e c g 4 2 5 1 s a b c d e f g 

41. Dijkstra’s algorithm While P is not empty:
1. Select the next vertex u to visit
u = P.deleteMin()
2. Update D(w) for each vertex w in adj(u):
If D(u) + weight(u,w) < D(w),
a. D(w) = D(u) + weight(u,w) b. Update the priority queue to
reflect new distance for w

42. Dijkstra’s algorithm s f a b d e c g 4 2 5 1
Visited s a b c d e f g 

43. Dijkstra’s algorithm b c a d e f g 2 4 5  Visited s (D = 0) s f a b d1
Visited
s (D = 0) b c a d e f g 2 4 5 

44. Dijkstra’s algorithm d c a e f g 3 4 5 6  Visited s (D = 0) b (D = 2)1
Visited
s (D = 0)
b (D = 2) d c a e f g 3 4 5 6 

45. Dijkstra’s algorithm c a e f g 4 6  Visited s (D = 0) b (D = 2)5 1
Visited
s (D = 0)
b (D = 2)
d (D = 3) c a e f g 4 6 

46. Dijkstra’s algorithm a e f g 4 6  Visited s (D = 0) b (D = 2)c g 4 2 5 1
Visited
s (D = 0)
b (D = 2)
d (D = 3)
c (D = 4) a e f g 4 6 

47. Dijkstra’s algorithm e f g 6  Visited s (D = 0) b (D = 2) d (D = 3)c g 4 2 5 1
Visited
s (D = 0)
b (D = 2)
d (D = 3)
c (D = 4)
a (D = 4)
... e f g 6 

48. Dijkstra’s algorithm Visited s (D = 0) b (D = 2) d (D = 3) c (D = 4)f a b d e c g 4 2 5 1
Visited
s (D = 0)
b (D = 2)
d (D = 3)
c (D = 4)
a (D = 4)
e (D = 6)
f (D = 6)
g (D = )

49. Features of Dijkstra’s Algorithm
A greedy algorithm
“Visits” every vertex only once, when it becomes the vertex with minimal distance amongst those still in the priority queue
Distances may be revised multiple times: current values represent ‘best guess’ based on our observations so far
Once a vertex is visited we are guaranteed to have found the shortest path to that vertex…. why?

50. Correctness (via contradiction)
Assume u is the first vertex visited such that D(u) is not a shortest path (thus the true shortest path must pass through some unvisited vertex)
Let x represent the first unvisited vertex on the true shortest path to u u x s
visited
unvisited
D(x) must represent a shortest path to x, and D(x)  Dshortest(u).
However, Dijkstra’s always visits the vertex with the smallest distance next, so we can’t possibly visit u before we visit x

51. Performance (using a heap)
Initialization of priority queue: O(|V|)
Visitation loop: |V| calls
deleteMin(): O(log|V|)
Each edge is considered only once during entire execution, for a total of |E| updates of the priority queue, each O(log|V|)
Overall cost:
O(|V|log|V| + |E|log|V|)

52. Unweighted case O(|V| + |E|)
If all weights are 1, no need for priority queue -- an ordinary queue suffices:
newly enqueued nodes have larger distance from already visited nodes
Visitation loop: |V| calls
dequeue(): O(1)
Each edge is considered only once during entire execution, for a total of |E| enqueues, each O(log 1).
Overall cost:
Question: which algorithm is obtained this way?
O(|V| + |E|)

53. Representing shortest paths
We now have an algorithms to compute the length of the shortest path between s and w.
But what if we actually want to find the vertices on the shortest path?

54. Representing shortest paths
We now have an algorithms to compute the length of the shortest path between s and w.
But what if we actually want to find the vertices on the shortest path?
Fact: if s=s0,s1,...,sn=w is the shortest path from s to w, then s=s0, s1,...,sn-1 is the shortest path from s to sn-1.
Idea: With each D(w), remember the previous node P(w) = sn-1 in the shortest path.

55. Dijkstra with predecessors
While P is not empty:
1. Select the next vertex u to visit
u = P.deleteMin()
2. Update D(w) for each vertex w
adjacent to u:
If D(u) + weight(u,w) < D(w),
a. D(w) = D(u) + weight(u,w)
b. P(w) = u
c. Update the priority queue to
reflect new values for D(w) and P(w)

56. Negative weights?
Dijkstra’s greedy algorithm can only guarantee shortest paths for non-negative weights s f a b d e c g 4 2 5
- 3 1 b c a d e f g 2 4 5 
visiting b incorrectly produces a path of distance 2
How can we address this problem?

57. The Bellman-Ford algorithm
(see Weiss, Section 14.4)
Returns a boolean:
TRUE if and only if there is no negative-weight cycle reachable from the source: a simple cycle <v0, v1,…,vk>, where v0=vk and
FALSE otherwise
If returned TRUE, it also produces the shortest paths
Question: why do we avoid negative cycles?

58. Bellman-Ford algorithm
Initialization
a. Set D(s) = 0
b. For all vertices v  V, v  s, set D(v) =  s f a b d e c g 4 2 5
- 3 1
- 2 s a b c d e f g 

59. Bellman-Ford algorithm
Path updates and negative cycle check:
1. Do |V|-1 times:
For each edge (u,v) in |E|,
If D(u) + weight(u,v) < D(v)
D(v) = D(u) + weight(u,v)
2. For each edge (u,v) in |E|:
return false
3. Return true

60. Bellman-Ford path updates
Assume edges are examined in lexicographic order
(i.e., (b,d), (b,e), (c,b), (c,f), (d,a), (f,e), (s,a), (s,b), (s,c) ) s f a b d e c g 4 2 5
- 3 1
- 2
Iteration 1: s a b c d e f g 5 2 4 

61. Bellman-Ford path updates
Assume edges are examined in lexicographic order
(i.e., (b,d), (b,e), (c,b), (c,f), (d,a), (f,e), (s,a), (s,b), (s,c) ) s f a b d e c g 4 2 5
- 3 1
- 2
Iteration 2: s a b c d e f g 5 2 4 3 

62. Bellman-Ford path updates
Assume edges are examined in lexicographic order
(i.e., (b,d), (b,e), (c,b), (c,f), (d,a), (f,e), (s,a), (s,b), (s,c) ) s f a b d e c g 4 2 5
- 3 1
- 2
Iteration 2: s a b c d e f g 5 2 4 3 6 

63. Bellman-Ford path updates
Assume edges are examined in lexicographic order
(i.e., (b,d), (b,e), (c,b), (c,f), (d,a), (f,e), (s,a), (s,b), (s,c) ) s f a b d e c g 4 2 5
- 3 1
- 2
Iteration 2: s a b c d e f g 5 1 4 3 6 

64. Bellman-Ford path updates
Assume edges are examined in lexicographic order
(i.e., (b,d), (b,e), (c,b), (c,f), (d,a), (f,e), (s,a), (s,b), (s,c) ) s f a b d e c g 4 2 5
- 3 1
- 2
Iteration 2: s a b c d e f g 5 1 4 3 6 

65. Bellman-Ford path updates
Assume edges are examined in lexicographic order
(i.e., (b,d), (b,e), (c,b), (c,f), (d,a), (f,e), (s,a), (s,b), (s,c) ) s f a b d e c g 4 2 5
- 3 1
- 2
Iteration 2: s a b c d e f g 4 1 3 6 

66. Bellman-Ford path updates
Assume edges are examined in lexicographic order
(i.e., (b,d), (b,e), (c,b), (c,f), (d,a), (f,e), (s,a), (s,b), (s,c) ) s f a b d e c g 4 2 5
- 3 1
- 2
Iteration 2: s a b c d e f g 4 1 3 6 
etcetera...

67. Bellman-Ford cycle check
After Iteration 7: s f a b d e c g 4 2 5
- 3 1
- 2 s a b c d e f g 3 1 4 2 6 
Performs one final iteration for all edges
If any weights change at this point, a negative cycle exists.
For this graph, the algorithm returns TRUE.

68. Key features
If the graph contains no negative-weight cycles reachable from the source vertex, after |V| - 1 iterations all distance estimates represent shortest paths…why?
We assumed edges were considered in the same order for each iteration. Would the algorithm still work if we changed the order for every iteration?

69. Correctness
Case 1: Graph G=(V,E) doesn’t contain any negative-weight cycles reachable from the source vertex s
Consider a shortest path p = < s, v1,..., vk>, which must have k  |V| - 1 edges
To show: after k iterations, D(vk)= length(p).
Thus, after |V|-1 passes, the algorithm correctly computes the length of all shortest paths.
The algorithm returns TRUE, because in the cycle-check phase, no distance changes.

70. Correctness By induction on k: For k = 0:
D(s) = 0 after initialization
For k = i > 0:
Assume D(vi-1) is a shortest path after iteration (i-1)
At the i-th iteration:
- D(vi-1) is not changed
D(vi) = D(vi-1) + weight(vi-1,vi)
Thus, D(vi) reflects the shortest path to vi

71. Correctness
Case 2: Graph G=(V,E) contains a negative-weight cycle < v0, v1,..., vk =v0> reachable from the source vertex s
Proof by contradiction:
Assume the algorithm returns TRUE
Thus, D(vi-1) + weight(vi-1, vi)  D(vi) for i = 1,…,k
Summing the inequalities for the cycle:
leads to a contradiction since the first sums on each side are equal (each vertex appears exactly once since vk=v0) and the sum of weights must be less than 0.

72. Performance Initialization: O(|V|) Path update and cycle check:
|V| calls checking |E| edges, O(|VE|)
Overall cost:
O(|VE|)