1. CS 15-121 Introduction to Data Structures
Lecture 13: Graphs II
Lecture 13: Graphs II

2. Topics Two Greedy Algorithms: Shortest Path
Dijkstra’s Algorithm (Main, Ch. 14)
Minimum Spanning Trees
Prim’s Algorithm (CLR, Ch. 24)
A classic Dynamic Programming Algorithm:
Floyd Warshall (CLR, Ch. 25)
Using Prim to approximate TSP
Lecture 13: Graphs II

3. Graphs Revisited Nodes and links between them
May be linked in any pattern (unlike trees)
Vertex: a node in the graph
Edge: a connection between nodes
Lecture 13: Graphs II

4. Undirected Graphs Vertices drawn with circles Edges drawn with lines
Vertices are labelled
Edges are labelled e3 e2 e0 e1 e4 e5
Visual Layout Doesn’t Matter!
Lecture 13: Graphs II

5. Undirected Graphs
An undirected graph is a finite set of vertices along with a finite set of edges. The empty graph is an empty set of vertices and an empty set of edges.
Each edge connects two vertices
The order of the connection is unimportant
Lecture 13: Graphs II

6. Directed Graphs A directed graph: finite set of vertices and edges
Each edge connects two vertices, called the source and target; the edge connects the source to the target (order is significant)
Simple Graphs: have no loops and no multiple edges
Lecture 13: Graphs II

7. Directed Graphs Arrows are used to represent the
edges; arrows start at the source
vertex and point to the target vertex
Useful for modelling directional
phenomena, like geographical travel,
or game-playing where moves are
irreversible
Examples: modelling states in a chess
game, or Tic-Tac-Toe
Lecture 13: Graphs II

8. Finding if a Path Exists
When problems are represented by graphs, a solution is often some path from a start node to a goal node
Example: modeling network connectivity between computers
As we’ve seen, DFS and BFS can be used to find path(a,b)
Lecture 13: Graphs II

9. Minimal Paths
Weighted Graphs: each edge has an associated cost or weight V0 V1 V2 V3 1 6 3 2 7
Several paths from V0 to V2
One path with lowest total cost: {(V0,V3) (V3,V1) (V1,V2)} = = 6
Paths with fewer edges exist, but with higher cost!
Lecture 13: Graphs II

10. Finding a Minimal Path
The shortest (minimal) path has the lowest cost, not the fewest edges. BFS will find the path with fewest edges.
Examples: Minimizing travel cost between A & B; minimizing wiring length between two points
Lecture 13: Graphs II

11. Definitions
A weighted edge is an edge with a non-negative integer weight
The weight of a path is the total sum of the weights of its edges
If two vertices are connected by at least one path, then the shortest path is the path with the smallest weight (can be more than one)
Lecture 13: Graphs II

12. Dijkstra’s Algorithm
Given a weighted graph, find the shortest path between two vertices
Start by finding the weight of the shortest path (shortest distance)
Dijkstra’s Algorithm finds the shortest distance from the start node to every other node in the graph
Lecture 13: Graphs II

13. Dijkstra’s Algorithm
Use an integer array called distance with one component for each vertex in the graph
Goal: Completely fill the distance array so that for each vertex v, the value of distance[v] is the weight of the shortest path from start to v (this is also the loop invariant)
Lecture 13: Graphs II

14. Dijkstra’s Algorithm
Step 1: Fill in distance with ? (infinity) at every location, with the exception of distance[start] = 0
Step 2: Initialize a set of vertices called allowedVertices to the empty set; a permitted path starts at start and contains only vertices in allowedVertices
Lecture 13: Graphs II

15. Dijkstra’s Algorithm
Step 3: Loop, adding one more vertex to allowedVertices and updating the distance array
3a: Let next be the closest vertex to the start vertex that is not yet in allowedVertices
3b: Add the next to allowedVertices
3c: Update distance array to maintain the invariant
Lecture 13: Graphs II

16. Example V0 V5 V3 V1 2 9 6 15 V2 V4 8 7 3 1 start distance [0] [1] [2]
[3]
[4]
[5] ?
next
[0]
[1]
[2]
[3]
[4]
[5] 2 ? 9
next
[0]
[1]
[2]
[3]
[4]
[5] 2 10 17 ? 8
next
Lecture 13: Graphs II

17. Example V0 V5 V3 V1 2 9 6 15 V2 V4 8 7 3 1 start [0] [1] [2] [3] [4]
[5] 2 10 17 11 8
next
[0]
[1]
[2]
[3]
[4]
[5] 2 10 11 8
next
[0]
[1]
[2]
[3]
[4]
[5] 2 10 11 8
Lecture 13: Graphs II

18. Shortest Path
We’ve computed the weight of the shortest path from start to every other node
What about the shortest path itself? (sequence of nodes from start to some v)
Need to maintain predecessor information for each node
Lecture 13: Graphs II

19. Predecessor Information
For each vertex v, keep track of which vertex was the next vertex when distance[v] was given a new (smaller) value
Use an array called predecessor: for each vertex v, predecessor[v] is the value of next when distance[v] was last updated
Lecture 13: Graphs II

20. Example w/PredecessorV0 V5 V3 V1 2 9 6 15 V2 V4 8 7 3 1
start
distance
predecessor
[0]
[1]
[2]
[3]
[4]
[5] ?
next
[0]
[1]
[2]
[3]
[4]
[5] 2 ? 9
next
[0]
[1]
[2]
[3]
[4]
[5] 2 10 17 ? 8
next 1 1 1
Lecture 13: Graphs II

21. Example w/PredecessorV0 V5 V3 V1 2 9 6 15 V2 V4 8 7 3 1
start
[0]
[1]
[2]
[3]
[4]
[5] 2 10 17 11 8
next 1 1 5 1
[0]
[1]
[2]
[3]
[4]
[5] 2 10 11 8
next 1 2 5 1
[0]
[1]
[2]
[3]
[4]
[5] 2 10 11 8 1 2 5 1
Lecture 13: Graphs II

22. Printing the Shortest Path
// Printing the vertices on the shortest path
// from start to v.
// Print in reverse order from v to start.
vertexOnPath = v;
System.out.println(vertexOnPath);
while (vertexOnPath != start) {
vertexOnPath = predecessor[vertexOnPath]; }
Lecture 13: Graphs II

23. A Digression Write a routine that computes the n’th Fibonacci number.
Method 1:
int fib(int n) {
//pre: n >= from Wikipedia introduced Hindu-Arabic numerals to Europe
if (n == 1 || n ==2) return 1;
else return fib(n-1) + fib(n-2); }
Lecture 13: Graphs II

24. A Digression Method 2: //pre: n >= 1 if(n == 1 || n == 2) return 1;
int fib(int n) {
//pre: n >= 1
if(n == 1 || n == 2) return 1;
else {
first = 1; second = 1;
k = 3;
while (k <= n) {
sum = first + second;
first = second;
second = sum;
k = k + 1; }
return sum;
Which is better, method 1 or method 2?
Is recursion always bad like method 1?
Dynamic programming solutions
try to avoid redundant computations.
Lecture 13: Graphs II

25. Floyd Warshall All Pairs
for each vertex u in G do for each vertex v in G do cost[u,v] = c[u,v] for each vertex w in G do cost[u,v] = min(cost[u,v], cost[u,w] + cost[w,v]
From
wikipedia
Compute the shortest
path from each vertex to
every other vertex.
This is a classic
dynamic programming
algorithm. “programming”
refers to the fact that it’s
a tabular method.
Lecture 13: Graphs II

26. Graph G 2 3 2 6 4 1 3 1 1 4
Lecture 13: Graphs II

27. Initial Cost matrix 1 2 3 4
inf 6
Lecture 13: Graphs II

28. path from u to 1 plus the cost of the path from 1 to v?
After w = 1 1 2 3 4
inf 6
Which is cheaper, the current cost of the path from u to v or the cost of the
path from u to 1 plus the cost of the path from 1 to v?
Lecture 13: Graphs II

29. path from u to 2 plus the cost of the path from 2 to v?
After w = 2 1 2 3 4 10 6 5
inf
Which is cheaper, the current cost of the path from u to v or the cost of the
path from u to 2 plus the cost of the path from 2 to v?
The cost of the current path may already involve 1.
The cost of the paths from u to 2 and from 2 to v may already include
the vertex 1 from the previous iteration.
Lecture 13: Graphs II

30. path from u to 3 plus the cost of the path from 3 to v?
After w = 3 1 2 3 4 10 6 5
Which is cheaper, the cost of the current path from u to v or the cost of the
path from u to 3 plus the cost of the path from 3 to v?
The cost of the current path may already include 1 and 2.
The cost of the paths from u to 3 and from 3 to v may already include
the vertex 1 and 2 from previous iterations.
Lecture 13: Graphs II

31. path from u to 4 plus the cost of the path from 4 to v?
After w = 4 1 2 3 4 5 6
Which is cheaper, the cost of the current path from u to v or the cost of the
path from u to 4 plus the cost of the path from 4 to v?
The cost of the current path may already include 1 and 2 and 3.
The cost of the paths from u to 4 and from 4 to v may already include
the vertices 1, 2 and 3 from previous iterations.
Quiz:
What’s the
runtime
complexity
of Floyd
Warshall?
Lecture 13: Graphs II

32. Why does Floyd Warshall Work? (From CLR)
Theorem: Subpaths of shortest paths are shortest paths.
Let dijk be the weight of the shortest path from vertex i to vertex j for which all intermediate vertices are in the set {1,2,…,k}.
When k = 0, a path from vertex i to vertex j with no intermediate vertex numbered higher than 0 has no intermediate vertices at all. Such a path has at most one edge and hence dij0= wij = the weight of the edge from i to j.
dijk = wij if k = 0.
= min (dij(k-1) , dik(k-1) + dkj(k-1)) if k >= 1.
Lecture 13: Graphs II

33. Minimum Spanning Trees
Suppose we want to find the lowest cost solution to connect n vertices with (n - 1) edges
Example: Wiring pins of electrical components
Solution: Find a minimum spanning tree (MST)
Lecture 13: Graphs II

34. Minimum Spanning Tree Assume:
A connected, undirected graph, G = (V, E)
V is the set of vertices
E is the set of edges
for each edge (u, v) in E, we have a weight w(u, v) specifying the cost to connect u and v
Lecture 13: Graphs II

35. Minimum Spanning Tree Goal:
Find an acyclic subset of the edge list, T, which connects all of the vertices;
The total cost (the sum of w(u, v) for edges in T) is minimized.
Lecture 13: Graphs II

36. Example a b c d e f g i h 7 4 8 11 6 2 1 14 10 9 Total weight: 37
Unique?...
No. Replacing (b,c) with
(a,h) yields another ST
with weight = 37.
Lecture 13: Graphs II

37. Generic MST
Maintain a set of edges A which is always a subset of some minimal spanning tree
At each step, an edge (u,v) is found which can be added to A without violating this invariant; A U {(u,v)} is also a subset of a MST. We call (u,v) a safe edge.
Lecture 13: Graphs II

38. Generic MST GenericMST(G,w) A = null
while A does not form a spanning tree
do: find an edge (u,v) that is safe for A
A = A U {(u,v)}
return A
To implement MST, we need two things:
a rule for recognizing safe edges;
an algorithm that uses the rule to find safe edges.
Lecture 13: Graphs II

39. Definitions
A cut (S, V-S) of an undirected graph G = (V,E) is a partition of V 8 7 S b c d 4 9 2 11 14 a i 4 e 7 6 8 10 h g f 1 2
V-S A:
Lecture 13: Graphs II

40. Definitions
An edge (u,v) crosses the cut (S,V-S) if one of its endpoints is in S and the other is in V-S.
A cut respects the set A of edges if no edge in A crosses the cut
A crossing edge is a light edge if its weight is the minimum of any edge crossing the cut
Lecture 13: Graphs II

41. Theorem
Let G = (V,E) be a connected, undirected graph with weight function w. Let A be a subset of E that is included in some minimum spanning tree for G; let (S, V-S) be any cut of G that respects A; and let (u,v) be a light edge crossing (S, V-S). Then (u,v) is safe for A.
Lecture 13: Graphs II

42. Prim’s Algorithm Greed sometimes pays off.
Edges in A always form a single tree
Start from an arbitrary root vertex r and grow until the tree spans V
At each step, a light edge connecting A to V-A is added
Store nodes in V-A in a priority queue based on a key
Lecture 13: Graphs II

43. Prim’s Algorithm
For each vertex v, key[v] is the minimum weight of any edge connecting v to a vertex in the tree we are building
The field pi[v] names the “parent” of v in the tree we are building
A = {(v,pi[v]): v in V - {r} - Q}
Lecture 13: Graphs II

44. Prim’s Algorithm MST-Prim(G,w,r) Q = V[G] foreach u in Q
do: key[u] = ? // initialize to ‘infinity’
key[r] = 0
pi[r] = null
while Q is not empty
do: u = ExtractMin(Q)
// find light edge; u = r first time through
foreach v in Adj[u]
do: if v in Q && w(u,v) < key[v]
// update adjacent nodes
then pi[v] = u
key[v] = w(u,v)
Lecture 13: Graphs II

45. a b c d e f g i h 7 4 8 11 6 2 1 14 10 9 weight parent a b c d e f g h
nil 4 8 7 10 2 a b c d e f g h i
nil ?
nil 4 ? 8 a
nil 4 8 7 10 2 1 g a b c f
nil 4 8 ? a b
nil 4 8 7 10 2 1 g a h b c f
nil 4 8 7 9 2 1 g a d b c f
nil 4 8 7 ? 2 a c b
nil 4 8 7 9 2 1 g a e b c f d
nil 4 8 7 ? 6 2 i a b c
Lecture 13: Graphs II

46. Using Prim to Approximate TSP
Suppose we have a complete graph satisfying the triangle inequality.
Given a minimal tour H*, chop off an edge to form a spanning tree ST.
c(ST) <= c(H*)
c(MST) <= c(ST) <= c(H*)
Walk the MST using each edge twice. c(walk) = 2c(MST) <= 2c(H*).
Use the triangle inequality to make the walk a tour H. This is a preorder traversal.
c(H) <= c(walk) <= 2c(H*)
So, compute MST using Prim, and create a tour. You are within twice the optimal tour.
Lecture 13: Graphs II