CS6045: Advanced Algorithms Graph Algorithms
Graph Representation A graph G = (V, E) V = set of vertices E = set of edges = subset of V V Thus |E| = O(|V|2)
Graph Variations Variations: A connected graph has a path from every vertex to every other In an undirected graph: Edge (u,v) = edge (v,u) No self-loops In a directed graph: Edge (u,v) goes from vertex u to vertex v, notated uv
Graph Variations More variations: A weighted graph associates weights with either the edges or the vertices e.g., a road map: edges might be weighted with distance A hypergraph allows multiple edges between the same vertices e.g., the call graph in a program (a function can get called from multiple points in another function)
Graphs We will typically express running times in terms of |E| and |V| If |E| |V|2 the graph is dense If |E| |V| the graph is sparse If you know you are dealing with dense or sparse graphs, different data structures may make sense
Representing Graphs Assume V = {1, 2, …, n} An adjacency matrix represents the graph as a n x n matrix A: A[i, j] = 1, if edge (i, j) E = 0, if edge (i, j) E A[i, j] = weight on the edge, if edge (i, j) E = 0, if edge (i, j) E
Graphs: Adjacency Matrix Example: A 1 2 3 4 ?? 1 a 2 d 4 b c 3
Graphs: Adjacency Matrix Example: A 1 2 3 4 1 a 2 d 4 b c 3 How much storage does the adjacency matrix require? A: O(V2)
Graphs: Adjacency Matrix The adjacency matrix is a dense representation Usually too much storage for large graphs But can be very efficient for small graphs Most large interesting graphs are sparse e.g., planar graphs, in which no edges cross, have |E| = O(|V|) by Euler’s formula For this reason the adjacency list is often a more appropriate representation
Graphs: Adjacency List Adjacency list: for each vertex v V, store a list of vertices adjacent to v Example: For an undirected graph
Graphs: Adjacency List Adjacency list: for each vertex v V, store a list of vertices adjacent to v Example: For a directed graph
Graphs: Adjacency List How much storage is required? The degree of a vertex v = # of edges Directed graphs have in-degree, out-degree For directed graphs, # of items in adjacency lists is out-degree(v) = |E| takes O(V + E) storage For undirected graphs, # items in adjacency lists is degree(v) = 2 |E| (handshaking lemma) also O(V + E) storage So: Adjacency lists take O(V+E) storage
Graph Searching Given: a graph G = (V, E), directed or undirected Goal: methodically explore every vertex and every edge Ultimately: build a tree on the graph Pick a vertex as the root Choose certain edges to produce a tree Note: might also build a forest if graph is not connected
Breadth-First Search “Explore” a graph, turning it into a tree One vertex at a time Expand frontier of explored vertices across the breadth of the frontier Builds a tree over the graph Pick a source vertex to be the root Find (“discover”) its children, then their children, etc.
Breadth-First Search Again will associate vertex “colors” to guide the algorithm White vertices have not been discovered All vertices start out white Grey vertices are discovered but not fully explored They may be adjacent to white vertices Black vertices are discovered and fully explored They are adjacent only to black and gray vertices Explore vertices by scanning adjacency list of grey vertices
Breadth-First Search What does v->d represent? BFS(G, s) { initialize vertices V.d = ; s.d = 0; Q = {s}; // Q is a queue; initialize to s while (Q not empty) { u = Dequeue(Q); for each v u->adj { if (v->color == WHITE) v->color = GREY; v->d = u->d + 1; v->p = u; Enqueue(Q, v); } u->color = BLACK; What does v->d represent? What does v->p represent?
Breadth-First Search: Example u v w x y
Breadth-First Search: Example u v w x y Q: s
Breadth-First Search: Example u 1 1 v w x y Q: w r
Breadth-First Search: Example u 1 2 1 2 v w x y Q: r t x
Breadth-First Search: Example u 1 2 2 1 2 v w x y Q: t x v
Breadth-First Search: Example u 1 2 3 2 1 2 v w x y Q: x v u
Breadth-First Search: Example u 1 2 3 2 1 2 3 v w x y Q: v u y
Breadth-First Search: Example u 1 2 3 2 1 2 3 v w x y Q: u y
Breadth-First Search: Example u 1 2 3 2 1 2 3 v w x y Q: y
Breadth-First Search: Example u 1 2 3 2 1 2 3 v w x y Q: Ø
BFS: The Code Again Touch every vertex: O(V) BFS(G, s) { initialize vertices V.d = ; s.d = 0; Q = {s}; while (Q not empty) { u = Dequeue(Q); for each v u->adj { if (v->color == WHITE) v->color = GREY; v->d = u->d + 1; v->p = u; Enqueue(Q, v); } u->color = BLACK; u = every vertex, but only once v = every vertex that appears in some other vert’s adjacency list What will be the running time? Total running time: O(V+E)
BFS: The Code Again What will be the storage cost BFS(G, s) { initialize vertices V.d = ; s.d = 0; Q = {s}; while (Q not empty) { u = Dequeue(Q); for each v u->adj { if (v->color == WHITE) v->color = GREY; v->d = u->d + 1; v->p = u; Enqueue(Q, v); } u->color = BLACK; What will be the storage cost in addition to storing the tree? Total space used: O(max(degree(v))) = O(E)
Breadth-First Search: Properties BFS calculates the shortest-path distance to the source node Shortest-path distance (s,v) = minimum number of edges from s to v, or if v not reachable from s BFS builds breadth-first tree, in which paths to root represent shortest paths in G Thus can use BFS to calculate shortest path from one vertex to another in O(V+E) time
Depth-First Search Depth-first search is another strategy for exploring a graph Explore “deeper” in the graph whenever possible Edges are explored out of the most recently discovered vertex v that still has unexplored edges When all of v’s edges have been explored, backtrack to the vertex from which v was discovered
Depth-First Search Vertices initially colored white Then colored gray when discovered Then black when finished
Depth-First Search: The Code DFS(G) { for each vertex u G->V u->color = WHITE; } time = 0; if (u->color == WHITE) DFS_Visit(u); DFS_Visit(u) { u->color = GREY; time = time+1; u->d = time; for each v u->Adj[] if (v->color == WHITE) DFS_Visit(v); } u->color = BLACK; u->f = time; What does u->d represent? What does u->f represent? Will all vertices eventually be colored black? What will be the running time?
Depth-First Search: The Code DFS(G) { for each vertex u G->V u->color = WHITE; } time = 0; if (u->color == WHITE) DFS_Visit(u); DFS_Visit(u) { u->color = GREY; time = time+1; u->d = time; for each v u->Adj[] if (v->color == WHITE) DFS_Visit(v); } u->color = BLACK; u->f = time; Running time: O(n2) because call DFS_Visit on each vertex, and the loop over Adj[] can run as many as |V| times
Depth-First Search: The Code DFS(G) { for each vertex u G->V u->color = WHITE; } time = 0; if (u->color == WHITE) DFS_Visit(u); DFS_Visit(u) { u->color = GREY; time = time+1; u->d = time; for each v u->Adj[] if (v->color == WHITE) DFS_Visit(v); } u->color = BLACK; u->f = time; BUT, there is actually a tighter bound. Running time of DFS = O(V+E)
DFS Example source vertex 1 2 3 6 7 8 4 5
DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1| | | | | | | |
DFS Example source vertex d | f 1 2 3 1| | | 4 5 2| | | | | 6 7 8
DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1| | | 2| | 3| | |
DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1| | | 2| | 3|4 | |
DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1| | | 2| | 3|4 5| |
DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1| | | 2| | 3|4 5|6 |
DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1| | | 2|7 | 3|4 5|6 |
DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1| 8| | 2|7 | 3|4 5|6
What do the grey vertices represent? DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1| 8| | 2|7 9| 3|4 5|6 | What do the grey vertices represent?
DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1| 8| | 2|7 9|10 3|4 5|6 |
DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1| 8|11 | 2|7 9|10 3|4 5|6 |
DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1|12 8|11 | 2|7 9|10 3|4 5|6 |
DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1|12 8|11 13| 2|7 9|10 3|4 5|6 |
DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1|12 8|11 13| 2|7 9|10 3|4 5|6 14|
DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1|12 8|11 13| 2|7 9|10 3|4 5|6 14|15
DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1|12 8|11 13|16 2|7 9|10 3|4 5|6 14|15
DFS Properties Parenthesis Theorem Corollary For all u, v, exactly one of the following holds: The intervals [u.d, u.f] and [v.d, v.f] are disjoint, neither of u and v is a descendant of the other The interval [u.d, u.f] is contained entirely within the interval [v.d, v.f], u is a descendant of v in a depth-first tree The interval [u.d, u.f] is contained entirely within the interval [v.d, v.f], v is a ancestor of u in a depth-first tree Corollary v is a proper descendant of u if and only if u.d<v.d<v.f<u.f
DFS: Kinds of edges DFS introduces an important distinction among edges in the original graph: Tree edge: encounter new (white) vertex The tree edges form a spanning forest Can tree edges form cycles? Why or why not?
DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1|12 8|11 13|16 2|7 9|10 3|4 5|6 14|15 Tree edges
DFS: Kinds of edges DFS introduces an important distinction among edges in the original graph: Tree edge: encounter new (white) vertex Back edge: from descendent to ancestor Encounter a grey vertex (grey to grey)
DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1|12 8|11 13|16 2|7 9|10 3|4 5|6 14|15 Tree edges Back edges
DFS: Kinds of edges DFS introduces an important distinction among edges in the original graph: Tree edge: encounter new (white) vertex Back edge: from descendent to ancestor Forward edge: from ancestor to descendent Not a tree edge From grey node to black node
DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1|12 8|11 13|16 2|7 9|10 3|4 5|6 14|15 Tree edges Back edges Forward edges
DFS: Kinds of edges DFS introduces an important distinction among edges in the original graph: Tree edge: encounter new (white) vertex Back edge: from descendent to ancestor Forward edge: from ancestor to descendent Cross edge: between a tree or subtrees Any other edges From a grey node to a black node
DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1|12 8|11 13|16 2|7 9|10 3|4 5|6 14|15 Tree edges Back edges Forward edges Cross edges
DFS: Kinds of edges DFS introduces an important distinction among edges in the original graph: Tree edge: encounter new (white) vertex Back edge: from descendent to ancestor Forward edge: from ancestor to descendent Cross edge: between a tree or subtrees Note: If G is undirected, a DFS produces only tree and back edges. No forward or cross edges.
DFS And Graph Cycles Theorem: An undirected graph is acyclic iff a DFS yields no back edges If acyclic, no back edges (because a back edge implies a cycle If no back edges, acyclic No back edges implies only tree edges Only tree edges implies we have a tree or a forest Which by definition is acyclic Thus, can run DFS to find whether a graph has a cycle
Directed Acyclic Graphs A directed acyclic graph or DAG is a directed graph with no directed cycles:
Topological Sort Topological sort of a DAG: Linear ordering of all vertices in graph G such that vertex u appears somewhere before vertex v if edge (u, v) G Real-world example: getting dressed
Getting Dressed Underwear Socks Watch Pants Shoes Shirt Belt Tie Jacket
Getting Dressed Underwear Socks Watch Pants Shoes Shirt Belt Tie Jacket Socks Underwear Pants Shoes Watch Shirt Belt Tie Jacket
Topological Sort Algorithm { Run DFS to compute finish time v.f for all v V Output vertices in order of decreasing finishing times } Time: O(V+E)
Getting Dressed 11/16 Underwear Socks 17/18 Watch 9/10 12/15 Pants Shoes 13/14 Shirt 1/8 6/7 Belt Tie 2/5 Jacket 3/4 Socks Underwear Pants Shoes Watch Shirt Belt Tie Jacket 17/18 11/16 12/15 13/14 9/10 1/8 6/7 2/5 3/4
Minimum Spanning Tree Problem: given a connected, undirected, weighted graph, find a spanning tree (T) using edges that minimize the total weight, i.e., minimize (𝑢,𝑣)∈𝑇 𝑤(𝑢,𝑣)
Minimum Spanning Tree Which edges form the minimum spanning tree (MST) of the below graph? A 6 4 5 9 H B C 14 2 10 15 G E D 3 8 F
Minimum Spanning Tree Answer: A 6 4 5 9 H B C 14 2 10 15 G E D 3 8 F
Minimum Spanning Tree MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees Let T be an MST of G with an edge (u,v) in the middle Removing (u,v) partitions T into two trees T1 and T2 Claim: T1 is an MST of G1 = (V1,E1), and T2 is an MST of G2 = (V2,E2) Proof: w(T) = w(u,v) + w(T1) + w(T2)
Minimum Spanning Tree MSTs satisfy the greedy-choice property: Let T be MST of G, and let A T be subtree of T Let (u,v) be min-weight edge connecting A to the rest of the Graph (V - A) Then (u,v) T
Prim’s Algorithm Run on example graph MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 9 5 14 2 10 15 3 8 Run on example graph
Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 9 5 14 2 10 15 3 8
Prim’s Algorithm r Pick a start vertex r MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 9 5 14 2 10 15 r 3 8 Pick a start vertex r
Red vertices have been removed from Q Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 9 5 14 2 10 15 u 3 8 Red vertices have been removed from Q
Red arrows indicate parent pointers Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 9 5 14 2 10 15 u 3 8 3 Red arrows indicate parent pointers
Prim’s Algorithm 14 u 3 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 9 5 14 14 2 10 15 u 3 8 3
Prim’s Algorithm 14 3 u MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 9 5 14 14 2 10 15 3 8 3 u
Prim’s Algorithm 14 8 3 u MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 9 5 14 14 2 10 15 8 3 8 3 u
Prim’s Algorithm 10 8 3 u MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 9 5 10 14 2 10 15 8 3 8 3 u
Prim’s Algorithm 10 8 3 u MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 9 5 10 14 2 10 15 8 3 8 3 u
Prim’s Algorithm 10 2 8 3 u MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 9 5 10 2 14 2 10 15 8 3 8 3 u
Prim’s Algorithm 10 2 8 15 3 u MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 9 5 10 2 14 2 10 15 8 15 3 8 3 u
Prim’s Algorithm u 10 2 8 15 3 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 u 9 5 10 2 14 2 10 15 8 15 3 8 3
Prim’s Algorithm u 10 2 9 8 15 3 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 u 9 5 10 2 9 14 2 10 15 8 15 3 8 3
Prim’s Algorithm 4 u 10 2 9 8 15 3 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 4 6 4 u 9 5 10 2 9 14 2 10 15 8 15 3 8 3
Prim’s Algorithm 4 u 5 2 9 8 15 3 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 4 6 4 u 9 5 5 2 9 14 2 10 15 8 15 3 8 3
Prim’s Algorithm u 4 5 2 9 8 15 3 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 4 6 4 9 5 5 2 9 14 2 10 15 8 15 3 8 3
Prim’s Algorithm 4 u 5 2 9 8 15 3 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 4 6 4 9 5 u 5 2 9 14 2 10 15 8 15 3 8 3
Prim’s Algorithm u 4 5 2 9 8 15 3 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 4 6 4 9 5 5 2 9 14 2 10 15 8 15 3 8 3
Prim’s Algorithm 4 5 2 9 8 15 3 u MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 4 6 4 9 5 5 2 9 14 2 10 15 8 15 3 8 3 u
Prim’s Algorithm What is the hidden cost in this code? MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); What is the hidden cost in this code? A: DECREASE-KEY operation on the min-heap What will be the running time? A: Depends on queue binary heap: O(E lg V)
Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1?
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1
Kruskal’s Algorithm 2? 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5? 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8? 25 5 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9? 14 17 8 25 5 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13? 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14? 17 8 25 5 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17? 8 25 5 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1
Kruskal’s Algorithm 2 19? Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21? 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25? 5 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1
Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1
Correctness Of Kruskal’s Algorithm Sketch of a proof that this algorithm produces an MST for T: Assume algorithm is wrong: result is not an MST Then algorithm adds a wrong edge at some point If it adds a wrong edge, there must be a lower weight edge But algorithm chooses lowest weight edge at each step. Contradiction
Kruskal’s Algorithm What will affect the running time? Kruskal() Sort O(V) MakeSet() calls O(E) FindSet() calls O(V) Union() calls Kruskal() { T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); }
Kruskal’s Algorithm: Running Time To summarize: Sort edges: O(E lg E) O(V) MakeSet()’s O(E) FindSet()’s O(V) Union()’s Overall O(E lg E)
Single-Source Shortest Path Problem: given a weighted directed graph G, find the minimum-weight path from a given source vertex s to another vertex v “Shortest-path” = minimum weight Weight of path is sum of edges Might not be unique
Shortest Path Properties Again, we have optimal substructure: the shortest path consists of shortest subpaths: Proof: suppose some subpath is not a shortest path There must then exist a shorter subpath Could substitute the shorter subpath for a shorter path But then overall path is not shortest path. Contradiction
Shortest Path Properties Define (u,v) to be the weight of the shortest path from u to v Shortest paths satisfy the triangle inequality: (u,v) (u,x) + (x,v) x u v This path is no longer than any other path
Shortest Path Properties In graphs with negative weight cycles, some shortest paths will not exist Shortest path can’t contain cycles < 0
Relaxation A key technique in shortest path algorithms is relaxation Idea: for all v, maintain upper bound d[v] on (s,v) Relax(u,v,w) { if (d[v] > d[u]+w) then d[v]=d[u]+w; } 9 5 2 7 Relax 6 5 2 Relax
Bellman-Ford Algorithm for each v V d[v] = ; d[s] = 0; for i=1 to |V|-1 for each edge (u,v) E Relax(u,v, w(u,v)); if (d[v] > d[u] + w(u,v)) return “no solution”; Relax(u,v,w): if (d[v] > d[u]+) then d[v]=d[u]+w Initialize d[], which will converge to shortest-path value Relaxation: Make |V|-1 passes, relaxing each edge Test for solution: have we converged yet? i.e., negative cycle?
Bellman-Ford Algorithm for each v V d[v] = ; d[s] = 0; for i=1 to |V|-1 for each edge (u,v) E Relax(u,v, w(u,v)); if (d[v] > d[u] + w(u,v)) return “no solution”; What will be the running time? A: O(VE)
Bellman-Ford Algorithm for each v V d[v] = ; d[s] = 0; for i=1 to |V|-1 for each edge (u,v) E Relax(u,v, w(u,v)); if (d[v] > d[u] + w(u,v)) return “no solution”; Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w B s -1 2 A E 2 3 1 -3 4 5 C D
Bellman-Ford Algorithm for each v V d[v] = ; d[s] = 0; for i=1 to |V|-1 for each edge (u,v) E Relax(u,v, w(u,v)); if (d[v] > d[u] + w(u,v)) return “no solution”; Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w B -1 s -1 2 A E 2 3 1 -3 4 4 5 C D
Bellman-Ford Algorithm for each v V d[v] = ; d[s] = 0; for i=1 to |V|-1 for each edge (u,v) E Relax(u,v, w(u,v)); if (d[v] > d[u] + w(u,v)) return “no solution”; Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w B -1 s -1 2 A E 1 2 3 1 -3 4 2 1 5 C D
Bellman-Ford Algorithm for each v V d[v] = ; d[s] = 0; for i=1 to |V|-1 for each edge (u,v) E Relax(u,v, w(u,v)); if (d[v] > d[u] + w(u,v)) return “no solution”; Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w B -1 s -1 2 A E 1 2 3 1 -3 4 2 -2 5 C D
Bellman-Ford Algorithm for each v V d[v] = ; d[s] = 0; for i=1 to |V|-1 for each edge (u,v) E Relax(u,v, w(u,v)); if (d[v] > d[u] + w(u,v)) return “no solution”; Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w B -1 s -1 2 A E 1 2 3 1 -3 4 2 -2 5 C D
Bellman-Ford Algorithm for each v V d[v] = ; d[s] = 0; for i=1 to |V|-1 for each edge (u,v) E Relax(u,v, w(u,v)); if (d[v] > d[u] + w(u,v)) return “no solution”; Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w B -1 s -1 2 A E 1 2 3 1 -3 4 2 -2 5 C D
Bellman-Ford Algorithm Note that order in which edges are processed affects how quickly it converges Correctness: show d[v] = (s,v) after |V|-1 passes
DAG Shortest Paths Problem: finding shortest paths in DAG Bellman-Ford takes O(VE) time. How can we do better? Idea: use topological sort DAG-Shortest-Paths(G,w,s) topologically sort the vertices for each v V d[v] = ; d[s] = 0; for each vertex u, taken in topologically sorted order for each vertex v Adj[u] Relax(u,v,w)
DAG Shortest Paths A B E C D 2 7 6 -1 -2 1 6 2 A B E C D 7 -1 -2 1 A B E C D 2 7 6 -1 -2 1 6 2 A B E C D 7 -1 -2 1 5 7 6 2 A B E C D -1 -2 1 5 3 6 2 A B E C D 7 -1 -2 1
DAG Shortest Paths Problem: finding shortest paths in DAG Idea: use topological sort If were lucky and processes vertices on each shortest path from left to right, would be done in one pass Every path in a DAG is subsequence of topologically sorted vertex order, so processing verts in that order, we will do each path in forward order (will never relax edges out of vertices before doing all edges into vertices). Thus: just one pass. Time: O(V + E)
Dijkstra’s Algorithm If no negative edge weights, Dijkstra’s essentially a weighted version of breadth-first search Similar to breadth-first search Grow a tree gradually, advancing from vertices taken from a queue Also similar to Prim’s algorithm for MST Use a priority queue keyed on d[v]
Dijkstra’s Algorithm 10 4 3 2 1 5 A B C D Dijkstra(G) for each v V d[v] = ; d[s] = 0; S = ; Q = V; while (Q ) u = ExtractMin(Q); S = S U {u}; for each v u->Adj[] if (d[v] > d[u]+w(u,v)) d[v] = d[u]+w(u,v); Relaxation Step
Dijkstra’s Algorithm B Dijkstra(G) for each v V d[v] = ; d[s] = 0; S = ; Q = V; while (Q ) u = ExtractMin(Q); S = S U {u}; for each v u->Adj[] if (d[v] > d[u]+w(u,v)) d[v] = d[u]+w(u,v); 2 s A 10 D 4 3 5 1 C
Dijkstra’s Algorithm B Dijkstra(G) for each v V d[v] = ; d[s] = 0; S = ; Q = V; while (Q ) u = ExtractMin(Q); S = S U {u}; for each v u->Adj[] if (d[v] > d[u]+w(u,v)) d[v] = d[u]+w(u,v); 10 2 s A 10 D 4 3 5 1 5 C S = {A}
Dijkstra’s Algorithm B Dijkstra(G) for each v V d[v] = ; d[s] = 0; S = ; Q = V; while (Q ) u = ExtractMin(Q); S = S U {u}; for each v u->Adj[] if (d[v] > d[u]+w(u,v)) d[v] = d[u]+w(u,v); 9 2 s A 10 D 4 3 6 5 1 5 C S = {A, C}
Dijkstra’s Algorithm B Dijkstra(G) for each v V d[v] = ; d[s] = 0; S = ; Q = V; while (Q ) u = ExtractMin(Q); S = S U {u}; for each v u->Adj[] if (d[v] > d[u]+w(u,v)) d[v] = d[u]+w(u,v); 8 2 s A 10 D 4 3 6 5 1 5 C S = {A, C, D}
Dijkstra’s Algorithm B Dijkstra(G) for each v V d[v] = ; d[s] = 0; S = ; Q = V; while (Q ) u = ExtractMin(Q); S = S U {u}; for each v u->Adj[] if (d[v] > d[u]+w(u,v)) d[v] = d[u]+w(u,v); 8 2 s A 10 D 4 3 6 5 1 5 C S = {A, C, D, B}
Dijkstra’s Algorithm B Dijkstra(G) for each v V d[v] = ; d[s] = 0; S = ; Q = V; while (Q ) u = ExtractMin(Q); S = S U {u}; for each v u->Adj[] if (d[v] > d[u]+w(u,v)) d[v] = d[u]+w(u,v); 8 2 s A 10 D 4 3 6 5 1 5 C
Dijkstra’s Algorithm Dijkstra(G) for each v V d[v] = ; d[s] = 0; S = ; Q = V; while (Q ) u = ExtractMin(Q); S = S U {u}; for each v u->Adj[] if (d[v] > d[u]+w(u,v)) d[v] = d[u]+w(u,v); How many times is ExtractMin() called? How many times is Relax() called? What will be the total running time? A: O(E lg V) using binary heap for Q