14 Graph Algorithms Hongfei Yan June 8, 2016.

14 Graph Algorithms Hongfei Yan June 8, 2016

Assignment #11 on chapter 14: Graph Algorithms
dfs and bfs topological ordering shortest path minimum spanning tree

Contents 01 Python Primer (P2-51)
02 Object-Oriented Programming (P57-103) 03 Algorithm Analysis (P ) 04 Recursion (P ) 05 Array-Based Sequences (P ) 06 Stacks, Queues, and Deques (P ) 07 Linked Lists (P ) 08 Trees (P ) 09 Priority Queues (P ) 10 Maps, Hash Tables, and Skip Lists (P ) 11 Search Trees (P ) 12 Sorting and Selection (P ) 13 Text Processing (P ) 14 Graph Algorithms (P ) 15 Memory Management and B-Trees (P ) ISLR_Print6

Contents 14.1 Graphs 14.2 Data Structures for Graphs 14.3 Graph Traversals 14.4 Tansitive Closure 14.5 Directed Acyclic Graphs 14.6 Shortest Paths 14.7 Minimum Spanning Trees

14.1 Graphs ORD SFO LAX DFW 1843 802 1743 337 1233 Graphs
1/16/2019 6:47 PM 14.1 Graphs 1843 ORD SFO 802 1743 337 LAX 1233 DFW

Graphs PVD ORD SFO LGA HNL LAX DFW MIA A graph is a pair (V, E), where
V is a set of nodes, called vertices E is a collection of pairs of vertices, called edges Vertices and edges are positions and store elements Example: A vertex represents an airport and stores the three-letter airport code An edge represents a flight route between two airports and stores the mileage of the route 849 PVD 1843 ORD 142 SFO 802 LGA 1743 337 1387 HNL 2555 1099 LAX 1233 DFW 1120 MIA

Edge Types flight AA 1206 ORD PVD 849 miles ORD PVD Directed edge
ordered pair of vertices (u,v) first vertex u is the origin second vertex v is the destination e.g., a flight Undirected edge unordered pair of vertices (u,v) e.g., a flight route Directed graph all the edges are directed e.g., route network Undirected graph all the edges are undirected e.g., flight network flight AA 1206 ORD PVD 849 miles ORD PVD

Applications Electronic circuits Transportation networks
Printed circuit board Integrated circuit Transportation networks Highway network Flight network Computer networks Local area network Internet Web Databases Entity-relationship diagram

Terminology X U V W Z Y a c b e d f g h i j
End vertices (or endpoints) of an edge U and V are the endpoints of a Edges incident on a vertex a, d, and b are incident on V Adjacent vertices U and V are adjacent Degree of a vertex X has degree 5 Parallel edges h and i are parallel edges Self-loop j is a self-loop X U V W Z Y a c b e d f g h i j

Terminology (cont.) Path Simple path Examples V a b P1 d U X Z P2 h c
sequence of alternating vertices and edges begins with a vertex ends with a vertex each edge is preceded and followed by its endpoints Simple path path such that all its vertices and edges are distinct Examples P1=(V,b,X,h,Z) is a simple path P2=(U,c,W,e,X,g,Y,f,W,d,V) is a path that is not simple V a b P1 d U X Z P2 h c e W g f Y

Terminology (cont.) V a b d U X Z C2 h e C1 c W g f Y Cycle
circular sequence of alternating vertices and edges each edge is preceded and followed by its endpoints Simple cycle cycle such that all its vertices and edges are distinct Examples C1=(V,b,X,g,Y,f,W,c,U,a,) is a simple cycle C2=(U,c,W,e,X,g,Y,f,W,d,V,a,) is a cycle that is not simple V a b d U X Z C2 h e C1 c W g f Y

Properties Sv deg(v) = 2m Property 1 Property 2 Notation Example n = 4
Proof: each edge is counted twice Property 2 In an undirected graph with no self-loops and no multiple edges m  n (n - 1)/2 Proof: each vertex has degree at most (n - 1) Notation n number of vertices m number of edges deg(v) degree of vertex v Example n = 4 m = 6 deg(v) = 3 What is the bound for a directed graph?

Vertices and Edges A graph is a collection of vertices and edges.
We model the abstraction as a combination of three data types: Vertex, Edge, and Graph. A Vertex is a lightweight object that stores an arbitrary element provided by the user (e.g., an airport code) We assume it supports a method, element(), to retrieve the stored element. An Edge stores an associated object (e.g., a flight number, travel distance, cost), retrieved with the element( ) method. In addition, we assume that an Edge supports the following methods:

Graph ADT

14.2 Data Structures for Graphs

Edge List Structure Vertex object Edge object Vertex sequence
element reference to position in vertex sequence Edge object origin vertex object destination vertex object reference to position in edge sequence Vertex sequence sequence of vertex objects Edge sequence sequence of edge objects In an edge list, we maintain an unordered list of all edges. This minimally suffices, but there is no efficient way to locate a particular edge (u,v), or the set of all edges incident to a vertex v.

Adjacency List Structure
Incidence sequence for each vertex sequence of references to edge objects of incident edges Augmented edge objects references to associated positions in incidence sequences of end vertices In an adjacency list, we maintain, for each vertex, a separate list containing those edges that are incident to the vertex. The complete set of edges can be determined by taking the union of the smaller sets, while the organization allows us to more efficiently find all edges incident to a given vertex.

Adjacency map structure
An adjacency map is very similar to an adjacency list, but the secondary container of all edges incident to a vertex is organized as a map, rather than as a list, with the adjacent vertex serving as a key. This allows for access to a specific edge (u,v) in O(1) expected time.

Adjacency Matrix Structure
Edge list structure Augmented vertex objects Integer key (index) associated with vertex 2D-array adjacency array Reference to edge object for adjacent vertices Null for non nonadjacent vertices The “old fashioned” version just has 0 for no edge and 1 for edge An adjacency matrix provides worst-case O(1) access to a specific edge (u,v) by maintaining an n×n matrix, for a graph with n vertices. Each entry is dedicated to storing a reference to the edge (u,v) for a particular pair of vertices u and v; if no such edge exists, the entry will be None.

Performance n vertices, m edges no parallel edges no self-loops

Python Graph Implementation
We use a variant of the adjacency map representation. For each vertex v, we use a Python dictionary to represent the secondary incidence map I(v). The list V is replaced by a top-level dictionary D that maps each vertex v to its incidence map I(v). Note that we can iterate through all vertices by generating the set of keys for dictionary D. A vertex does not need to explicitly maintain a reference to its position in D, because it can be determined in O(1) expected time. Running time bounds for the adjacency-list graph ADT operations, given above, become expected bounds.

Vertex Class

Edge Class

Graph, Part 1

Graph, end

14.3 Graph Traversals Depth-First Search DFS Implementation and Extensions Breadth-First Search 14.4 Transitive Closure 14.5 Directed Acyclic Graphs Topological Ordering

14.3.1 Depth-First Search D B A C E Depth-First Search
1/16/2019 6:47 PM Depth-First Search D B A C E

Subgraphs A subgraph S of a graph G is a graph such that
The vertices of S are a subset of the vertices of G The edges of S are a subset of the edges of G A spanning subgraph of G is a subgraph that contains all the vertices of G Subgraph Spanning subgraph

Non connected graph with two connected components
Connectivity A graph is connected if there is a path between every pair of vertices A connected component of a graph G is a maximal connected subgraph of G Connected graph Non connected graph with two connected components

Trees and Forests A (free) tree is an undirected graph T such that
T is connected T has no cycles This definition of tree is different from the one of a rooted tree A forest is an undirected graph without cycles The connected components of a forest are trees Tree Forest

Spanning Trees and Forests
A spanning tree of a connected graph is a spanning subgraph that is a tree A spanning tree is not unique unless the graph is a tree Spanning trees have applications to the design of communication networks A spanning forest of a graph is a spanning subgraph that is a forest Graph Spanning tree

Depth-First Search Depth-first search (DFS) is a general technique for traversing a graph A DFS traversal of a graph G Visits all the vertices and edges of G Determines whether G is connected Computes the connected components of G Computes a spanning forest of G DFS on a graph with n vertices and m edges takes O(n + m ) time DFS can be further extended to solve other graph problems Find and report a path between two given vertices Find a cycle in the graph Depth-first search is to graphs what Euler tour is to binary trees

DFS Algorithm The algorithm uses a mechanism for setting and getting “labels” of vertices and edges Algorithm DFS(G, v) Input graph G and a start vertex v of G Output labeling of the edges of G in the connected component of v as discovery edges and back edges setLabel(v, VISITED) for all e  G.incidentEdges(v) if getLabel(e) = UNEXPLORED w  opposite(v,e) if getLabel(w) = UNEXPLORED setLabel(e, DISCOVERY) DFS(G, w) else setLabel(e, BACK) Algorithm DFS(G) Input graph G Output labeling of the edges of G as discovery edges and back edges for all u  G.vertices() setLabel(u, UNEXPLORED) for all e  G.edges() setLabel(e, UNEXPLORED) for all v  G.vertices() if getLabel(v) = UNEXPLORED DFS(G, v)

Python Implementation

Example unexplored vertex visited vertex unexplored edge
B A C E A A visited vertex unexplored edge discovery edge back edge D B A C E D B A C E Depth-First Search

Example (cont.) D B A C E D B A C E D B A C E D B A C E
Depth-First Search

DFS and Maze Traversal The DFS algorithm is similar to a classic strategy for exploring a maze We mark each intersection, corner and dead end (vertex) visited We mark each corridor (edge ) traversed We keep track of the path back to the entrance (start vertex) by means of a rope (recursion stack)

Properties of DFS Property 1 Property 2
DFS(G, v) visits all the vertices and edges in the connected component of v Property 2 The discovery edges labeled by DFS(G, v) form a spanning tree of the connected component of v D B A C E

Analysis of DFS Setting/getting a vertex/edge label takes O(1) time
Each vertex is labeled twice once as UNEXPLORED once as VISITED Each edge is labeled twice once as DISCOVERY or BACK Method incidentEdges is called once for each vertex DFS runs in O(n + m) time provided the graph is represented by the adjacency list structure Recall that Sv deg(v) = 2m Depth-First Search

Depth-First Search 1/16/2019 6:47 PM Path Finding Algorithm pathDFS(G, v, z) setLabel(v, VISITED) S.push(v) if v = z return S.elements() for all e  G.incidentEdges(v) if getLabel(e) = UNEXPLORED w  opposite(v,e) if getLabel(w) = UNEXPLORED setLabel(e, DISCOVERY) S.push(e) pathDFS(G, w, z) S.pop(e) else setLabel(e, BACK) S.pop(v) We can specialize the DFS algorithm to find a path between two given vertices u and z using the template method pattern We call DFS(G, u) with u as the start vertex We use a stack S to keep track of the path between the start vertex and the current vertex As soon as destination vertex z is encountered, we return the path as the contents of the stack Code Fragment 14.6: Function to reconstruct a directed path from u to v, given the trace of discovery from a DFS started at u. The function returns an ordered list of vertices on the path.

Cycle Finding Algorithm cycleDFS(G, v, z) setLabel(v, VISITED) S.push(v) for all e  G.incidentEdges(v) if getLabel(e) = UNEXPLORED w  opposite(v,e) S.push(e) if getLabel(w) = UNEXPLORED setLabel(e, DISCOVERY) pathDFS(G, w, z) S.pop(e) else T  new empty stack repeat o  S.pop() T.push(o) until o = w return T.elements() S.pop(v) We can specialize the DFS algorithm to find a simple cycle using the template method pattern We use a stack S to keep track of the path between the start vertex and the current vertex As soon as a back edge (v, w) is encountered, we return the cycle as the portion of the stack from the top to vertex w

14.3.3 Breadth-First Search L0 L1 L2 C B A E D F Breadth-First Search
1/16/2019 6:47 PM Breadth-First Search C B A E D L0 L1 F L2

Breadth-First Search Breadth-first search (BFS) is a general technique for traversing a graph A BFS traversal of a graph G Visits all the vertices and edges of G Determines whether G is connected Computes the connected components of G Computes a spanning forest of G BFS on a graph with n vertices and m edges takes O(n + m ) time BFS can be further extended to solve other graph problems Find and report a path with the minimum number of edges between two given vertices Find a simple cycle, if there is one

BFS Algorithm Algorithm BFS(G, s)
L0  new empty sequence L0.addLast(s) setLabel(s, VISITED) i  0 while Li.isEmpty() Li +1  new empty sequence for all v  Li.elements() for all e  G.incidentEdges(v) if getLabel(e) = UNEXPLORED w  opposite(v,e) if getLabel(w) = UNEXPLORED setLabel(e, DISCOVERY) setLabel(w, VISITED) Li +1.addLast(w) else setLabel(e, CROSS) i  i +1 The algorithm uses a mechanism for setting and getting “labels” of vertices and edges Algorithm BFS(G) Input graph G Output labeling of the edges and partition of the vertices of G for all u  G.vertices() setLabel(u, UNEXPLORED) for all e  G.edges() setLabel(e, UNEXPLORED) for all v  G.vertices() if getLabel(v) = UNEXPLORED BFS(G, v)

Example unexplored vertex visited vertex unexplored edge
C B A E D L0 L1 F A unexplored vertex A visited vertex unexplored edge discovery edge cross edge L0 L0 A A L1 L1 B C D B C D E F E F Breadth-First Search

Example (cont.) L0 L1 L0 L1 L2 L0 L1 L2 L0 L1 L2 C B A E D F C B A E D
Breadth-First Search

Example (cont.) L0 L1 L2 L0 L1 L2 L0 L1 L2 C B A E D F A B C D E F C B
Breadth-First Search

Properties Notation Property 1 Property 2 Property 3
Gs: connected component of s Property 1 BFS(G, s) visits all the vertices and edges of Gs Property 2 The discovery edges labeled by BFS(G, s) form a spanning tree Ts of Gs Property 3 For each vertex v in Li The path of Ts from s to v has i edges Every path from s to v in Gs has at least i edges A B C D E F L0 A L1 B C D L2 E F

Analysis Setting/getting a vertex/edge label takes O(1) time
Each vertex is labeled twice once as UNEXPLORED once as VISITED Each edge is labeled twice once as DISCOVERY or CROSS Each vertex is inserted once into a sequence Li Method incidentEdges is called once for each vertex BFS runs in O(n + m) time provided the graph is represented by the adjacency list structure Recall that Sv deg(v) = 2m

Applications Using the template method pattern, we can specialize the BFS traversal of a graph G to solve the following problems in O(n + m) time Compute the connected components of G Compute a spanning forest of G Find a simple cycle in G, or report that G is a forest Given two vertices of G, find a path in G between them with the minimum number of edges, or report that no such path exists

DFS vs. BFS Applications DFS BFS DFS BFS
Spanning forest, connected components, paths, cycles  Shortest paths Biconnected components C B A E D L0 L1 F L2 A B C D E F DFS BFS

DFS vs. BFS (cont.) Back edge (v,w) Cross edge (v,w) DFS BFS
w is an ancestor of v in the tree of discovery edges Cross edge (v,w) w is in the same level as v or in the next level C B A E D L0 L1 F L2 A B C D E F DFS BFS Breadth-First Search

14.5 Directed Graphs Shortest Path 1/16/2019 6:47 PM BOS ORD JFK SFO
DFW LAX MIA

Digraphs A digraph is a graph whose edges are all directed
Short for “directed graph” Applications one-way streets flights task scheduling A C E B D Directed Graphs

Digraph Properties A graph G=(V,E) such that
B D Digraph Properties A graph G=(V,E) such that Each edge goes in one direction: Edge (a,b) goes from a to b, but not b to a If G is simple, m < n(n - 1) If we keep in-edges and out-edges in separate adjacency lists, we can perform listing of incoming edges and outgoing edges in time proportional to their size

Digraph Application Scheduling: edge (a,b) means task a must be completed before b can be started ics21 ics22 ics23 ics51 ics53 ics52 ics161 ics131 ics141 ics121 ics171 The good life ics151 Directed Graphs

Directed DFS We can specialize the traversal algorithms (DFS and BFS) to digraphs by traversing edges only along their direction In the directed DFS algorithm, we have four types of edges discovery edges back edges forward edges cross edges A directed DFS starting at a vertex s determines the vertices reachable from s E D C B A Directed Graphs

Reachability DFS tree rooted at v: vertices reachable from v via directed paths E D E D C A C F E D A B C F A B Directed Graphs

Strong Connectivity Each vertex can reach all other vertices a g c d e
b e f g Directed Graphs

Strong Connectivity Algorithm
Pick a vertex v in G Perform a DFS from v in G If there’s a w not visited, print “no” Let G’ be G with edges reversed Perform a DFS from v in G’ Else, print “yes” Running time: O(n+m) a G: g c d e b f a G’: g c d e b f Directed Graphs

Strongly Connected Components
Maximal subgraphs such that each vertex can reach all other vertices in the subgraph Can also be done in O(n+m) time using DFS, but is more complicated (similar to biconnectivity). a d c b e f g { a , c , g } { f , d , e , b } Directed Graphs

Transitive Closure D E Given a digraph G, the transitive closure of G is the digraph G* such that G* has the same vertices as G if G has a directed path from u to v (u  v), G* has a directed edge from u to v The transitive closure provides reachability information about a digraph B G C A D E B C A G* Directed Graphs

Computing the Transitive Closure
If there's a way to get from A to B and from B to C, then there's a way to get from A to C. We can perform DFS starting at each vertex O(n(n+m)) Alternatively ... Use dynamic programming: The Floyd-Warshall Algorithm Directed Graphs

Floyd-Warshall Transitive Closure
Idea #1: Number the vertices 1, 2, …, n. Idea #2: Consider paths that use only vertices numbered 1, 2, …, k, as intermediate vertices: Uses only vertices numbered 1,…,k (add this edge if it’s not already in) i j Uses only vertices numbered 1,…,k-1 Uses only vertices numbered 1,…,k-1 k Directed Graphs

Floyd-Warshall’s Algorithm
Number vertices v1 , …, vn Compute digraphs G0, …, Gn G0=G Gk has directed edge (vi, vj) if G has a directed path from vi to vj with intermediate vertices in {v1 , …, vk} We have that Gn = G* In phase k, digraph Gk is computed from Gk - 1 Running time: O(n3), assuming areAdjacent is O(1) (e.g., adjacency matrix) Algorithm FloydWarshall(G) Input digraph G Output transitive closure G* of G i  1 for all v  G.vertices() denote v as vi i  i + 1 G0  G for k  1 to n do Gk  Gk - 1 for i  1 to n (i  k) do for j  1 to n (j  i, k) do if Gk - 1.areAdjacent(vi, vk)  Gk - 1.areAdjacent(vk, vj) if Gk.areAdjacent(vi, vj) Gk.insertDirectedEdge(vi, vj , k) return Gn Directed Graphs

Directed Graphs

Floyd-Warshall Example
BOS v ORD 4 JFK v 2 v 6 SFO DFW LAX v 3 v 1 MIA v 5 Directed Graphs

Floyd-Warshall, Iteration 1

Floyd-Warshall, Conclusion

14.5.1 DAGs and Topological Ordering
A directed acyclic graph (DAG) is a digraph that has no directed cycles A topological ordering of a digraph is a numbering v1 , …, vn of the vertices such that for every edge (vi , vj), we have i < j Example: in a task scheduling digraph, a topological ordering a task sequence that satisfies the precedence constraints Theorem A digraph admits a topological ordering if and only if it is a DAG B C A DAG G v4 v5 D E v2 B v3 C v1 Topological ordering of G A Directed Graphs

Topological Sorting Number vertices, so that (u,v) in E implies u < v 1 A typical student day wake up 2 3 eat study computer sci. 4 5 nap more c.s. 7 play 8 write c.s. program 6 9 work out bake cookies 10 sleep 11 dream about graphs

Algorithm for Topological Sorting
Note: This algorithm is different than the one in the book Running time: O(n + m) Algorithm TopologicalSort(G) H  G // Temporary copy of G n  G.numVertices() while H is not empty do Let v be a vertex with no outgoing edges Label v  n n  n - 1 Remove v from H

Implementation with DFS
Simulate the algorithm by using depth-first search O(n+m) time. Algorithm topologicalDFS(G, v) Input graph G and a start vertex v of G Output labeling of the vertices of G in the connected component of v setLabel(v, VISITED) for all e  G.outEdges(v) { outgoing edges } w  opposite(v,e) if getLabel(w) = UNEXPLORED { e is a discovery edge } topologicalDFS(G, w) else { e is a forward or cross edge } Label v with topological number n n  n - 1 Algorithm topologicalDFS(G) Input dag G Output topological ordering of G n  G.numVertices() for all u  G.vertices() setLabel(u, UNEXPLORED) for all v  G.vertices() if getLabel(v) = UNEXPLORED topologicalDFS(G, v) Directed Graphs

Topological Sorting Example
Directed Graphs

9 Directed Graphs

8 9 Directed Graphs

7 8 9 Directed Graphs

6 7 8 9 Directed Graphs

6 5 7 8 9 Directed Graphs

4 6 5 7 8 9 Directed Graphs

3 4 6 5 7 8 9 Directed Graphs

2 3 4 6 5 7 8 9 Directed Graphs

2 1 3 4 6 5 7 8 9 Directed Graphs

14.6 Shortest Paths C B A E D F 3 2 8 5 4 7 1 9 Shortest Path
1/16/2019 6:47 PM 14.6 Shortest Paths C B A E D F 3 2 8 5 4 7 1 9

Weighted Graphs PVD ORD SFO LGA HNL LAX DFW MIA
In a weighted graph, each edge has an associated numerical value, called the weight of the edge Edge weights may represent, distances, costs, etc. Example: In a flight route graph, the weight of an edge represents the distance in miles between the endpoint airports 849 PVD 1843 ORD 142 SFO 802 LGA 1743 1205 337 1387 HNL 2555 1099 LAX 1233 DFW 1120 MIA Shortest Paths

Shortest Paths PVD ORD SFO LGA HNL LAX DFW MIA
Given a weighted graph and two vertices u and v, we want to find a path of minimum total weight between u and v. Length of a path is the sum of the weights of its edges. Example: Shortest path between Providence and Honolulu Applications Internet packet routing Flight reservations Driving directions 849 PVD 1843 ORD 142 SFO 802 LGA 1743 1205 337 1387 HNL 2555 1099 LAX 1233 DFW 1120 MIA Shortest Paths

Shortest Path Properties
Property 1: A subpath of a shortest path is itself a shortest path Property 2: There is a tree of shortest paths from a start vertex to all the other vertices Example: Tree of shortest paths from Providence 849 PVD 1843 ORD 142 SFO 802 LGA 1743 1205 337 1387 HNL 2555 1099 LAX 1233 DFW 1120 MIA Shortest Paths

Dijkstra’s Algorithm The distance of a vertex v from a vertex s is the length of a shortest path between s and v Dijkstra’s algorithm computes the distances of all the vertices from a given start vertex s Assumptions: the graph is connected the edges are undirected the edge weights are nonnegative We grow a “cloud” of vertices, beginning with s and eventually covering all the vertices We store with each vertex v a label d(v) representing the distance of v from s in the subgraph consisting of the cloud and its adjacent vertices At each step We add to the cloud the vertex u outside the cloud with the smallest distance label, d(u) We update the labels of the vertices adjacent to u

Edge Relaxation Consider an edge e = (u,z) such that d(z) = 75
u is the vertex most recently added to the cloud z is not in the cloud The relaxation of edge e updates distance d(z) as follows: d(z)  min{d(z),d(u) + weight(e)} d(u) = 50 10 d(z) = 75 e u s z d(u) = 50 10 d(z) = 60 u e s z

Example C B A E D F 3 2 8 5 4 7 1 9 A 4 8 2 8 2 4 7 1 B C D 3 9   2 5 E F A 4 A 8 8 4 2 2 8 2 3 7 2 3 7 1 7 1 B C D B C D 3 9 3 9 5 11 5 8 2 5 2 5 E F E F Shortest Paths

Example (cont.) A 4 8 2 7 2 3 7 1 B C D 3 9 5 8 2 5 E F A 8 4 2 7 2 3
A 4 8 2 7 2 3 7 1 B C D 3 9 5 8 2 5 E F A 8 4 2 7 2 3 7 1 B C D 3 9 5 8 2 5 E F Shortest Paths

Dijkstra’s Algorithm

Analysis of Dijkstra’s Algorithm
Graph operations We find all the incident edges once for each vertex Label operations We set/get the distance and locator labels of vertex z O(deg(z)) times Setting/getting a label takes O(1) time Priority queue operations Each vertex is inserted once into and removed once from the priority queue, where each insertion or removal takes O(log n) time The key of a vertex in the priority queue is modified at most deg(w) times, where each key change takes O(log n) time Dijkstra’s algorithm runs in O((n + m) log n) time provided the graph is represented by the adjacency list/map structure Recall that Sv deg(v) = 2m The running time can also be expressed as O(m log n) since the graph is connected

Why Dijkstra’s Algorithm Works
Dijkstra’s algorithm is based on the greedy method. It adds vertices by increasing distance. Suppose it didn’t find all shortest distances. Let F be the first wrong vertex the algorithm processed. When the previous node, D, on the true shortest path was considered, its distance was correct But the edge (D,F) was relaxed at that time! Thus, so long as d(F)>d(D), F’s distance cannot be wrong. That is, there is no wrong vertex A 4 8 2 7 2 3 7 1 B C D 3 9 5 8 2 5 E F Shortest Paths

Why It Doesn’t Work for Negative-Weight Edges
Dijkstra’s algorithm is based on the greedy method. It adds vertices by increasing distance. A 4 8 If a node with a negative incident edge were to be added late to the cloud, it could mess up distances for vertices already in the cloud. 6 7 5 4 7 1 B C D -8 5 9 2 5 E F C’s true distance is 1, but it is already in the cloud with d(C)=5! Shortest Paths

Bellman-Ford Algorithm (not in book)
Works even with negative- weight edges Must assume directed edges (for otherwise we would have negative-weight cycles) Iteration i finds all shortest paths that use i edges. Running time: O(nm). Can be extended to detect a negative-weight cycle if it exists How? Algorithm BellmanFord(G, s) for all v  G.vertices() if v = s setDistance(v, 0) else setDistance(v, ) for i  1 to n - 1 do for each e  G.edges() { relax edge e } u  G.origin(e) z  G.opposite(u,e) r  getDistance(u) + weight(e) if r < getDistance(z) setDistance(z,r) Shortest Paths

Bellman-Ford Example Nodes are labeled with their d(v) values 4 4 8 8
4 4 8 8 -2 -2 8 -2 4 7 1 7 1       3 9 3 9 -2 5 -2 5     4 4 8 8 -2 -2 5 7 1 -1 7 1 8 -2 4 5 -2 -1 1 3 9 3 9 6 9 4 -2 5 -2 5   1 9 Shortest Paths

DAG-based Algorithm (not in book)
Algorithm DagDistances(G, s) for all v  G.vertices() if v = s setDistance(v, 0) else setDistance(v, ) { Perform a topological sort of the vertices } for u  1 to n do {in topological order} for each e  G.outEdges(u) { relax edge e } z  G.opposite(u,e) r  getDistance(u) + weight(e) if r < getDistance(z) setDistance(z,r) Works even with negative- weight edges Uses topological order Doesn’t use any fancy data structures Is much faster than Dijkstra’s algorithm Running time: O(n+m). Shortest Paths

DAG Example Nodes are labeled with their d(v) values 4 4 8 8 -2 -2 8
1 1 4 4 8 8 -2 -2 4 8 -2 4 3 7 2 1 3 7 2 1 4       3 9 3 9 -5 5 -5 5     6 5 6 5 1 1 4 4 8 8 -2 -2 5 3 7 2 1 4 -1 3 7 2 1 4 8 -2 4 5 -2 -1 9 3 3 9 1 7 4 -5 5 -5 5   1 7 6 5 6 5 Shortest Paths (two steps)

14.5 Minimum Spanning Trees
1/16/2019 6:47 PM 14.5 Minimum Spanning Trees

Minimum Spanning Trees
Spanning subgraph Subgraph of a graph G containing all the vertices of G Spanning tree Spanning subgraph that is itself a (free) tree Minimum spanning tree (MST) Spanning tree of a weighted graph with minimum total edge weight Applications Communications networks Transportation networks ORD 10 1 PIT DEN 6 7 9 3 DCA STL 4 8 5 2 DFW ATL

Cycle Property 8 4 2 3 6 7 9 e C f Cycle Property: Let T be a minimum spanning tree of a weighted graph G Let e be an edge of G that is not in T and C let be the cycle formed by e with T For every edge f of C, weight(f)  weight(e) Proof: By contradiction If weight(f) > weight(e) we can get a spanning tree of smaller weight by replacing e with f Replacing f with e yields a better spanning tree 8 4 2 3 6 7 9 C e f Minimum Spanning Trees

Partition Property U V 7 f Partition Property: Consider a partition of the vertices of G into subsets U and V Let e be an edge of minimum weight across the partition There is a minimum spanning tree of G containing edge e Proof: Let T be an MST of G If T does not contain e, consider the cycle C formed by e with T and let f be an edge of C across the partition By the cycle property, weight(f)  weight(e) Thus, weight(f) = weight(e) We obtain another MST by replacing f with e 4 9 5 2 8 8 3 e 7 Replacing f with e yields another MST U V 7 f 4 9 5 2 8 8 3 e 7 Minimum Spanning Trees

Prim-Jarnik’s Algorithm
Similar to Dijkstra’s algorithm We pick an arbitrary vertex s and we grow the MST as a cloud of vertices, starting from s We store with each vertex v label d(v) representing the smallest weight of an edge connecting v to a vertex in the cloud At each step: We add to the cloud the vertex u outside the cloud with the smallest distance label We update the labels of the vertices adjacent to u

Prim-Jarnik Pseudo-code

Example  7 7 D 7 D 2 2 B 4 B 4 8 9  5 9  5 5 2 F 2 F C C 8 8 3 3 8 8 E E A 7 A 7 7 7 7 7 7 D 2 7 D 2 B 4 B 4 5 9  5 5 9 4 2 F 5 C 2 F 8 C 8 3 8 3 8 E A E 7 7 A 7 7 Minimum Spanning Trees

Example (contd.) 7 7 D 2 B 4 9 4 5 5 2 F C 8 3 8 E A 3 7 7 7 D 2 B 4 5 9 4 5 2 F C 8 3 8 E A 3 7 Minimum Spanning Trees

Analysis Graph operations We cycle through the incident edges once for each vertex Label operations We set/get the distance, parent and locator labels of vertex z O(deg(z)) times Setting/getting a label takes O(1) time Priority queue operations Each vertex is inserted once into and removed once from the priority queue, where each insertion or removal takes O(log n) time The key of a vertex w in the priority queue is modified at most deg(w) times, where each key change takes O(log n) time Prim-Jarnik’s algorithm runs in O((n + m) log n) time provided the graph is represented by the adjacency list structure Recall that Sv deg(v) = 2m The running time is O(m log n) since the graph is connected Minimum Spanning Trees

Kruskal’s Approach Maintain a partition of the vertices into clusters
Initially, single-vertex clusters Keep an MST for each cluster Merge “closest” clusters and their MSTs A priority queue stores the edges outside clusters Key: weight Element: edge At the end of the algorithm One cluster and one MST

Kruskal’s Algorithm

Example G G 8 8 B 4 B 4 E 9 E 6 9 6 5 1 F 5 1 F C 3 11 C 3 11 2 2 7 H 7 H D D A 10 A 10 G G 8 8 B 4 B 4 9 E E 6 9 6 5 1 F 5 1 F C 3 3 11 C 11 2 2 7 H 7 H D D A A 10 10 Campus Tour

Example (contd.) four steps two steps G G 8 8 B 4 B 4 E 9 E 6 9 6 5 5
1 F 1 F C 3 C 3 11 11 2 2 7 H 7 H D D A 10 A 10 four steps two steps G G 8 8 B 4 B 4 E E 9 6 9 6 5 5 1 F 1 F C 3 11 C 3 11 2 2 7 H 7 H D D A A 10 10 Campus Tour

Data Structure for Kruskal’s Algorithm
The algorithm maintains a forest of trees A priority queue extracts the edges by increasing weight An edge is accepted it if connects distinct trees We need a data structure that maintains a partition, i.e., a collection of disjoint sets, with operations: makeSet(u): create a set consisting of u find(u): return the set storing u union(A, B): replace sets A and B with their union Minimum Spanning Trees

List-based Partition Each set is stored in a sequence Each element has a reference back to the set operation find(u) takes O(1) time, and returns the set of which u is a member. in operation union(A,B), we move the elements of the smaller set to the sequence of the larger set and update their references the time for operation union(A,B) is min(|A|, |B|) Whenever an element is processed, it goes into a set of size at least double, hence each element is processed at most log n times Minimum Spanning Trees

Partition-Based Implementation
Partition-based version of Kruskal’s Algorithm Cluster merges as unions Cluster locations as finds Running time O((n + m) log n) Priority Queue operations: O(m log n) Union-Find operations: O(n log n) Minimum Spanning Trees

Minimum Spanning Trees

Baruvka’s Algorithm (Exercise)
Like Kruskal’s Algorithm, Baruvka’s algorithm grows many clusters at once and maintains a forest T Each iteration of the while loop halves the number of connected components in forest T The running time is O(m log n) Algorithm BaruvkaMST(G) T  V {just the vertices of G} while T has fewer than n - 1 edges do for each connected component C in T do Let edge e be the smallest-weight edge from C to another component in T if e is not already in T then Add edge e to T return T Minimum Spanning Trees

Example of Baruvka’s Algorithm (animated)
Slide by Matt Stallmann included with permission. Example of Baruvka’s Algorithm (animated) 4 9 6 8 2 4 3 8 5 9 4 7 3 1 6 6 5 4 1 5 4 3 2 9 6 8 7 CSC 316

14.7.3 Union-Find Partition Structures
Minimum Spanning Tree 1/16/2019 6:47 PM Union-Find Partition Structures Union-Find

Partitions with Union-Find Operations
makeSet(x): Create a singleton set containing the element x and return the position storing x in this set union(A,B ): Return the set A U B, destroying the old A and B find(p): Return the set containing the element at position p Union-Find

List-based Implementation
Each set is stored in a sequence represented with a linked-list Each node should store an object containing the element and a reference to the set name Union-Find

Analysis of List-based Representation
When doing a union, always move elements from the smaller set to the larger set Each time an element is moved it goes to a set of size at least double its old set Thus, an element can be moved at most O(log n) times Total time needed to do n unions and finds is O(n log n). Union-Find

Tree-based Implementation
Each element is stored in a node, which contains a pointer to a set name A node v whose set pointer points back to v is also a set name Each set is a tree, rooted at a node with a self-referencing set pointer For example: The sets “1”, “2”, and “5”: 1 2 5 4 7 3 6 8 10 9 11 12 Union-Find

Union-Find Operations
5 To do a union, simply make the root of one tree point to the root of the other To do a find, follow set-name pointers from the starting node until reaching a node whose set-name pointer refers back to itself 2 8 10 3 6 11 9 12 5 2 8 10 3 6 11 9 12 Union-Find

Union-Find Heuristic 1 Union by size:
When performing a union, make the root of smaller tree point to the root of the larger Implies O(n log n) time for performing n union-find operations: Each time we follow a pointer, we are going to a subtree of size at least double the size of the previous subtree Thus, we will follow at most O(log n) pointers for any find. 5 2 8 10 3 6 11 9 12 Union-Find

Union-Find Heuristic 2 Path compression:
After performing a find, compress all the pointers on the path just traversed so that they all point to the root Implies O(n log* n) time for performing n union-find operations: Proof is somewhat involved… (and not in the book) 5 5 8 10 8 10 11 11 2 12 2 12 3 6 3 6 9 9 Union-Find

Union-Find

Proof of log* n Amortized Time
For each node v that is a root define n(v) to be the size of the subtree rooted at v (including v) identified a set with the root of its associated tree. We update the size field of v each time a set is unioned into v. Thus, if v is not a root, then n(v) is the largest the subtree rooted at v can be, which occurs just before we union v into some other node whose size is at least as large as v ’s. For any node v, then, define the rank of v, which we denote as r (v), as r (v) = [log n(v)]: Thus, n(v) ≥ 2r(v). Also, since there are at most n nodes in the tree of v, r (v) = [log n], for each node v. Union-Find

Proof of log* n Amortized Time (2)
For each node v with parent w: r (v ) > r (w ) Claim: There are at most n/ 2s nodes of rank s. Proof: Since r (v) < r (w), for any node v with parent w, ranks are monotonically increasing as we follow parent pointers up any tree. Thus, if r (v) = r (w) for two nodes v and w, then the nodes counted in n(v) must be separate and distinct from the nodes counted in n(w). If a node v is of rank s, then n(v) ≥ 2s. Therefore, since there are at most n nodes total, there can be at most n/ 2s that are of rank s. Union-Find

Definition: Tower of two’s function: t(i) = 2t(i-1) Nodes v and u are in the same rank group g if g = log*(r(v)) = log*(r(u)): Since the largest rank is log n, the largest rank group is log*(log n) = (log* n) - 1 Union-Find

Charge 1 cyber-dollar per pointer hop during a find: If w is the root or if w is in a different rank group than v, then charge the find operation one cyber-dollar. Otherwise (w is not a root and v and w are in the same rank group), charge the node v one cyber-dollar. Since there are most (log* n)-1 rank groups, this rule guarantees that any find operation is charged at most log* n cyber-dollars. Union-Find

After we charge a node v then v will get a new parent, which is a node higher up in v ’s tree. The rank of v ’s new parent will be greater than the rank of v ’s old parent w. Thus, any node v can be charged at most the number of different ranks that are in v ’s rank group. If v is in rank group g > 0, then v can be charged at most t(g)-t(g-1) times before v has a parent in a higher rank group (and from that point on, v will never be charged again). In other words, the total number, C, of cyber-dollars that can ever be charged to nodes can be bounded by Union-Find

Proof of log* n Amortized Time (end)
Bounding n(g): Returning to C: Union-Find

14 Graph Algorithms Hongfei Yan June 8, 2016.

Similar presentations

Presentation on theme: "14 Graph Algorithms Hongfei Yan June 8, 2016."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

14 Graph Algorithms Hongfei Yan June 8, 2016.

Similar presentations

Presentation on theme: "14 Graph Algorithms Hongfei Yan June 8, 2016."— Presentation transcript:

Similar presentations

About project

Feedback