1. Chapter 23 Minimum Spanning Trees
Kruskal
Prim
[Source: Cormen et al. textbook except where noted]
I joined the UMass Lowell Computer Science faculty this summer. This collection of slides is intended to familiarize the reader/viewer with my field of research (Computational Geometry), summarize my previous research results in this field and outline my plan for Computational Geometry research at UMass Lowell.

2. Minimum Spanning Trees
This refers to a number of problems (many of practical importance) that can be reduced to the following abstraction: let G = (V, E) be a connected, undirected graph. You are also given a function w : E  R (usually non-negative, but not always so required). You are then asked to find an acyclic subset T Í E that connects all the vertices and whose total weight is minimized.
The acyclicity of T implies that T is a tree; the requirement that it connect all the vertices implies that it is a spanning tree; the minimization implies that it is a minimum such object.

3. Minimum Spanning Trees
MST Example

4. Minimum Spanning Trees
Algorithms
We will construct two algorithms as variants of a “generic” one. Both will be greedy algorithms (requiring that we show all necessary greedy properties are satisfied).
We start with a loop invariant: let A be a set of edges; prior to each iteration, A is a subset of some minimum spanning tree.
At each step we determine an edge (u, v) (a safe edge) that we can add to A without violating the invariant.

5. Minimum Spanning Trees
The Generic Algorithm

6. Minimum Spanning Trees
The Generic Algorithm
How do we find a safe edge? It exists, because of the assumption on A (part of a minimum spanning tree)…
Def.: a cut (S, V – S) of an undirected graph G = (V, E) is a partition of V.
Def.: an edge (u, v) crosses the cut (S, V – S) if one of its endpoints is in S and the other in V.
Def.: a cut respects a set A of edges if no edge in A crosses the cut.
Def.: An edge is a light edge crossing a cut if its weight is minimum among all edges crossing the cut. Uniqueness is not required. This can be generalized to properties other than crossing.

7. Minimum Spanning Trees
The Generic Algorithm

8. Minimum Spanning Trees
The Generic Algorithm
Theorem Let G = (V, E) be a connected undirected graph with a real-valued weight function w defined on E. Let A be a subset of E that is included in some minimum spanning tree for G, let (S, V – S) be any cut of G that respects A, and let (u, v) be a light edge crossing (S, V – S). Then edge (u, v) is safe for A.
Proof: let T be an MST, A Í T, A ≠ T. Assume T does not contain the light edge (u, v). If it does, we are done. If not, we construct another MST T’ that contains both A and (u, v).
T {(u, v)} must contain a cycle, with edges on a simple path p from u to v in T. u and v are on opposite sides of the cut (S, V – S), and at least one edge in T lies on p and crosses the cut.

9. Minimum Spanning Trees
The Generic Algorithm
Proof (cont.) Let (x, y) be any such edge – it cannot be in A, because the cut respects A. Since (x, y) is on the unique simple path from u to v in T, removing (x, y) breaks T into two components. Adding (u, v) reconnects them to form a new spanning tree T’ = (T – {(x, y)}) {(u, v)}. But T’ is also an MST: since (u, v) is a light edge crossing (S, V – S) and (x, y) also crosses the cut, w(u, v) ≤ w(x, y) and w(T’) = w(T) – w(x, y) + w(u, v) ≤ w(T). The minimality of T implies w(T) ≤ w(T’), so T’ must be minimal, also.
Is (u, v) safe? Since A Í T and (x, y) Ï A, we have A Í T’.
Thus A {(u, v)} Í T’. Since T’ is an MST, (u, v) is safe for A.

10. Minimum Spanning Trees
The Generic Algorithm
Corollary Let G = (V, E) be a connected undirected graph with a real-valued weight function w defined on E. Let A be a subset of E that is included in some minimum spanning tree for G, let C = (VC, EC) be a connected component (tree) in the forest GA = (V, A). If (u, v) is a light edge connecting C to some other component in GA, then (u, v) is safe for A.
Proof: the cut (VC, V – VC) respects A, and (u, v) is a light edge for this cut. Therefore (u, v) is safe for A.

11. Minimum Spanning Trees
The Generic Algorithm
We now move to realizations of the generic algorithm.
We choose two distinct ones,
one based on a sorted (non-decreasing) list of edges and the set operations Make-Set, Find-Set and Union;
the other based on priority queues of edges and the operation Extract-Min.

12. Minimum Spanning Trees
Kruskal’s Algorithm

13. Minimum Spanning Trees
Kruskal’s Algorithm
This algorithm uses the disjoint-set datatype introduced in Ch. 21. The user-accessible operations are Make-Set(x), Union(x, y) and Find-Set(x), where x and y stand for either singletons or sets of which they are representative elements.
We can show (easily) that a sequence of m Make-Set, Union and Find-Set operations (with an easy implementation), n of which are Make-Set takes O(m lg n) time. A more complex implementation (with union-by-rank and path compression) and analysis gives a bound O(m•a(n)), where a(n) ≤ 4 for any value of n relevant to our universe.

14. Minimum Spanning Trees
Kruskal’s Algorithm
Line 1 takes O(1); lines 2 and 3 take O(n), but we will account for it later; line 4 takes O(|E| lg |E|); the for loop (ll. 5-8) performs O(|E|) Find-Set and Union operations, which, with the |V| Make-Set operations take a total of O((|V| + |E|)a(|V|)) time. The assumption that G is connected implies |E| ≥ |V| - 1, so that the disjoint-set operations take O(|E|a(|V|)). Since a(|V|) = O(lg |V|) = O(lg |E|), the total time of Kruskal’s algorithm is O(|E| lg |E|) and |E| < |V|2 reduces this to O(|E| lg |V|).

15. Minimum Spanning Trees
Kruskal’s Algorithm
We observe that the set A is a forest, whose vertices are all those of the given graph. A safe edge is added to A: it is always a least-weight edge that connects two distinct components.

16. Minimum Spanning Trees
Prim’s Algorithm

17. Minimum Spanning Trees
Prim’s Algorithm
The edges in the set A always make up a single tree.
The tree starts from an arbitrary root vertex r and grows until it spans all vertices in V.
Each step adds to the tree A a light edge that connects A to an isolated vertex (one on which no edge of A is incident)
By Cor. 23.2, this process this adds only edges that are safe for A, so that, at the end, A forms an MST.
The strategy is greedy, since at each step it adds to the tree an edge of minimum weight.

18. Minimum Spanning Trees
Prim’s Algorithm: Invariant
Prior to each iteration of the while loop (ll. 6-11):
A = {(v, v.p) : v Î V – {r} – Q}
The vertices already placed into the minimum spanning tree are those in V – Q
For all vertices v Î Q, if v.p ≠ NIL, then v.key < ¥ and v.key is the weight of a light edge (v, v.p) connecting v to some vertex already placed into the MST.

19. Minimum Spanning Trees
Prim’s Algorithm: Maintenance
L. 7 extracts a vertex u in Q incident on a light edge that crosses the cut (V – Q, Q) (except for the first iteration, which extracts r).
Removing u from Q adds it to V – Q of vertices in the tree, adding (u, u.p) to A.
The for loop updates the key and p attributes of each vertex adjacent to u, but not in the tree, thus maintaining the third part of the loop invariant.

20. Minimum Spanning Trees
Prim’s Algorithm: Time
This will depend on the implementation of the priority queue. But;
Creating a priority queue is O(|V|).
Extracting elements is O(lg |V|) for each extraction.
Decrease-Key operations, necessary because the keys of elements in the queue can change, will also take time O(lg |V|). Since this can happen for each edge we examine, total time can be O(|E| lg |V|).

21. Minimum Spanning Tree: Greedy Algorithms
Invariant: Minimum weight spanning forest
Becomes single tree at end
Time: O(|E|lg|E|) given fast FIND-SET, UNION
Produces minimum weight tree of edges that includes every vertex. A B C D E F G 2 1 3 4 5 6 8 7
Time: O(|E|lg|V|) = O(|E|lg|E|) slightly faster with fast priority queue
Invariant: Minimum weight tree
Spans all vertices at end
for Undirected, Connected, Weighted Graph
G=(V,E)
source: textbook Cormen et al.

22. Minimum Spanning Trees
Review problem:
For the undirected, weighted graph below, show 2 different Minimum Spanning Trees. Draw each using one of the 2 graph copies below. Thicken an edge to make it part of a spanning tree. What is the sum of the edge weights for each of your Minimum Spanning Trees? A B C D E F G 2 1 3 4 5 6 8 7

23. Chapter 24 Shortest Paths
Dijkstra
[Source: Cormen et al. textbook except where noted]
I joined the UMass Lowell Computer Science faculty this summer. This collection of slides is intended to familiarize the reader/viewer with my field of research (Computational Geometry), summarize my previous research results in this field and outline my plan for Computational Geometry research at UMass Lowell.

24. BFS as a Basis for Shortest Path Algorithms
for unweighted, undirected graph G=(V,E)
Time: O(|V| + |E|) adj list
O(|V|2) adj matrix
Source/Sink Shortest Path
Problem: Given 2 vertices u, v, find the shortest path in G from u to v.
Solution: BFS starting at u. Stop at v.
Single-Source Shortest Paths
Problem: Given a vertex u, find the shortest path in G from u to each vertex.
Solution: BFS starting at u. Full BFS tree.
All-Pairs Shortest Paths
Problem: Find the shortest path in G from each vertex u to each vertex v.
Solution: For each u: BFS starting at u; full BFS tree.
Time: O(|V|(|V| + |E|)) adj list
O(|V|3) adj matrix
source: based on Sedgewick, Graph Algorithms

25. Shortest Path Applications
for weighted, directed graph G=(V,E)
Weight ~ Cost ~ Distance
Road maps
Airline routes
Telecommunications network routing
VLSI design routing
source: based on Sedgewick, Graph Algorithms

26. Shortest Path Trees
Shortest Path Tree gives shortest path from root to each other vertex
source: Sedgewick, Graph Algorithms

27. Shortest Path Principles: Relaxation
Graph Initialization
We start with a weighted, directed graph with a weight function w defined on its edges. We will generally consider a distinguished vertex, called the source, and usually denoted by s.

28. Shortest Path Principles: Relaxation
“Relax” a constraint to try to improve solution
“Rubber band” analogy [Sedgewick]
Relaxation of an Edge (u,v):
test if shortest path to v [found so far] can be improved by going through u A B C D E F G 2 1 3 4 5 6 8 7

29. Shortest Path Principles: Relaxation

30. Shortest Path Principles: Bellman-Ford

31. Dag-Shortest-Paths

32. Single Source Shortest Paths Dijkstra’s Algorithm
for (nonnegative) weighted, directed graph G=(V,E)
source: textbook Cormen et al.

33. Single Source Shortest Paths Dijkstra’s Algorithm
for (nonnegative) weighted, directed graph G=(V,E)
source: textbook Cormen et al.

34. Single Source Shortest Paths Dijkstra’s Algorithm
for (nonnegative) weighted, directed graph G=(V,E)
See separate ShortestPath slide show A B C D E F G 2 1 3 4 5 6 8 7
source: textbook Cormen et al.

35. Single Source Shortest Paths Dijkstra’s Algorithm
Review problem:
For the directed, weighted graph below, find the shortest path that begins at vertex A and ends at vertex F. List the vertices in the order that they appear on that path. What is the sum of the edge weights of that path? A B C D E F G 2 1 3 4 5 6 8 7
Why can’t Dijkstra’s algorithm handle negative-weight edges?

36. Single Source Shortest Paths Dijkstra’s Algorithm
for (nonnegative) weighted, directed graph G=(V,E)
source: Sedgewick, Graph Algorithms

37. Shortest Path Algorithms
source: Sedgewick, Graph Algorithms