Decrease and Conquer Decrease and conquer technique Insertion sort DFS and BFS Topological sort

Expected Outcomes Students should be able to Explain the idea and steps of decrease and conquer Explain the ideas of insertion sort, DFS, BFS, topological sort Analyze the time complexity of the above algorithms

Decrease and Conquer Exploring the relationship between a solution to a given instance of (e.g., P(n) )a problem and a solution to a smaller instance (e.g., P(n/2) or P(n) )of the same problem. Use top down(recursive) or bottom up (iterative) to solve the problem. Example, n! A top down (recursive) solution A bottom up (iterative) solution

Examples of Decrease and Conquer Decrease by a constant: the size of the problem is reduced by the same constant on each iteration/recursion of the algorithm. Insertion sort Graph search algorithms: DFS BFS Algorithms for generating permutations, subsets Decrease by a constant factor: the size of the problem is reduced by the same constant factor on each iteration/recursion of the algorithm. Binary search Fake-coin problems Variable-size decrease: the size reduction pattern varies from one iteration of an algorithm to another. Eucd’s algorithm

A Typical Decrease by One Technique subproblem of size n-1 a solution to the a solution to the original problem a problem of size n e.g., n!

A Typical Decrease by a Constant Factor (half) Technique subproblem of size n/2 a solution to the a solution to the original problem a problem of size n e.g., Binary search

A Typical Divide and Conquer Technique subproblem 2 of size n/2 subproblem 1 a solution to the original problem a problem of size n e.g., mergesort

What’s the Difference? Consider the problem of exponentiation: Compute an Decrease by one Bottom-up: iterative (brute Force) Top-down:recursive Divide and conquer: Decrease by a constant factor: an= a*a*a*a*...*a an= an* a if n > 1 = a if n = 1 an= a  n/2  * a  n/2  if n > 1 = a if n = 1 an = (an/2 ) 2 if n is even and positive = (a(n)/2 ) 2 * a if n is odd and > 1 = a if n = 1

Insertion Sort A decrease by one algorithm A bottom-up (iterative) solution A top-down (recursive) solution

Insertion Sort: an Iterative Solution ALGORITHM InsertionSortIter(A[0..n]) //An iterative implementation of insertion sort //Input: An array A[0..n] of n orderable elements //Output: Array A[0..n] sorted in nondecreasing order for i  1 to n – 1 do // i: the index of the first element of the unsorted part. key = A[i] for j  i – 1 to 0 do // j: the index of the sorted part of the array if (key< A[j]) //compare the key with each element in the sorted part A[j+1]  A[j] A[j +1]  key //insert the key to the sorted part of the array Efficiency?

Insertion Sort: a Recursive Solution ALGORITHM InsertionSortRecur(A[0..n], n) //A recursive implementation of insertion sort //Input: An array A[0..n] of n orderable elements //Output: Array A[0..n] sorted in nondecreasing order if n > 1 InsertionSortRecur(A[0..n-2], n-2) Insert(A[0..n-2], n) //insert A[n] to A[0..n-2] ALGORITHM Insert(A[0..m], m) //Insert A[m] to the sorted subarray A[0..m] of A[0..n] //Input: A subarray A[0..m] of A[0..n] //Output: Array A[0..m] sorted in nondecreasing order key = A[m] for j  m – 1 to 0 do if (key< A[j]) A[j+1]  A[j] else break A[j +1]  key Index of the last element Index of the element/key to be inserted. Recurrence relation?

Exercises Design a recursive decrease-by-one algorithm for finding the position of the largest element in an array of n real numbers. Determine the time efficiency of this algorithm and compare it with that of the brute-force algorithm for the same problem.

Graph Traversal Graph traversal algorithms: Many problems require processing all graph vertices in a systematic fashion Graph traversal algorithms: Depth-first search Breadth-first search

Depth-first search The idea Similar to preorder tree traversals traverse “deeper” whenever possible. On each iteration, the algorithm proceeds to an unvisited vertex that is adjacent to the one it is currently in. If there are more than one neighbors, break the tie by the alphabetic order of the vertices. When reaching a dead end ( a vertex with no adjacent unvisited vertices), the algorithm backs up one edge to the parent and tries to continue visiting unvisited vertices from there. The algorithm halts after backing up to the starting vertex, with the latter being a dead end. Similar to preorder tree traversals It’s convenient to use a stack to trace the operation of depth-first search. Push a vertex onto the stack when the vertex is reached for the first time. Pop a vertex off the stack when it becomes a dead end.

Example – undirected graphs b e f c d g h Traversal using alphabetical order of vertices. Work through this example in detail, showing the traversal by highghting edges on the graph and showing how the stack evolves: 8 h 3 7 d 4 stack shown growing upwards 4 e 1 6 c 5 number on left is order that vertex was pushed onto stack 3 f 2 5 g 6 number on right is order that vertex was popped from stack 2 b 7 overlap is because after e, f are popped off stack, g and c 1 a 8 are pushed onto stack in their former locations. order pushed onto stack: a b f e g c d h order popped from stack: e f h d c g b a * show dfs tree as it gets constructed, back edges Depth-first traversal:Give the order in which the vertices were reached. a b f e g c d h

Depth-first search (DFS) Pseudocode for Depth-first-search of graph G=(V,E) DFS(G) // Implements a DFS traversal of a given Graph // Input: Graph G = {V, E} // Output: Graph G with its vertices marked with consecutive integers in the order they’ve been first encounter by the DFS traversal mark each vertex in V with 0 as a mark of being “unvisited” count  0 //visiting sequence number for each vertex v∈ V do if v is marked with 0 //w has not been visited yet. dfs(v) dfs(v) //Use depth-first to visit a connected component starting from vertex v. count  count + 1 mark v with count //visit vertex v for each vertex w in V adjacent to v do if w is marked with 0 //w has not been visited yet. dfs(w)

Another Algorithm: Four Arrays for DFS Algorithm color[u]: the color of each vertex visited (The state of a vertex, u, is stored in a color variable as follows:) Color[u] = WHITE means undiscovered Color[u] = GRAY means discovered but not finished processing Color[u] = BLACK means finished processing. [u]: the predecessor of u, indicating the vertex from which u is discovered. (depth-first search uses π[v] to record the parent of vertex v. We have π[v] = NIL if and only if vertex v is the root of a depth-first tree.) d[u]: discovery time, a counter indicating when vertex u is discovered. (When vertex v is changed from white to gray the time is recorded in d[v]. ) f[u]: finishing time, a counter indicating when the processing of vertex u (and the processing of all its descendants ) is finished. (When vertex v is changed from gray to black the time is recorded in f[v].)

DFS Algorithm DFS (V, E) 1. for each vertex u in V[G] 2. do color[u] ← WHITE 3. π[u] ← NIL 4. time ← 0 5. for each vertex u in V[G] 6. do if color[u] ← WHITE 7. then DFS-Visit(u) ▷ build a new DFS-tree from u

DFS Algorithm (Continued) DFS-Visit(u) 1. color[u] ← GRAY ▷ discover u 2. time ← time + 1 3. d[u] ← time 4. for each vertex v adjacent to u ▷ explore (u, v) 5. do if color[v] ← WHITE 6. then π[v] ← u 7. DFS-Visit(v) 8. color[u] ← BLACK 9. time ← time + 1 10. f[u] ← time ▷ we are done with u

DFS Algorithm (Example) 1/ a d b c g e f

Example(continued) 1/ a d 2/ b c g e f

Example(continued) 1/ a d 2/ b 3/ c g e f

Example(Continued) 1/ a d 2/ b 3/ c g e f 4/

Example(Continued) 1/ a d 2/ b 3/ c g e f 4/5

Example(Continued) 1/ a d 2/ b 3/6 c g e f 4/5

Example(Continued) 1/ a d 2/7 b 3/6 c g e f 4/5

Example(Continued) 1/8 a d 2/7 b 3/6 c g e f 4/5

Example(Continued) 1/8 a 9/ d 2/7 b 3/6 c g e f 4/5

Example(Continued) 1/8 a 9/ d 2/7 b 3/6 c g e 10/ f 4/5

Example(Continued) 1/8 a 9/ d 2/7 b 3/6 c 10/11 g e f 4/5

Example(continued) 1/8 a 9/ d 2/7 b 3/6 12/13 c g e 10/11 f 4/5

Example(Continued) a b f e c g d 2/7 1/8 3/6 4/5 9/14 10/11 12/13 

What does DFS do? Given a graph G, it traverses all vertices of G and Constructed a collection of rooted trees together with a set of source vertices (roots) Outputs two arrays d[ ]/f[ ] Note: forest is stored in array [ ], the [ ] value of a root is null. DFS forest: DFS creates a forest F = (V, Ef), where Ef = {([u], u)| where [ ] is computed in the DFS-VISIT calls}

Properties of DFS Definition: in a rooted forest, on the simple path from the root to each vertex, a vertex u is called ancestor of all the vertices following it, and descendant of all the vertices preceding it. Note: u = [v] if and only if DFS-VISIT(v) was called during a search of u’s adjacency st. u is called the parent of v. So: vertex v is a descendant of vertex u in the depth-first forest if and only if v is discovered during the time in which u is gray.

Example a b f e c g d 2/7 1/8 3/6 4/5 9/14 10/11 12/13 

Properties of DFS Parenthesis Theorem: In any DFS of a graph, for each pair of vertices u, v, exactly one of the following conditions holds: u is a descendant of v, and [d[u], f[u]] is a subinterval of [d[v], f[v]]. u is an ancestor of v, and [d[v], f[v]] is a subinterval of [d[u], f[u]]. Neither u is a descendant of v nor v is a descendant of u, and [d[u], f[u]] and [d[v], f[v]] are disjoint. The nth Catalan Number:

Nesting of descendants’ intervals Corollary: vertex v is a proper descendant of vertex u in the depth-first forest for a graph G if and only if d[u]< d[v] < f[v] < f[u].

White-path theorem Theorem: In a depth-first forest of a graph G = (V, E), vertex v is a descendant of u if and only if at time d[u], vertex v can be reached from u along a path consisting entirely of white vertices.

Example – directed graph b c d e f g h Depth-first traversal: Give the order in which the vertices were reached. a b f g h e c d

Depth-first search: Notes DFS can be implemented with graphs represented as: Adjacency matrices: Θ(|V |2) Adjacency linked lists: Θ(|V |+|E| )

Applications of DFS Find the strongly connected components of a directed graph. Find the articulation points of an undirected graph Topological sort of a directed graph Classification of edges. Verify if an undirected graph is connected or not. Verify if a directed graph is semi-connected or not. Verify if a directed graph is singly connected or not. ……

Breadth-first search (BFS) Archetype for Prim’s minimum-spanning-tree algorithm Archetype for Dijkstra’s single-source shortest-paths algorithm The idea Traverse “wider” whenever possible. Discover all vertices at distance k from s (on level k) before discovering any vertices at distance k +1 (at level k+1) Similar to level-by-level tree traversals Instead of a stack, breadth-first uses a queue.

Example – undirected graph b e f c d g h Breadth-first traversal: a b e f g c h d

Breadth-first search algorithm BFS(G) count  0 mark each vertex with 0 for each vertex v∈ V do if v is marked with 0 bfs(v) bfs(v) count  count + 1 mark v with count //visit v initiaze queue with v //enqueue while queue is not empty do a  front of queue //dequeue for each vertex w adjacent to a do if w is marked with 0 //w hasn’t been visited. mark w with count //visit w add w to the end of the queue //enqueue remove a from the front of the queue

Example – directed graph b c d e f g h Breadth-first traversal: a b e f g h c d

Another Implementation: The Breadth-First Search The idea of the BFS: Visit the vertices as follows: Visit all vertices directly connected to starting vertex Visit all vertices that are two edges away from the starting vertex Visit all vertices that are three edges away from the starting vertex … … Initially, s is made GRAY, others are colored WHITE. When a gray vertex is processed, its color is changed to black, and the color of all white neighbors is changed to gray. Gray vertices are kept in a queue Q.

What does the BFS do? Given a graph G = (V, E), the BFS returns: The shortest distance d[v] from s to v The predecessor or parent [v], which is used to derive a shortest path from s to vertex v. In addition to the two arrays d[v] and [v], BFS also uses another array color[v], which has three possible values: WHITE: represented “undiscovered” vertices; GRAY: represented “discovered” but not “processed” vertices; BLACK: represented “processed” vertices.

The Breadth-First Search (more details) G is given by its adjacency-sts. Initiazation: First Part: nes 1 – 4 Second Part: nes 5 - 9 Main Part: nes 10 – 18 Enqueue(Q, v): add a vertex v to the end of the queue Q Dequeue(Q): Extract the first vertex in the queue Q

Breadth-first search: Notes BFS has same efficiency as DFS and can be implemented with graphs represented as: Adjacency matrices: Θ(|V |2) Adjacency linked lists: Θ(|V |+|E| ) Applications: checking connectivity finding connected components finding paths between two vertices with the smallest number of edges Use a tree.

Is the problem solvable if the digraph contains a cycle? Topological Sorting Problem: Given a Directed Acycc Graph(DAG) G = (V, E), find a near ordering of all its vertices such that if G contains an edge (u, v), then u appears before v in the ordering. Example: Give an order of the courses so that the prerequisites are met. C4 C1 C3 Is the problem solvable if the digraph contains a cycle? C5 C2

The DFS-Based Algorithm DFS traversal noting the order in which vertices are popped off stack (the order in which the dead end vertices appear) Reverse the above order. Questions Can we use the order in which vertices are pushed onto the DFS stack (instead of the order they are popped off it) to solve the topological sorting problem? Does it matter from which node we start? Θ(|V |+|E| ) using adjacency nked sts

The Source Removal Algorithms Based on a direct implementation of the decrease-by-one techniques. Repeatedly identify and remove a source vertex, ie, a vertex that has no incoming edges, and delete it along with all the edges outgoing from it.