1. Design and Analysis of Algorithms - Chapter 51 Decrease and Conquer 1. Reduce problem instance to smaller instance of the same problem 2. Solve smaller instance 3. Extend solution of smaller instance to obtain solution to original problem b Also referred to as inductive or incremental approach

2. Design and Analysis of Algorithms - Chapter 52 Examples of Decrease and Conquer b Decrease by one: Insertion sortInsertion sort Graph search algorithms:Graph search algorithms: –DFS, BFS, Topological sorting Algorithms for generating permutations, subsetsAlgorithms for generating permutations, subsets b Decrease by a constant factor Binary searchBinary search Fake-coin problemsFake-coin problems Multiplication à la russeMultiplication à la russe Josephus problemJosephus problem b Variable-size decrease Euclid’s algorithmEuclid’s algorithm Selection by partitionSelection by partition

3. Design and Analysis of Algorithms - Chapter 53 What’s the difference? Consider the problem of exponentiation: Compute a n b Brute Force: b Divide and conquer: b Decrease by one: b Decrease by constant factor:

4. Design and Analysis of Algorithms - Chapter 54 Insertion Sort Algorithm InsertionSort(A[0..n-1]) for i  1 to n-1 do v  A[i]; j  i-1; while j>=0 and A[j]>v do A[j+1]  A[j]; j  j-1 A[j+1]  v Improvements: sentinel, binary insertion, Shellsort Efficiency

5. Design and Analysis of Algorithms - Chapter 55 Graph Traversal b Many problems require processing all graph vertices in systematic fashion b Graph traversal algorithms: Depth-first searchDepth-first search Breadth-first searchBreadth-first search

6. Design and Analysis of Algorithms - Chapter 56 Depth-first search b Explore a graph G=(V,E) always moving away from last visited vertex b Similar to preorder tree traversals b Algorithm DFS(G) count :=0 mark each vertex with 0 (unvisited) for each vertex v ∈ V do if v is marked with 0 dfs(v) dfs(v) count  count + 1 mark v with count for each vertex w adjacent to v do if w is marked with 0 dfs(w)

7. Design and Analysis of Algorithms - Chapter 57 Example – undirected graph b Depth-first traversal: ab ef cd gh

8. Design and Analysis of Algorithms - Chapter 58 Types of edges in a DFS forest b Tree edges: edges comprising forest b Back edges: edges to ancestor nodes b Forward edges: edges to descendants (digraphs only) b Cross edges: none of the above

9. Design and Analysis of Algorithms - Chapter 59 Example – directed graph b Depth-first traversal: ab ef cd gh

10. Design and Analysis of Algorithms - Chapter 510 Depth-first search: Notes b DFS can be implemented with graphs represented as: Adjacency matrices: Θ(V 2 )Adjacency matrices: Θ(V 2 ) Adjacency linked lists: Θ(V+E)Adjacency linked lists: Θ(V+E) b Yields two distinct ordering of vertices: preorder: as vertices are first encountered (pushed onto stack)preorder: as vertices are first encountered (pushed onto stack) postorder: as vertices become dead-ends (popped off stack)postorder: as vertices become dead-ends (popped off stack) b Applications (Hopcroft-Tarjan): checking connectivity, finding connected componentschecking connectivity, finding connected components checking acyclicity, articulation pointschecking acyclicity, articulation points searching state-space of problems for solution (AI)searching state-space of problems for solution (AI)

11. Design and Analysis of Algorithms - Chapter 511 Breadth-first search b Explore graph moving across to all the neighbors of last visited vertex b Similar to level-by-level tree traversals b Instead of a stack, breadth-first uses queue b Applications: same as DFS, but can also find paths from a vertex to all other vertices with the smallest number of edges

12. Design and Analysis of Algorithms - Chapter 512 Breadth-first search algorithm bfs(v) count  count + 1 mark v with count initialize queue with v while queue is not empty do a  front of queue for each vertex w adjacent to a do if w is marked with 0 count  count + 1 mark w with count add w to the end of the queue remove a from the front of the queue Algorithm BFS(G) count  0 mark each vertex with 0 for each vertex v ∈ V do bfs(v)

13. Design and Analysis of Algorithms - Chapter 513 Example – undirected graph b Breadth-first traversal: ab ef cd gh

14. Design and Analysis of Algorithms - Chapter 514 Example – directed graph b Breadth-first traversal: ab ef cd gh

15. Design and Analysis of Algorithms - Chapter 515 Breadth-first search: Notes b BFS has same efficiency as DFS and can be implemented with graphs represented as: Adjacency matrices: Θ(V 2 )Adjacency matrices: Θ(V 2 ) Adjacency linked lists: Θ(V+E)Adjacency linked lists: Θ(V+E) b Yields single ordering of vertices (order added/ deleted from queue is the same)

16. Design and Analysis of Algorithms - Chapter 516 Directed acyclic graph (dag) b A directed graph with no cycles b Arise in modeling many problems, eg: prerequisite structureprerequisite structure food chainsfood chains b Imply partial ordering on the domain

17. Design and Analysis of Algorithms - Chapter 517 Topological sorting b Problem: find a total order consistent with a partial order fish human shrimp sheep wheatplankton tiger Order them so that they don’t have to wait for any of their food (i.e., from lower to higher, consistent with food chain) NB: problem is solvable iff graph is dag

18. Design and Analysis of Algorithms - Chapter 518 Topological sorting Algorithms 1. DFS-based algorithm: DFS traversal noting order vertices are popped off stackDFS traversal noting order vertices are popped off stack Reverse order solves topological sortingReverse order solves topological sorting Back edges encountered?→ NOT a dag!Back edges encountered?→ NOT a dag! 2. Source removal algorithm Repeatedly identify and remove a source vertex, ie, a vertex that has no incoming edgesRepeatedly identify and remove a source vertex, ie, a vertex that has no incoming edges b Both Θ(V+E) using adjacency linked lists

19. Design and Analysis of Algorithms - Chapter 519 Generating permutations Useful in exhaustive search algorithms Minimal-change requirement Given: a permutation for k-1 objects Method: Generate a new permutation by inserting the k-th object in all possible places, moving in one direction (e.g. left to right). Then, insert the (k+1)- th object by alternating the direction (right to left) Example

20. Design and Analysis of Algorithms - Chapter 520 Generating permutations – Johnson-Trotter Concept: use arrows to show the moving direction of an object. An object is called mobile if it points to a smaller object. No need to create smaller instances. Algorithm Johnson-Trotter Initialize the first permutation 1 2 3 … n while there exist a mobile integer k do find the largest mobile integer k swap k and the adjacent integer its arrow points to reverse the direction of all integers that are larger than k Example 123, 132, 312, 321, 231, 213 Efficiency

21. Design and Analysis of Algorithms - Chapter 521 Generating permutations The Johnson-Trotter algorithm does not produce permutations in lexicographical order Example 123, 132, 213, 231, 312, 321 Idea: start form the end and check a n-1 <a n. Then consider the a n-2 position and put the smallest out of the three digits, the remaining two in increasing order. Then backtrack.

22. Design and Analysis of Algorithms - Chapter 522 Generating subsets Useful in the knapsack problem (exhaustive solution). Power set : a set of 2 n sets (Decrease-by-one) Idea: create the powerset with {1,2,…, n-1} and then insert n in each set off them. Example:

23. Design and Analysis of Algorithms - Chapter 523 Generating subsets Use of bitstring to represent sets: 101 ≈ {a 1, a 3 } Example: for 3 objects, we have 8 sets represented with 000, 001, 010, …, 110, 111. Extensions b no lexicographical order in terms of objects; squashed order b minimal change order; Gray code; 000, 001, 011, 010, 110, 111, 101, 100

24. Design and Analysis of Algorithms - Chapter 524 Fake-coin problem b Among n identical looking coins, one is fake. Assume that the fake is lighter. b Divide the coins in two piles of n/2 coins (plus 1 if n is odd) b Recurrence relation: b What if we divide in three piles ?

25. Design and Analysis of Algorithms - Chapter 525 Multiplication ala russe b Or the Russian peasant method b Let n and m two positive integers. b If n is even, then : n. m = n/2. 2m b If n is odd, then : n. m = (n-1)/2. 2m + m b Example : 50. 65

26. Design and Analysis of Algorithms - Chapter 526 Josephus problem b Historic problem b N men are going to perish one by one, where each man dispatches his neighbor. Find the last one who survives. b Example for N = 6, 7 b Recurrence for the position of the survivor

27. Design and Analysis of Algorithms - Chapter 527 Variable-size-decrease: Computing a median b Order statistics: find the k-th number b One solution is to sort and pick the k-th b Better to partition ala quicksort b Example : 4, 1, 10, 9, 7, 12, 8, 2, 15 b Efficiency in best, worst and average case

28. Design and Analysis of Algorithms - Chapter 528 Interpolation search b Binary search is not used as panacea b Given a sorted array A, with A[l] the minimum, A[r] the maximum. Find the position of value v. b Solution : try at position x = l + (v-A[l])(r-l)/(A[r]-A[l]) b Efficiency in best, worst and average case

29. Design and Analysis of Algorithms - Chapter 529 Binary search trees b Arrange keys in a binary tree with the binary search tree property: k <k>k What about repeated keys? What about repeated keys? Example 1: Example 1: 5, 10, 3, 1, 7, 12, 9 Example 2: Example 2: 4, 5, 7, 2, 1, 3, 6

30. Design and Analysis of Algorithms - Chapter 530 Searching and insertion in binary search trees b Searching – straightforward b Insertion – search for key, insert at leaf where search terminated b All operations: worst case # key comparisons = h+1 b lgn ≤ h ≤ n–1 with average (random files) 1.41 lg n b Thus all operations have: worst case: Θ(n)worst case: Θ(n) average case: Θ(lgn)average case: Θ(lgn) b Bonus: inorder traversal produces sorted list (treesort)