Chapter 5: Decrease and Conquer The Design and Analysis of Algorithms Chapter 5: Decrease and Conquer

Chapter 5. Decrease and Conquer Algorithms Basic Idea Decrease by a Constant (usually one) Insertion sort, graph search Permutations, subsets Decrease by a Constant Factor Fake-coin problem, Multiplication a la Russe Variable Size Decrease Conclusion

Basic Idea Reduce problem instance to smaller instance of the same problem and extend solution Solve smaller instance Extend solution of smaller instance to obtain solution to original problem

Basic Idea

Examples of Decrease-and-Conquer Algorithms Decrease by one: Insertion sort Graph search algorithms: DFS BFS Topological sorting Algorithms for generating permutations, subsets

Examples of Decrease-and-Conquer Algorithms Decrease by one: Insertion sort Graph search algorithms: DFS BFS Topological sorting Algorithms for generating permutations, subsets

Examples of Decrease-and-Conquer Algorithms Decrease by a constant factor Binary search Fake-coin problems Multiplication à la russe Josephus problem Variable-size decrease Euclid’s algorithm Selection by partition

Decrease by One: Insertion Sort Algorithm to sort n elements: Sort n-1 elements of the array Insert the n-th element Complexity: (n2) in the worst and the average case, and (n) on almost sorted arrays.

Decrease by One: Graph Search Depth-first search algorithm: dfs(v) process(v) mark v as visited for all vertices i adjacent to v not visited dfs(i) Complexity: Θ(V2) if represented by an adjacency table, Θ(|V| + |E|) if represented by adjacency lists

Graph Search: DFS

Graph Search: BFS Breadth-first search algorithm: procedure bfs(v) q := make_queue() enqueue(q,v) mark v as visited while q is not empty v = dequeue(q) process v for all unvisited vertices v' adjacent to v mark v' as visited enqueue(q,v')

Graph Search: BFS

Graph Search Complexity of DFS and BFS: Θ(V2) if represented by an adjacency table, Θ(|V| + |E|) if represented by adjacency lists. Various applications in AI problems and graph problems (e.g. finding cycles, articulation points)

Permutations of Size n Generating permutations of size n find all permutations of size n-1 of elements a1, a2, .., a n-1 construct permutations of n elements as: append an to each permutation of size n-1 for each permutation of size n-1 for k from 1 to n-1 insert an in front of ak

Johnson-Trotter Algorithm The straight-forward implementation is very inefficient since it obtains all lower-level permutations (permutations of size less than n). Trotter (1962) and Johnson (1963): an algorithm to obtain all permutations of size n without going through shorter permutations. Directed integers. Mobile integer: greater than the integer it points to

Johnson-Trotter Algorithm Initialize the first permutation with <1 <2 ... <n while the last permutation has a mobile integer do find the largest mobile integer k swap k and the adjacent integer it is looking at reverse the direction of all integers larger than k

Johnson-Trotter Algorithm Example: <1 <2 <3 <4 largest mobile element is 4 <1 <2 <4 <3 swap 3 and 4; largest mobile element is 4 <1 <4 <2 <3 swap 2 and 4; largest mobile element is 4 <4 <1 <2 <3 swap 1 and 4; largest mobile element is 3 4> <1 <3 <2 swap 2 and 3; change direction of all greater than 3; largest mobile element is 4 <1 4> <3 <2 swap 1 and 4; largest mobile element is 4 <1 <3 4> <2 swap 3 and 4; largest mobile element is 4 <1 <3 <2 4> swap 2 and 4; largest mobile element is 3 . . . . . . . . <2 <1 3> 4>. “minimal-change” algorithm; runs in (N!) time

Subsets and Gray Codes Let Sn-1 be the set of all subsets of n-1 elements, Sn-1 = {A1, A2, … Am}, m = 2n-1 Sn = {A1, A2, … Am, A1  an , A2  an , … Am  an } No need to generate all power sets of smaller sets. Frank Gray (1953): a minimal-change algorithm for generating all binary sequences of length n - “binary reflected Gray code”.

Subsets and Gray Codes base reflect prepend reflect prepend 0 0 00 00 000 1 1 01 01 001 1 11 11 011 0 10 10 010 10 110 11 111 01 101 00 100 The complexity is (2n)

Decrease by a Constant Factor The Fake-coin problem Multiplication a la Russe Usually logarithmic in complexity

The Fake Coin Problem You have 27 coins among which one is fake and it is lighter than the others. You have also a balance scale and you can compare the weight of any two sets of coins. How can you find the fake coin with three measurements only? How many measurements if the number of coins is N?

Multiplications a la Russe We can multiply two positive integers using only addition and division by 2. The algorithm is based on the observation that N*M = (N/2) * (M * 2) if N is even, and N*M = ((N-1)/2) * (M*2) + M if N is odd The base case is N = 1: 1*M = M

Multiplications a la Russe

Variable Size Decrease The reduction pattern varies from one iteration of the algorithm to another. Examples are Euclid’s algorithm for computing the greatest common divisor , Search and Insertion in binary search trees, and others.

Conclusion Very efficient solutions. The algorithms exploit a relation between the instance of the problem and a smaller instance of the same problem Recursive in nature Implementation: recursion or iteration