1. Chap 5: Decrease & conquer

2. Objectives To introduce the decrease-and-conquer mind set To show a variety of decrease-and-conquer solutions: Depth-First Graph Traversal Breadth-First Graph Traversal Fake-Coin Problem Interpolation Search To discuss the strengths and weaknesses of a decrease-and-conquer strategy

3. Decrease and Conquer 1.Reduce problem instance to smaller instance of the same problem and extend solution 2.Solve smaller instance 3.Extend solution of smaller instance to obtain solution to original problem Also called inductive or incremental SUBPROBLEM OF SIZE n -1 A SOLUTION TO THE ORIGINAL PROBLEM A SOLUTION TO SUBPROBLEM A PROBLEM OF SIZE n

4. Divide-and-Conquer Illustrated SUBPROBLEM 2 OF SIZE n /2 SUBPROBLEM 1 OF SIZE n /2 A SOLUTION TO SUBPROBLEM 1 A SOLUTION TO THE ORIGINAL PROBLEM A SOLUTION TO SUBPROBLEM 2 A PROBLEM OF SIZE n

5. Flavours of Decrease and Conquer Decrease by a constant (usually 1): instance is reduced by the same constant on each iteration Insertion sort Graph Searching: DFS, BFS Generating combinatorials Decrease by a constant factor (usually 2): instance is reduced by same multiple on each iteration Binary search Fake-coin problem Variable-size decrease : size reduction pattern varies from one iteration to the next Euclid’s algorithm Interpolation Search

6. Flashback: Knapsack by Exhaustive Search Efficiency: Ω(2 n ) SubsetTotal Wgt Total Value {1} 2 kgR200 {2} 5 kgR300 {3} 10 kgR500 {4} 5 kgR100 {1,2} 7 kgR500 {1,3} 12 kgR700 {1,4} 7 kgR300 {2,3} 15 kgR800 SubsetTotal Wgt Total Value {2,4} 10 kgR400 {3,4} 15 kgR600 {1,2,3} 17 kgn/a {1,2,4} 12 kgR600 {1,3,4} 17 kgn/a {2,3,4} 20 kgn/a {1,2,3,4} 22 kgn/a

7. Flashback: Generating Subsets Combinatorics uses Decrease (by one) and Conquer Algorithms Generate all 2 n subsets of A = {a 1, …, a n } Divide into subsets of {a 1, …, a n-1 } that contain a n and those that don’t Sneaky Solution: establish a correspondence between bit strings and subsets. Bit n denotes presence (1) or absence (0) of element a n Generate numbers from 0 to 2n-1  convert to bit strings  interpret as subsets Examples: 000 = Ø, 010 = {a 2 }, 110 = {a 1, a 2 }

8. Flashback: Generating Permutations Generate all n! reorderings of {1, …, n} Generate all (n-1)! permutations of {1, …, n-1} Insert n into each possible position (starting from the right or left, alternately) Implemented by the Johnson-Trotter algorithm Satisfies Minimal-Change requirement Next permutation obtained by swapping two elements of previous Useful for updating style algorithms Example: Start: 1 Insert 2: 12 21 Insert 3: 123 132 312 321 231 213

9. Decrease and Conquer 1.Reduce problem instance to smaller instance of the same problem and extend solution 2.Solve smaller instance 3.Extend solution of smaller instance to obtain solution to original problem Also called inductive or incremental SUBPROBLEM OF SIZE n -1 A SOLUTION TO THE ORIGINAL PROBLEM A SOLUTION TO SUBPROBLEM A PROBLEM OF SIZE n

10. Exercise: Spot the Difference Problem: Derive algorithms for computing a n using: 1.Brute Force 2.Divide and conquer 3.Decrease by one 4.Decrease by constant factor (halve the problem size) Hint: each can be described in a single line

11. Exercise: Spot the Difference Problem: Derive algorithms for computing a n using: 1.Brute Force : a n = a*a*a*a*...*a n times 2.Divide and conquer : a n = ( a n /2 ) * (a n /2 ) 3.Decrease by one : a n = ( a n- 1 )* a 4.Decrease by constant factor (halve the problem size) : a n = ( a n /2 ) 2

12. Graph Traversal Many problems require processing all graph vertices in a systematic fashion Data Structures Reminder: Graph traversal strategies: Depth-first search (traversal for the Brave) Breadth-first search (traversal for the Cautious) ab cd abcd a0111 b1001 c1001 d1110 a b c d a bc d d ad ab c

13. Depth-First Search Explore graph always moving away from last visited vertex Similar to preorder tree traversal DFS(G): G = (V, E) count  0 mark each vertex as 0 for each vertex v ∈ V if v is marked as 0 dfs(v) dfs(v): count  count + 1 mark v with count for each vertex w adjacent to v DO if w is marked as 0 dfs(w)

14. Example: DFS ab ef cd gh Traversal Stack: (pre = push, post = pop) 1 a 8 2 b 7 3 f 2 4 e 1 5 g 6 6 c 5 7 d 4 8 h 3 Push order: a b f e g c d h Pop order:e f h d c g b a

15. DFS Forest DFS Forest: A graph representing the traversal structure Types of Edges: Tree edges: edge to next vertex in traversal Back edges: edge in graph to ancestor nodes Forward edges: edge in graph to descendants Cross edges: none of the above ab ef cd gh a bef cdgh Backedge

16. Notes on Depth-First Search Applicable to different graph structures: Adjacency matrices:  (V 2 ) Adjacency linked lists:  (V+E) Yields two orderings: preorder: as vertices are 1 st encountered (pushed) postorder: as vertices become dead-ends (popped) Applications: Checking connectivity, finding connected components Checking a-cyclicity Searching state-space of problems for solution (AI)

17. Breadth-First Search Move across to all neighbours of the last visited vertex Similar to level-by-level tree traversals Instead of a stack, breadth- first uses a queue BFS(G): G = (V, E) count  0 mark each vertex as 0 for each vertex v ∈ V if v is marked as 0 bfs(v) bfs(v): count  count + 1 mark v with count initialize queue with v while queue not empty DO a  front of queue for each vertex w adjacent to a if w is marked as 0 count  count + 1 mark w with count add w to queue remove a from queue

18. Example: BFS ab ef cd gh Traversal Queue: a 1 b 2 e 3 f 4 g 5 c 6 h 7 d 8 Order: a b e f g c h d a b e f c d g h Crossedge

19. Notes on Breadth First Search BFS has same efficiency as DFS and can be implemented with: Adjacency matrices:  (V 2 ) Adjacency linked lists:  (V+E) Yields single ordering of vertices Applications: Same as DFS, but can also find paths from a vertex to all other vertices with the smallest number of edges

20. Decrease and Conquer 1.Reduce problem instance to smaller instance of the same problem and extend solution 2.Solve smaller instance 3.Extend solution of smaller instance to obtain solution to original problem Also called inductive or incremental SUBPROBLEM of smaller size A SOLUTION TO THE ORIGINAL PROBLEM A SOLUTION TO SUBPROBLEM A PROBLEM OF SIZE n

21. BFS(G): G = (V, E) count  0 mark each vertex as 0 for each vertex v ∈ V if v is marked as 0 bfs(v) bfs(v): count  count + 1 mark v with count initialize queue with v while queue not empty DO a  front of queue for each vertex w adjacent to a if w is marked as 0 count  count + 1 mark w with count add w to queue remove a from queue DFS(G): G = (V, E) count  0 mark each vertex as 0 for each vertex v ∈ V if v is marked as 0 dfs(v)

22. bfs(v): count  count + 1 mark v with count initialize queue with v while queue not empty DO a  front of queue for each vertex w adjacent to a if w is marked as 0 count  count + 1 mark w with count add w to queue remove a from queue dfs(v): count  count + 1 mark v with count for each vertex w adjacent to v DO if w is marked as 0 dfs(w)

23. Example: BFS ab ef cd gh Traversal Queue: a 1 b 2 e 3 f 4 g 5 c 6 h 7 d 8 Order: a b e f g c h d a b e f c d g h Crossedge

24. Example: BFS ab ef cd gh Traversal Queue: a 1 b 2 e 3 f 4 g 5 c 6 h 7 d 8 Order: a b e f g c h d a b e f c d g h Crossedge

25. DFS Forest DFS Forest: A graph representing the traversal structure Types of Edges: Tree edges: edge to next vertex in traversal Back edges: edge in graph to ancestor nodes Forward edges: edge in graph to descendants Cross edges: none of the above ab ef cd gh a bef cdgh Backedge

26. Objectives To introduce the decrease-and-conquer mind set To show a variety of decrease-and-conquer solutions: Depth-First Graph Traversal Breadth-First Graph Traversal Fake-Coin Problem Interpolation Search To discuss the strengths and weaknesses of a decrease-and-conquer strategy

27. The Fake-Coin Problem: Decrease by a Constant Factor Problem: Among n identical looking coins, one is a fake (and weighs less) We have a balance scale which can compare any two sets of coins Algorithm: Divide into two size  n/2  piles (keeping a coin aside if n is odd) If they weigh the same then the extra coin is fake Otherwise proceed recursively with the lighter pile Efficiency: W(n) = W(  n/2  ) + 1 for n > 1 W(n) =  log 2 n  =  (log 2 n) But there is a better  (log 3 n) algorithm

28. Euclid’s GCD: Variable-Size Decrease Problem: Greatest Common Divisor of two integers m and n is the largest integer that divides both exactly Alternative Solutions: Consecutive integer checking (brute force) Identify common prime factors (transform and conquer) Euclid’s Solution: gcd(m, n) = gcd(n, m mod n) gcd(m, 0) = m Right-side args are smaller by neither a constant size nor factor Example: gcd(60, 24) = gcd(24, 12) = gcd(12, 0) = 12

29. Interpolation Search: Variable-Size Decrease Mimics the way humans search through a phone book (look near the beginning for ‘Brown’) Assumes that values between the leftmost (A[l]) and rightmost (A[r]) elements increase linearly Algorithm (key = v, find search index = i): Binary search with floating variable at index i Setup straight line through (l, A[l]) and (r, A[r]) Find point P = (x, y) on line at y = v, then i = x Efficiency: Average =  (log log n + 1) Worst =  (n) lir A[l] v A[r] index value

30. Strengths and Weaknesses of Decrease-and-Conquer Strengths: Can be implemented either top down (recursively) or bottom up (without recursion) Often very efficient (possibly  (log n) ) Leads to a powerful form of graph traversal (Breadth and Depth First Search) 7Weaknesses: Less widely applicable (especially decrease by a constant factor)