1. PROBLEM-SOLVING TECHNIQUES Rocky K. C. Chang November 10, 2015

2. Objectives Understand and be able to use several approaches to solve problems: Forward or backward reasoning Greedy or not greedy Dynamic programming Divide and conquer approach A binary approach

3. (Computer) Algorithms Mr. Webster said, “an algorithm is a procedure for solving a mathematical problem in a finite number of steps that frequently involves repetition of an operation; Broadly: a step-by-step procedure for solving a problem or accomplishing some end especially by a computer.” The number of steps may be finite (e.g., gcd) or infinite (e.g., the  computation).

4. What have you seen or will see? Involving numbers GCD  computation The rabbit population problem Birthday cake cutting Permutation and combinatorial problems Basketball score Optimization problems X-to-Y problem Rock-moving game Coin changing problems (new)

5. What have you seen or will see? Games Tower of Hanoi An integer-calling game A coin collection game (new) Involving a list of data Find the kth largest item in a list. (new)

6. Correctness and performance An algorithm is correct if it always gives the expected solution to a problem. A proof for correctness A proof for incorrectness A problem but more than one solution Which algorithm is better and why? Why do we care?

7. Forward and backward reasoning Forward reasoning: Start with what we know and advance toward the result to be obtained. Forward induction: Given that the smaller problem can be solved, what can we say about the solution to a larger problem? Backward reasoning: Start with the result to be obtained and advance toward what we know. Backward induction: Given that a larger problem can be solved, what can we say about the solution to a smaller problem?

8. For example, X-to-Y problem (Backward) Start from Y and reason about the value before reaching Y. (Forward) We will see how it can be done. Basketball score (Backward) Start from the given score and consider all the possible states reaching this score. (Forward) It can be done Tower of Hanoi (Backward) Start from the last couple steps and go backward to the starting point of the game (Forward) It can be done. Both approaches can be based on a recursive definition but with an iterative implementation.

9. Greedy or not greedy A greedy algorithm is one that always chooses the best option in each step. E.g., a forward-greedy approach to the X  Y problem. Use “  2” whenever possible. Not always the best solution, e.g., X = 3 and Y = 11.

10. Greedy is not necessarily bad! A backward “greedy” approach to the X  Y problem A base case: 2X > Y An inductive step partly based on a greedy approach

11. An electrician problem An electrician connects a white dot and a black dot using a wire. Given N white dots and N black dots, how to connect them using the least amount of wires? Assume that the wire connecting two adjacent dots is 1. For example: Design a greedy algorithm for this problem. Does this algorithm always give the correct solution?

12. When to be greedy? Usually involve in finding a max/min result in a problem, e.g., Minimum number of operations Minimum number of coins Shortest path from point A to point B. A few algorithms for graph problems are based on greedy approaches, e.g., Dijkstra’s shortest path algorithm Kruskal’s maximum spanning tree algorithm

13. Dynamic programming Dynamic programming is a method for solving a complex problem by breaking it down into a collection of simpler subproblems, solving each just once, and storing their solutions. It can be used to solve optimization problems, e.g., the X  Y problem and coin changing problem, and combinatorial problems, such as the basketball scores problem and the different combinations of the coins. Some shortest-path algorithms are also based on dynamic programming, such as the Warshall’s algorithm.

14. DP: Fibonacci numbers Use dynamic programming to find the Fibonacci numbers. Modify your code for Q1 of A7 by Using a list to store Fib(0), Fib(1), …, Fib(n) Initializing Fib(0) = 0 and Fib(1) = 1, and Calculating Fib(i), i > 1. Use a time module to compute the total time for computing Fib(n) by adding the following codes: start_time = time.time() print("--- %s seconds ---" % (time.time() - start_time))

15. DP: The X  Y problem Let M be a list containing the solutions to the problem of making X to X, X+1, X+2,..., Y. M[y] is the minimum no. of operations to make X equal to y, where y = X, X+1, X+2,..., Y. Therefore, the solution for this problem is given by M[Y]. The idea is that we can obtain M[Y] from the solutions to M[y], y < Y, by updating their optimal solutions.

16. DP: The X  Y problem The algorithm: Initialization: M[X] = 0, M[i] = Y, i = X+1,..., Y (just big enough) for i from X to Y-1 do M[i+1]  min{M[i]+1, M[i+1]} if 2i <= Y, then M[2i]  M[i] + 1 end for Implement it in Python.

17. DP: A coin-changing problem For a currency with coins c 1, c 2,..., c N (cents), what is the minimum number of coins needed to make x cents of change? For example, assume that we have 1-cent, 5-cent, 10- cent, and 25-cent coins. For x = 63 cents, the minimum number of coins is 6: 2 25-cent coins, 1 10-cent coin, and $3 1-cent coins. Does a forward greedy approach always work and why? Solve this problem using backward reasoning, i.e., recursion. Solve this problem using dynamic programming.

18. Divide and conquer approach Solve the original problem by breaking it down into a smaller instances of the same problem. Lend it to a natural recursive implementation. Examples: GCD: a single smaller problem Tower of Hanoi: a single smaller problem but 2 instances Binary research: reduce the problem size by a half each time. Mergesort: break it into 2 smaller problems each time.

19. A binary approach Solve a problem by Binary complement, Binary representation of numbers, Bit-by-bit processing, Parity considerations, and The reduction of an integer task into a 0/1 task.

20. The memoryless game Observation 1: A response call should be some function of the dealer’s call. The dealer’s call is only information we have. Would a simple digit reversing work, e.g., 123  321? Observation 2: A binary complement approach may be effective. 123 = 01111011  10000100 = 132 Work?

21. The memoryless game (cont’d) Observation 3: Apply a different notion of complement, such as complement of the distance. The distance of an integer i from 0 is i, and its complementing distance is N – i. E.g., N = 299, 123  299 – 123 = 176. Observation 4: The notion of complement can be applied to the least significant bit only. E.g., 123 = 01111011  01111010 = 122. E.g., 124 = 01111100  01111101 = 125. If i is even, i  i + 1; else, i  i – 1.

22. The X  Y problem revisited Invented by George Lai, a double degree student Express the values of X and Y in binary. Example 1: 12  50 1100  110010 The first 4 leftmost digits are the same. After a  2 operation, 1100 becomes 11000. After a +1 operation, 11000 becomes 11001. After a  2 operation, 11001 becomes 110010. Therefore ((12  2)+1)  2 = 50 (3 operations)

23. The X  Y problem revisited Example 2: 8  42 1000  101010 The first 4 leftmost digits are not the same. After a +1 operation, 1000 becomes 1001. After a +1 operation, 1001 becomes 1010. Now the first 4 leftmost digits are the same. After a  2 operation, 1010 becomes 10100. After a +1 operation, 10100 becomes 10101. After a  2 operation, 10101 becomes 101010. Therefore ((8+1+1)  2+1)  2 = 42 (5 operations)

24. The X  Y problem revisited Example 2: 10  35 1010  100011 The first 4 leftmost digits are not the same. After a +1 operation, 1010 becomes 1011. After a +1 operation, 1011 becomes 1100. After a +1 operation, 1100 becomes 1101. After a +1 operation, 1101 becomes 1110. After a +1 operation, 1110 becomes 1111. After a +1 operation, 1111 becomes 10000. After a +1 operation, 1111 becomes 10001. Now the first 5 leftmost digits are the same. After a  2 operation, 10001 becomes 100010. After a +1 operation, 100010 becomes 100011. Therefore (10+1+1+1+1+1+1+1)  2+1 = 35 (9 operations).

25. A coin collection game Given a line of coins, 2 players remove the coins and collect them. The set of coins is, e.g., {1, 5, 10, 25}. Each player in turn removes and collects a coin from either end of the line. The game ends when all the coins are removed and collected by the 2 players. The winner is the one who has collected the larger total value.

26. An example The line of coins: 10 25 1 25 1 5

27. A coin collection game Let N be the number of coins, which is even. Let the total value of the coins is odd. Is it a game of pure chance or is there a sure-win strategy? If it is the latter, do both players have sure-win strategies or just one of them? Develop a winning strategy for the player who starts the game.

28. Would a greedy approach work here?

29. How to solve using a forward reasoning approach?

30. How to solve using a backward reasoning approach?

31. How to solve using a binary approach?

32. Let’s denote the locations of the coins by 0 and 1, e.g., 0 1 0 1 0 1 0 1 for N = 8. Observation 1: The first 2 coins are removed from different parity locations. If player 1 chooses 0, then player 2 chooses 0, and vice versa. This holds for 3 rd and 4 th, 5 th and 6 th, and so forth. Observation 2: After an even number of turns, one end is 0 and the other 1. Observation 3: The first player always plays after an even number of turns.

33. Summary It is interesting to see that there are many ways to solve a problem. However, some of the most intuitive solution may not always give the correct solutions. There are various perspectives in solving a problem. Forward vs. backward reasoning Greedy vs. nongreedy Dynamic programming for solving optimization problems Divide and conquer Binary vs nonbinary Problem solving is both a science and an art!