1 Decrease-and-Conquer Approach Lecture 06 ITS033 – Programming & Algorithms Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program, Image and Vision Computing Lab. School of Information, Computer and Communication Technology (ICT) Sirindhorn International Institute of Technology (SIIT) Thammasat University X 2005

2 ITS033 Topic 01-Problems & Algorithmic Problem Solving Topic 01 - Problems & Algorithmic Problem Solving Topic 02 – Algorithm Representation & Efficiency Analysis Topic 03 - State Space of a problem Topic 04 - Brute Force Algorithm Topic 05 - Divide and Conquer Topic 06-Decrease and Conquer Topic 06 - Decrease and Conquer Topic 07 - Dynamics Programming Topic 08-Transform and Conquer Topic 08 - Transform and Conquer Topic 09 - Graph Algorithms Topic 10 - Minimum Spanning Tree Topic 11 - Shortest Path Problem Topic 12 - Coping with the Limitations of Algorithms Power and

3 This Week Overview Problem size reduction Problem size reduction Insertion Sort Insertion Sort Recursive programming Recursive programming Examples Examples  Factorial  Tower of Hanoi

4 Decrease & Conquer: Concept Lecture 06.1 ITS033 – Programming & Algorithms Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program, Image and Vision Computing Lab. School of Information, Computer and Communication Technology (ICT) Sirindhorn International Institute of Technology (SIIT) Thammasat University X 2005

5 Introduction The decrease-and-conquer technique is based on exploiting the relationship between a solution to a given instance of a problem and a solution to a smaller instance of the same problem. The decrease-and-conquer technique is based on exploiting the relationship between a solution to a given instance of a problem and a solution to a smaller instance of the same problem. Once such a relationship is established, it can be exploited either top down (recursively) or bottom up (without a recursion). Once such a relationship is established, it can be exploited either top down (recursively) or bottom up (without a recursion).

6 Introduction There are three major variations of decrease-and-conquer: There are three major variations of decrease-and-conquer: 1. Decrease by a constant 2. Decrease by a constant factor 3. Variable size decrease

7 Decrease by a constant In the decrease-by-a-constant variation, the size of an instance is reduced by the same constant on each iteration of the algorithm. In the decrease-by-a-constant variation, the size of an instance is reduced by the same constant on each iteration of the algorithm. Typically, this constant is equal to 1 Typically, this constant is equal to 1

8 Decrease by a Constant

9 Decrease by a constant Consider, as an example, the exponentiation problem of computing an for positive integer exponents. The relationship between a solution to an instance of size n and an instance of size n - 1 is obtained by the Consider, as an example, the exponentiation problem of computing an for positive integer exponents. The relationship between a solution to an instance of size n and an instance of size n - 1 is obtained by the obvious formula: a n = a n-1 x a So the function f (n) = an can be computed either “top down” by using its recursive definition So the function f (n) = an can be computed either “top down” by using its recursive definition or “bottom up” by multiplying a by itself n - 1 times. or “bottom up” by multiplying a by itself n - 1 times.

10 Decrease by a Constant Factor The decrease-by-a-constant-factor technique suggests reducing a problem’s instance by the same constant factor on each iteration of the algorithm. The decrease-by-a-constant-factor technique suggests reducing a problem’s instance by the same constant factor on each iteration of the algorithm. In most applications, this constant factor is equal to two. In most applications, this constant factor is equal to two.

11 Decrease by a Constant Factor

12 Decrease by a Constant Factor If the instance of size n is to compute a n, the instance of half its size will be to compute a n/2, with the obvious relationship between the two: a n = (a n/2 ) 2. If the instance of size n is to compute a n, the instance of half its size will be to compute a n/2, with the obvious relationship between the two: a n = (a n/2 ) 2. But since we consider instances of the exponentiation problem with integer exponents only, the former works only for even n. If n is odd, we have to compute a n-1 by using the rule for even-valued exponents and then multiply the result by But since we consider instances of the exponentiation problem with integer exponents only, the former works only for even n. If n is odd, we have to compute a n-1 by using the rule for even-valued exponents and then multiply the result by

13 Variable Size Decrease the variable-size-decrease variety of decrease-and-conquer, a size reduction pattern varies from one iteration of an algorithm to another. the variable-size-decrease variety of decrease-and-conquer, a size reduction pattern varies from one iteration of an algorithm to another. Euclid’s algorithm for computing the greatest common divisor provides a good example of such a situation. Euclid’s algorithm for computing the greatest common divisor provides a good example of such a situation.

14 Decrease & Conquer: Insertionsort Lecture 06.2 ITS033 – Programming & Algorithms Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program, Image and Vision Computing Lab. School of Information, Computer and Communication Technology (ICT) Sirindhorn International Institute of Technology (SIIT) Thammasat University X 2005

15 Insertion Sort we consider an application of the decrease-by-one technique to sorting an array A[0..n - 1]. we consider an application of the decrease-by-one technique to sorting an array A[0..n - 1]. Following the technique’s idea, we assume that the smaller problem of sorting the array A[0..n - 2] has already been solved to give us a sorted array of size n - 1: A[0]=... = A[n - 2]. Following the technique’s idea, we assume that the smaller problem of sorting the array A[0..n - 2] has already been solved to give us a sorted array of size n - 1: A[0]=... = A[n - 2]. How can we take advantage of this solution to the smaller problem to get a solution to the original problem by taking into account the element A[n - 1]? How can we take advantage of this solution to the smaller problem to get a solution to the original problem by taking into account the element A[n - 1]?

16 Insertion Sort we can scan the sorted subarray from right to left until the first element smaller than or equal to A[n - 1] is encountered and then insert A[n - 1] right after that element. =>straight insertion sort or simply insertion sort. we can scan the sorted subarray from right to left until the first element smaller than or equal to A[n - 1] is encountered and then insert A[n - 1] right after that element. =>straight insertion sort or simply insertion sort. Or we can use binary search to find an appropriate position for A[n - 1] in the sorted portion of the array. => binary insertion sort. Or we can use binary search to find an appropriate position for A[n - 1] in the sorted portion of the array. => binary insertion sort.

17 Insertion Sort

18 Insertion Sort

19 Insertion Sort Demo BRUTEFORCE unsortedactivesorted

20 Insertion Sort Demo unsortedactivesorted BRUTEFORCE

21 Insertion Sort Demo unsortedactivesorted BRUTEFORCE

22 Insertion Sort Demo unsortedactivesorted BRUTEFORCE

23 Insertion Sort Demo unsortedactivesorted BRUTEFORCE

24 Insertion Sort Demo unsortedactivesorted BRUTEFORCE

25 Insertion Sort Demo unsortedactivesorted BRUTEFORCE

26 Insertion Sort Demo unsortedactivesorted BRUTEFORCE

27 Insertion Sort Demo unsortedactivesorted BRUTEFORCE

28 Insertion Sort Demo unsortedactivesorted BRUTEFORCE

29 Insertion Sort Demo unsortedactivesorted BRUTEFORCE

30 Insertion Sort Demo unsortedactivesorted BRUTEFORCE

31 Insertion Sort Demo unsortedactivesorted BRUTEFORCE

32 Insertion Sort Demo unsortedactivesorted BRUTEFORCE

33 Insertion Sort Demo unsortedactivesorted BRUTEFORCE

34 Insertion Sort

35 Analysis –Worst Case The basic operation of the algorithm is the key comparison A[j ]> v. The basic operation of the algorithm is the key comparison A[j ]> v. The number of key comparisons in this algorithm obviously depends on the nature of the input. The number of key comparisons in this algorithm obviously depends on the nature of the input. In the worst case, A[j ]> v is executed the largest number In the worst case, A[j ]> v is executed the largest number of times, i.e., for every j = i - 1,..., 0. Since v = A[i], it happens if and only if A[j ]>A[i] for j = i - 1,..., 0. of times, i.e., for every j = i - 1,..., 0. Since v = A[i], it happens if and only if A[j ]>A[i] for j = i - 1,..., 0.

36 Analysis –Worst Case In other words, the worst-case input is an array of strictly decreasing values. The number of key comparisons for such an input is In other words, the worst-case input is an array of strictly decreasing values. The number of key comparisons for such an input is

37 Analysis – Best Case In the best case, the comparison A[j ]> v is executed only once on every iteration of the outer loop. It happens if and only if A[i - 1] = A[i] for every i =1,..., n-1, i.e., if the input array is already sorted in ascending order. In the best case, the comparison A[j ]> v is executed only once on every iteration of the outer loop. It happens if and only if A[i - 1] = A[i] for every i =1,..., n-1, i.e., if the input array is already sorted in ascending order. Thus, for sorted arrays, the number of key comparisons is Thus, for sorted arrays, the number of key comparisons is

38 Decrease & Conquer: Recursive Programming Lecture 06.3 ITS033 – Programming & Algorithms Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program, Image and Vision Computing Lab. School of Information, Computer and Communication Technology (ICT) Sirindhorn International Institute of Technology (SIIT) Thammasat University X 2005

39 Concept of Recursion A recursive definition is one which uses the word or concept being defined in the definition itself

40 Factorials How is this recursive? How is this recursive? n! = n × (n-1) × (n-2) × … × 3 × 2 × 1 = (n-1)! So: n! = n × (n-1) ! So: n! = n × (n-1) !  The factorial function is defined in terms of itself (i.e. recursively)

41 Recursive Calculation of Factorials In order for this to work, we need a stop case (the simplest case) In order for this to work, we need a stop case (the simplest case) Here: 0! = 1 Here: 0! = 1 n! = n × (n-1)!

42 This is Iterative Problem Solving

43 but, this is Recursion

44 long Factorial(int n) { long fact = 1; for (int i=2; i <= n; i++) fact = fact * i; return fact; } n! = n  (n-1)  (n-2)  …  1, if n > 0 Iterative Solution

45 Recursive definition - definition in which something is defined in terms of a smaller version of itself, e.g. Programming with Recursion n! = n  (n-1)!, if n > 0 1, if n = 0

46 Stopping Condition in a recursive definition is the case for which the solution can be stated nonrecursively General (recursive) case is the case for which the solution is expressed in terms of a smaller version of itself. Programming with Recursion n! = n  (n-1)!, if n > 0 1, if n = 0

47 long MyFact (int n) { if (n == 0) return 1; return (n * MyFact (n – 1)); } Recursive call - a call made to the function from within the function itself; Stopping cond. recursive case Recursion Solution

48 How does this work? int MyFact (int n) { if (n == 0) // The stop case { if (n == 0) // The stop case return 1; return 1; else else return n * MyFact (n-1); return n * MyFact (n-1); } // factorial } // factorial int x = MyFact(3); MyFact (3) 3*MyFact (2) 2*MyFact (1) 1*MyFact (0) 1126

49 Example #1 #include #include void iforgot_A(int n) { for (int i=1; i<=n; i++) for (int i=1; i<=n; i++) { printf("%d, I will remember to do my homework.\n",i); printf("%d, I will remember to do my homework.\n",i); } printf("Maybe NOT!"); printf("Maybe NOT!");} void main() { iforgot_A(5); iforgot_A(5); getch(); getch();} Iterative programming >> iforgot_A(5) 1, I will remember to do my homework. 2, I will remember to do my homework. 3, I will remember to do my homework. 4, I will remember to do my homework. 5, I will remember to do my homework. Maybe NOT!

50 Example #2 #include #include void iforgot_B(int n) { if (n>0) if (n>0) { printf("%d, I will remember to do my homework.\n",n); printf("%d, I will remember to do my homework.\n",n); iforgot_B(n-1); iforgot_B(n-1); } else else printf("Maybe NOT!"); printf("Maybe NOT!");} void main() { iforgot_B(5); iforgot_B(5); getch(); getch();} Recursive programming >> iforgot_B(5) 5, I will remember to do my homework. 4, I will remember to do my homework. 3, I will remember to do my homework. 2, I will remember to do my homework. 1, I will remember to do my homework. Maybe NOT!

51 Writing Recursive Functions Get an exact definition of the problem to be solved. Get an exact definition of the problem to be solved. Determine the size of the input of the problem. Determine the size of the input of the problem. Identify and solve the stopping condition(s) in which the problem can be expressed non-recursively. Identify and solve the stopping condition(s) in which the problem can be expressed non-recursively. Identify and solve the general case(s) correctly in terms of a smaller case of the same problem. Identify and solve the general case(s) correctly in terms of a smaller case of the same problem.

52 Divide or decrease problem Divide or decrease problem  One “step” makes the problem smaller (but of the same type)  Stopping case (solution is trivial) Concept 2 - Recursive Thinking

53 Recursion as problem solving technique Recursive methods defined in terms of themselves Recursive methods defined in terms of themselves  In code - will see a call to the method itself  Can have more than one “activation” of a method going at the same time Each activation Each activation  has own values of parameters  Returns to where it was called from System keeps track of this System keeps track of this

54 Stopping the recursion The recursion must always STOP The recursion must always STOP  Stopping condition is important Recursive solutions to problems Recursive solutions to problems

55 Stopping the recursion General pattern is General pattern is  test for stopping condition if not at stopping condition: if not at stopping condition: either either  do one step towards solution  call the method again to solve the rest or or  call the method again to solve most of the problem  do the final step

56 Implementation of Hanoi See the implementation of Tower of Hanoi in the lecture

57 Advantages of Recursion Advantages Advantages  Some problems have complicated iterative solutions, conceptually simple recursive ones  Good for dealing with dynamic data structures (size determined at run time).

58 Disadvantages of Recursion Disadvantages Disadvantages  Extra method calls use memory space & other resources  Thinking up recursive solution is hard at first  Believing that a recursive solution will work

59 Why Program Recursively? Recursive Code Non-Recursive Code typically has fewer lines. Code typically has fewer lines. Code can be conceptually simpler, depending on your perspective Code can be conceptually simpler, depending on your perspective Often easier to maintain! Often easier to maintain! Code is longer. Code is longer. Code executes faster, depending on hardware and programming language. Code executes faster, depending on hardware and programming language.

60 This Week’s Practice Write a recursive function to calculate Fibonacci numbers Write a recursive function to calculate Fibonacci numbers What is the result of f(6) ? What is the result of f(6) ?

61 Your recursive function

62 Fibonacci Recursive Tree fibo(6) fibo(5) + fibo(4) fibo(4) + fibo(3)fibo(3) + fibo(2) fibo(2) + fibo(1)

63 Decrease & Conquer: Homework ITS033 – Programming & Algorithms Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program, Image and Vision Computing Lab. School of Information, Computer and Communication Technology (ICT) Sirindhorn International Institute of Technology (SIIT) Thammasat University X 2005

64 Homework: Fake-Coin Problem Design an algorithm using Decrease and Conquer approach to solve Fake Coin Problem Among n identically looking coins, one is fake (lighter than genuine). Using balance scale to find that fake coin. How many time do you use the balance to find a fake coin from n coins ? Is it optimum ?

65 End of Chapter 6 Thank you!