1 CS 106 Computing Fundamentals II Chapter 79 “Recursion” Herbert G. Mayer, PSU CS status 6/24/2013 Initial content copied verbatim from CS 106 material developed by CS professors: Cynthia Brown & Robert Martin

2 Syllabus Recursion Recursion Mimic a While Loop Mimic a While Loop Recursion Recursion Step by Step Step by Step Recursion can be Clearer Recursion can be Clearer Recursion Issues Recursion Issues Functional Programs Functional Programs Sample: Fibonacci Sample: Fibonacci Merging Merging Merge Sort Outline Merge Sort Outline Analysis Analysis More ideas More ideas

3 Recursion Recursion is a powerful technique for thinking about certain types of algorithmsRecursion is a powerful technique for thinking about certain types of algorithms It can be used to simulate a loop, or other kinds of applicationsIt can be used to simulate a loop, or other kinds of applications Simplistically speaking, a recursive function is a function that calls itselfSimplistically speaking, a recursive function is a function that calls itself More correctly and completely: A recursive function is a function that is partly defined in simpler versions of itselfMore correctly and completely: A recursive function is a function that is partly defined in simpler versions of itself As with a while loop, there is a danger of an infinite recursion –referred to as infinite regress-- so there has to be a test for stopping it, and something (usually a parameter) must change between callsAs with a while loop, there is a danger of an infinite recursion –referred to as infinite regress-- so there has to be a test for stopping it, and something (usually a parameter) must change between calls 3

4 Mimic a While Loop Here’s the code for the while loop to build a multiplication table in our Loop Multiplication demo: Here’s the code for the while loop to build a multiplication table in our Loop Multiplication demo: j = 1 j = 1 Do While j <= MAX lstAnswer.AddItem(strM & " X " & _ CStr(j) & " = " & _CStr(numM * j)) lstAnswer.AddItem(strM & " X " & _ CStr(j) & " = " & _CStr(numM * j)) j = j + 1 j = j + 1Loop 4

5 A Recursive Procedure Procedure call to get things started: RecurMult(numM, strM, 1) Procedure code: Sub RecurMult(ByVal numM As Integer, ByVal strM As String, ByVal j As Integer) Sub RecurMult(ByVal numM As Integer, ByVal strM As String, ByVal j As Integer) If j <= MAX Then If j <= MAX Then lstAnswer.Items.Add(strM & " X " & _ CStr(j) & " = " & _CStr(numM * j)) lstAnswer.Items.Add(strM & " X " & _ CStr(j) & " = " & _CStr(numM * j)) RecurMult(numM, strM, j + 1) RecurMult(numM, strM, j + 1) End If End If End Sub End Sub 5

6 How does it work? Note MAX is global. In the initial call, parameter j is 1. We print the line of the table with j = 1 and do the next call, with j = 2.Note MAX is global. In the initial call, parameter j is 1. We print the line of the table with j = 1 and do the next call, with j = 2. In the second call, we print the line of the table with j = 2, and do the call with j = 3.In the second call, we print the line of the table with j = 2, and do the call with j = 3. This continues till we do a call with j > MAX. In that case we just return without initiating another call.This continues till we do a call with j > MAX. In that case we just return without initiating another call. This triggers all the other calls, in reverse sequence, to return, and we’re doneThis triggers all the other calls, in reverse sequence, to return, and we’re done 6

7 It’s not an easy concept… So now we’ll look at a simple example 7

8 Recursive Definition Sometimes it makes sense to define a quantity recursively: for example, the sum of the first n numbers: S(1) = 1 S(n) = n + S(n-1) for n>1 (As opposed to S(n) = … + n) 8

9 Questions for Recursive Definitions If I know how to compute the answer for n-1, how do I compute it for n? (Potentially you can use any values for arguments smaller than n in the definition)If I know how to compute the answer for n-1, how do I compute it for n? (Potentially you can use any values for arguments smaller than n in the definition) In our example, S(n) = S(n-1) + nIn our example, S(n) = S(n-1) + n How does the recursion stop? What is the “bottom” argument and what is the value for that argument?How does the recursion stop? What is the “bottom” argument and what is the value for that argument? In our example, S(1) = 1In our example, S(1) = 1 9

10 Recursive Computation Private Function SumOfN(ByVal n As Integer) As Integer If n <= 1 Then ‘expecting n = 1 but catch error SumOfN = 1 Else SumOfN = n + SumOfN(n – 1)) End If End Function X = SumOfN(5) returns = 15 10

11 Step by Step SumOfN(5) returns 5 + SumOfN(4) SumOfN(4) returns 4 + SumOfN(3) SumOfN(3) returns 3 + SumOfN(2) SumOfN(2) returns 2 + SumOfN(1) SumOfN(1) returns 1 Once we “hit bottom,” Sum0fN(1)returns its value; then SumOfN(2) can compute and return its value, and so on. 11

12 Compare to While Loop: j = n sum = 0 Do While j >= 1 sum = sum + j j = j – 1 Loop 12

13 Recursion can be Clearer Recursion directly implements the definition It is plain to see that it computes the correct value Coming up with the loop is not that easy for more complex recursive definitions, and its structure is quite different Note there is also a closed form for this recursion: S(n) = n(n + 1) / 2 Closed forms can be hard to find 13

14 Recursion Issues As with a while loop, a recursion can be infinite if we do not include a way to stop itAs with a while loop, a recursion can be infinite if we do not include a way to stop it The test to see if it is done should usually be the first thing a recursive function doesThe test to see if it is done should usually be the first thing a recursive function does Recursion uses more resources than a loop and it may not be possible to do a very large recursion (depends on language implementation)Recursion uses more resources than a loop and it may not be possible to do a very large recursion (depends on language implementation) 14

15 Functional Programming This is a style of programming that replaces loops with recursions and assignment statements with parameter/argument associationsThis is a style of programming that replaces loops with recursions and assignment statements with parameter/argument associations After you get the knack of doing it, it can result in very clear, concise programsAfter you get the knack of doing it, it can result in very clear, concise programs There are languages especially designed to support this styleThere are languages especially designed to support this style 15

16 Example: Fibonacci Numbers A program that implements Fibonacci numbers several ways. Here’s the recursive definition: Fib(0) = 0 Fib(1) = 1 Fib(n) = Fib(n – 1) + Fib(n – 2) 16

17 Demo: Fibonacci Numbers 17

18 Recursive Fib(5) 18 Fib(5) Fib(4) Fib(3) Fib(2) Fib(1) Fib(2) Fib(1 ) Fib(0) Fib(1) Fib (0) Fib(1) Fib(0)

19 Recursive Fib(5): Order of Calls 19 Fib(5) Fib(4) Fib(3) Fib(2) Fib(1) Fib(2) Fib(1 ) Fib(0) Fib(1) Fib (0) Fib(1) Fib(0)

20 Array Fib(5) In the first picture, we add elements 0 and 1 to get element 2 (Fib(2)), giving the second picture. Next add elements 1 and 2 to get Fib(3). Etc. (We start indexing with zero.)

21 Loop Fib(5) fn = fnm1 + fnm2 fnm2 = fnm1 fnm1 = fn 21 fn0 fnm11 fnm20 fn1 fnm11 fnm21 fn2 fnm12 fnm21 fn3 fnm13 fnm22 fn5 fnm15 fnm23 start step 1 step 2 step 3 step 4

22 Recursive Fib(5): Order of Calls 22 Fib(5) Fib(4) Fib(3) Fib(2) Fib(1) Fib(2) Fib(1 ) Fib(0) Fib(1) Fib (0) Fib(1) Fib(0)

23 Memoized Fib(5) 23 Fib(5) Fib(4) Fib(3) Fib(2) Fib(1 ) Fib(0)

24 Using Recursion in Sorting Motivation: to develop a fast sorting algorithmMotivation: to develop a fast sorting algorithm Recall Selection Sort: it takes time proportional to n 2, where we use the number of comparisons as a way to estimate the time, and n is the number of elements to be sortedRecall Selection Sort: it takes time proportional to n 2, where we use the number of comparisons as a way to estimate the time, and n is the number of elements to be sorted This is too slow for sorting large data sets, even on a very fast computerThis is too slow for sorting large data sets, even on a very fast computer

25 Why Selection Sort is Slow Selection sort and similar algorithms require us to do something similar to comparing every element to every other elementSelection sort and similar algorithms require us to do something similar to comparing every element to every other element We can be clever and avoid some of the comparisons but the basic nature of the algorithm-- that it takes time proportional to n 2 -- remains the same unless we use a radically different approachWe can be clever and avoid some of the comparisons but the basic nature of the algorithm-- that it takes time proportional to n 2 -- remains the same unless we use a radically different approach So in selection sort, instead of n + n +…+n, n times, = n*n, we have n + (n-1) + (n-2) + … , which equals n(n-1)/2. Less than half as big, but still roughly proportional to n 2, especially for large nSo in selection sort, instead of n + n +…+n, n times, = n*n, we have n + (n-1) + (n-2) + … , which equals n(n-1)/2. Less than half as big, but still roughly proportional to n 2, especially for large n

26 The Fundamental Idea The fundamental idea is to divide the problem roughly in half each time, solve the subproblems, and then put them back togetherThe fundamental idea is to divide the problem roughly in half each time, solve the subproblems, and then put them back together If done cleverly, this can give us a time proportional to n log n.If done cleverly, this can give us a time proportional to n log n. One way to do this is based on merging sorted lists. Let’s first look at how that works.One way to do this is based on merging sorted lists. Let’s first look at how that works.

27 Merging Say we have two sorted lists that we want to combine to make one sorted list. For example: List A: 1, 5, 8, 15, 19 List B: 2, 5, 7, 20, 22 Method: look at the first element in each list. Put the smaller one in the answer. If one list is empty, put all the elements from the other list in the answer

28 Step 1 List A: 1, 5, 8, 15, 19 List B: 2, 5, 7, 20, 22 Answer: [empty] List A: 5, 8, 15, 19 List B: 2, 5, 7, 20, 22 Answer: 1

29 Step 2 List A: 5, 8, 15, 19 List B: 2, 5, 7, 20, 22 Answer: 1 List A: 5, 8, 15, 19 List B: 5, 7, 20, 22 Answer: 1, 2

30 Step 3 List A: 5, 8, 15, 19 List B: 5, 7, 20, 22 Answer: 1, 2 List A: 8, 15, 19 List B: 5, 7, 20, 22 Answer: 1, 2, 5

31 Step 4 List A: 8, 15, 19 List B: 5, 7, 20, 22 Answer: 1, 2, 5 List A: 8, 15, 19 List B: 7, 20, 22 Answer: 1, 2, 5, 5

32 Step 5 List A: 8, 15, 19 List B: 7, 20, 22 Answer: 1, 2, 5, 5 List A: 8, 15, 19 List B: 20, 22 Answer: 1, 2, 5, 5, 7

33 Step 6 List A: 8, 15, 19 List B: 20, 22 Answer: 1, 2, 5, 5, 7 List A: 15, 19 List B: 20, 22 Answer: 1, 2, 5, 5, 7, 8

34 Step 7 List A: 15, 19 List B: 20, 22 Answer: 1, 2, 5, 5, 7, 8 List A: 19 List B: 20, 22 Answer: 1, 2, 5, 5, 7, 8, 15

35 Step 8 List A: 19 List B: 20, 22 Answer: 1, 2, 5, 5, 7, 8, 15 List A: List B: 20, 22 Answer: 1, 2, 5, 5, 7, 8, 15, 19

36 Step 9 List A: List B: 20, 22 Answer: 1, 2, 5, 5, 7, 8, 15, 19 List A: List B: Answer: 1, 2, 5, 5, 7, 8, 15, 19, 20, 22

37 Comparisons for Merging Until we empty one of the lists, it takes one comparison to get one element into the answerUntil we empty one of the lists, it takes one comparison to get one element into the answer So, roughly, the number of comparisons to merge two sorted lists is proportional to the total number of elements in the two lists.So, roughly, the number of comparisons to merge two sorted lists is proportional to the total number of elements in the two lists.

38 Merge Sort Outline Divide the list in half and sort each halfDivide the list in half and sort each half Merge the two sorted halves into a sorted listMerge the two sorted halves into a sorted list How do we sort each half? Using MergeSort!How do we sort each half? Using MergeSort! Huh? Isn’t this a circular definition or something?Huh? Isn’t this a circular definition or something?

39 Start at the Bottom We’ll think about it this way: suppose we have an unsorted list of 8 elements. We are going to divide it into 8 tiny lists of one element each, and merge them in pairsWe’ll think about it this way: suppose we have an unsorted list of 8 elements. We are going to divide it into 8 tiny lists of one element each, and merge them in pairs Here’s our example list. We’ll show it as an array so it is easy to talk about each element.Here’s our example list. We’ll show it as an array so it is easy to talk about each element

40 Merging One-Element Pairs

41 Merging Two-Element Pairs

42 Merging Four-Element Pairs

43 Analysis We double the size of the pairs each time. The number of times we can double to reach size n, starting with 1, is log n. So there are log n stages.We double the size of the pairs each time. The number of times we can double to reach size n, starting with 1, is log n. So there are log n stages. The time for each stage is proportional to n, since the total elements being merged each time is n.The time for each stage is proportional to n, since the total elements being merged each time is n. So the overall time is n log nSo the overall time is n log n

44 More Ideas There are lots more recursive sorting algorithmsThere are lots more recursive sorting algorithms For example, in Quicksort, we divide the problem in half (in time n) by putting the elements bigger than some given element in the back and the smaller ones in the front.For example, in Quicksort, we divide the problem in half (in time n) by putting the elements bigger than some given element in the back and the smaller ones in the front. Do the same to each piece till you get to size one: there are log n stagesDo the same to each piece till you get to size one: there are log n stages

45 Sorting Summary (1) Selection Sort and Bubble Sort are not recursive. They both take time proportional to n 2, though Bubble sort can be somewhat faster than Selection sortSelection Sort and Bubble Sort are not recursive. They both take time proportional to n 2, though Bubble sort can be somewhat faster than Selection sort Mergesort and Quicksort are both recursive in concept, though there are ways to avoid explicit recursions in the implementation. Mergesort is excellent especially for sorting data too big to all fit in memory. It always takes time proportional to n log nMergesort and Quicksort are both recursive in concept, though there are ways to avoid explicit recursions in the implementation. Mergesort is excellent especially for sorting data too big to all fit in memory. It always takes time proportional to n log n

46 Sorting Summary (2) Quicksort is recursive and usually very fast, proportional to n log n. In the worst case (sorted data!) it can take time proportional to n 2, though. Clever variations try to avoid this problem.Quicksort is recursive and usually very fast, proportional to n log n. In the worst case (sorted data!) it can take time proportional to n 2, though. Clever variations try to avoid this problem. The best possible time to sort just based on comparisons is proportional to n log n. You can do a bit better if, for example, you know that the data will be numbersThe best possible time to sort just based on comparisons is proportional to n log n. You can do a bit better if, for example, you know that the data will be numbers

47 Sorting Summary (3) For small amounts of data just use a simple algorithm, or rely on the ones built into Excel or VBAFor small amounts of data just use a simple algorithm, or rely on the ones built into Excel or VBA If your program handles huge amounts of data then writing your own fast sort is one thing to try for speeding it upIf your program handles huge amounts of data then writing your own fast sort is one thing to try for speeding it up Our main purpose here was to give you a feeling for the vast variety of clever algorithms that can be developed to perform a taskOur main purpose here was to give you a feeling for the vast variety of clever algorithms that can be developed to perform a task

48 The Sorting Sampler The sorting sampler lets you play around with several algorithms and look at how they behave when sorting the same data. The sorting sampler lets you play around with several algorithms and look at how they behave when sorting the same data.