Recursion

Recursion is a powerful technique for thinking about a process It can be used to simulate a loop, or for many other kinds of applications In recursion, a function or procedure calls itself As with a while loop, there is a danger of an infinite recursion, so there has to be a test for stopping it, and something (usually a parameter) must change between calls 2

Recursion Can Mimic a While Loop Here’s the code for the while loop to build a multiplication table in our Loop Multiplication demo: j = 1 Do While j <= MAX lstAnswer.AddItem(strM & " X " & _ CStr(j) & " = " & _ CStr(numM * j)) j = j + 1 Loop 3

A Recursive Procedure Procedure call to get things started: RecurMult(numM, strM, 1) Procedure code: Sub RecurMult(ByVal numM As Integer, ByVal strM As String, ByVal j As Integer) If j <= MAX Then lstAnswer.Items.Add(strM & " X " & _ CStr(j) & " = " & _ CStr(numM * j)) RecurMult(numM, strM, j + 1) End If End Sub 4

How does it work? Note MAX is global. In the initial call, parameter j is 1. We print the line of the table with j = 1 and do the next call, with j = 2. In the second call, we print the line of the table with j =2, and do the call with j = 3. This continues till we do a call with j > MAX. In that case we just return without initiating another call. This triggers all the other calls, in reverse sequence, to return, and we’re done. 5

It’s not an easy concept… So now we’ll look at a simple example 6

Recursive Definition Sometimes it makes sense to define a quantity recursively: for example, the sum of the first n numbers: S(1) = 1 S(n) = n + S(n-1) for n>1 (As opposed to S(n) = … + n) 7

Questions for Recursive Definitions If I know how to compute the answer for n-1, how do I compute it for n? (Potentially you can use any values for arguments smaller than n in the definition) In our example, S(n) = S(n-1) + n How does the recursion stop? What is the “bottom” argument and what is the value for that argument? In our example, S(1) = 1 8

Recursive Computation Private Function SumOfN(ByVal n As Integer) As Integer If n <= 1 Then ‘expecting n = 1 but catch error SumOfN = 1 Else SumOfN = n + SumOfN(n – 1)) End If End Function X = SumOfN(5) returns = 15 9

Step by Step SumOfN(5) returns 5 + SumOfN(4) SumOfN(4) returns 4 + SumOfN(3) SumOfN(3) returns 3 + SumOfN(2) SumOfN(2) returns 2 + SumOfN(1) SumOfN(1) returns 1 Once we “hit bottom,” Sum0fN(1)returns its value; then SumOfN(2) can compute and return its value, and so on. 10

Compare to While Loop: j = n sum = 0 Do While j >= 1 sum = sum + j j = j – 1 Loop 11

Recursion can be Clearer Recursion directly implements the definition It is plain to see that it computes the correct value Coming up with the loop is not that easy for more complex recursive definitions, and its structure is quite different Note there is also a closed form for this recursion: S(n) = n(n + 1) / 2 Closed forms can be hard to find 12

Recursion Issues As with a while loop, a recursion can be infinite if we do not include a way to stop it The test to see if it is done should usually be the first thing a recursive function does Recursion uses more resources than a loop and it may not be possible to do a very large recursion (depends on language implementation) 13

Functional Programming This is a style of programming that replaces loops with recursions and assignment statements with parameter/argument associations After you get the knack of doing it, it can result in very clear, concise programs There are languages especially designed to support this style 14

Example: Fibonacci Numbers A program that implements Fibonacci numbers several ways. Here’s the recursive definition: Fib(0) = 0 Fib(1) = 1 Fib(n) = Fib(n – 1) + Fib(n – 2) 15

Demo: Fibonacci Numbers 16

Recursive Fib(5) 17 Fib(5) Fib(4) Fib(3) Fib(2) Fib(1) Fib(2) Fib(1) Fib(0) Fib(1) Fib (0) Fib(1) Fib(0)

Recursive Fib(5): Order of Calls 18 Fib(5) Fib(4) Fib(3) Fib(2) Fib(1) Fib(2) Fib(1) Fib(0) Fib(1) Fib (0) Fib(1) Fib(0)

Array Fib(5) In the first picture, we add elements 0 and 1 to get element 2 (Fib(2)), giving the second picture. Next add elements 1 and 2 to get Fib(3). Etc. (We start indexing with zero.)

Loop Fib(5) fn = fnm1 + fnm2 fnm2 = fnm1 fnm1 = fn 20 fn0 fnm11 fnm20 fn1 fnm11 fnm21 fn2 fnm12 fnm21 fn3 fnm13 fnm22 fn5 fnm15 fnm23 start step 1 step 2 step 3 step 4

Recursive Fib(5): Order of Calls 21 Fib(5) Fib(4) Fib(3) Fib(2) Fib(1) Fib(2) Fib(1) Fib(0) Fib(1) Fib (0) Fib(1) Fib(0)

Memoized Fib(5) 22 Fib(5) Fib(4) Fib(3) Fib(2) Fib(1) Fib(0)

Using Recursion in Sorting Motivation: to develop a fast sorting algorithm Recall Selection Sort: it takes time proportional to n 2, where we use the number of comparisons as a way to estimate the time, and n is the number of elements to be sorted This is too slow for sorting large data sets, even on a very fast computer

Why Selection Sort is Slow Selection sort and similar algorithms require us to do something similar to comparing every element to every other element We can be clever and avoid some of the comparisons but the basic nature of the algorithm-- that it takes time proportional to n 2 -- remains the same unless we use a radically different approach So in selection sort, instead of n + n +…+n, n times, = n*n, we have n + (n-1) + (n-2) + … , which equals n(n-1)/2. Less than half as big, but still roughly proportional to n 2, especially for large n

The Fundamental Idea The fundamental idea is to divide the problem roughly in half each time, solve the subproblems, and then put them back together If done cleverly, this can give us a time proportional to n log n. One way to do this is based on merging sorted lists. Let’s first look at how that works.

Merging Say we have two sorted lists that we want to combine to make one sorted list. For example: List A: 1, 5, 8, 15, 19 List B: 2, 5, 7, 20, 22 Method: look at the first element in each list. Put the smaller one in the answer. If one list is empty, put all the elements from the other list in the answer

Step 1 List A: 1, 5, 8, 15, 19 List B: 2, 5, 7, 20, 22 Answer: [empty] List A: 5, 8, 15, 19 List B: 2, 5, 7, 20, 22 Answer: 1

Step 2 List A: 5, 8, 15, 19 List B: 2, 5, 7, 20, 22 Answer: 1 List A: 5, 8, 15, 19 List B: 5, 7, 20, 22 Answer: 1, 2

Step 3 List A: 5, 8, 15, 19 List B: 5, 7, 20, 22 Answer: 1, 2 List A: 8, 15, 19 List B: 5, 7, 20, 22 Answer: 1, 2, 5

Step 4 List A: 8, 15, 19 List B: 5, 7, 20, 22 Answer: 1, 2, 5 List A: 8, 15, 19 List B: 7, 20, 22 Answer: 1, 2, 5, 5

Step 5 List A: 8, 15, 19 List B: 7, 20, 22 Answer: 1, 2, 5, 5 List A: 8, 15, 19 List B: 20, 22 Answer: 1, 2, 5, 5, 7

Step 6 List A: 8, 15, 19 List B: 20, 22 Answer: 1, 2, 5, 5, 7 List A: 15, 19 List B: 20, 22 Answer: 1, 2, 5, 5, 7, 8

Step 7 List A: 15, 19 List B: 20, 22 Answer: 1, 2, 5, 5, 7, 8 List A: 19 List B: 20, 22 Answer: 1, 2, 5, 5, 7, 8, 15

Step 8 List A: 19 List B: 20, 22 Answer: 1, 2, 5, 5, 7, 8, 15 List A: List B: 20, 22 Answer: 1, 2, 5, 5, 7, 8, 15, 19

Step 9 List A: List B: 20, 22 Answer: 1, 2, 5, 5, 7, 8, 15, 19 List A: List B: Answer: 1, 2, 5, 5, 7, 8, 15, 19, 20, 22

Comparisons for Merging Until we empty one of the lists, it takes one comparison to get one element into the answer So, roughly, the number of comparisons to merge two sorted lists is proportional to the total number of elements in the two lists.

Merge Sort Outline Divide the list in half and sort each half Merge the two sorted halves into a sorted list How do we sort each half? Using MergeSort! Huh? Isn’t this a circular definition or something?

Let’s Start at the Bottom We’ll think about it this way: suppose we have an unsorted list of 8 elements. We are going to divide it into 8 tiny lists of one element each, and merge them in pairs Here’s our example list. We’ll show it as an array so it is easy to talk about each element

Merging One-Element Pairs

Merging Two-Element Pairs

Merging Four-Element Pairs

Analysis We double the size of the pairs each time. The number of times we can double to reach size n, starting with 1, is log n. So there are log n stages. The time for each stage is proportional to n, since the total elements being merged each time is n. So the overall time is n log n

More Ideas There are lots more recursive sorting algorithms For example, in Quicksort, we divide the problem in half (in time n) by putting the elements bigger than some given element in the back and the smaller ones in the front. Do the same to each piece till you get to size one: there are log n stages

Sorting Summary (1) Selection Sort and Bubble Sort are not recursive. They both take time proportional to n 2, though Bubble sort can be somewhat faster than Selection sort Mergesort and Quicksort are both recursive in concept, though there are ways to avoid explicit recursions in the implementation. Mergesort is excellent especially for sorting data too big to all fit in memory. It always takes time proportional to n log n

Sorting Summary (2) Quicksort is recursive and usually very fast, proportional to n log n. In the worst case (sorted data!) it can take time proportional to n 2, though. Clever variations try to avoid this problem. The best possible time to sort just based on comparisons is proportional to n log n. You can do a bit better if, for example, you know that the data will be numbers

Sorting Summary (3) For small amounts of data just use a simple algorithm, or rely on the ones built into Excel or VBA If your program handles huge amounts of data then writing your own fast sort is one thing to try for speeding it up Our main purpose here was to give you a feeling for the vast variety of clever algorithms that can be developed to perform a task

The Sorting Sampler The sorting sampler lets you play around with several algorithms and look at how they behave when sorting the same data.