1. Computer Science 112 Fundamentals of Programming II Finding Faster Algorithms

2. Bubble Sort Strategy Compare the first two items and if they are out of order, exchange them Repeat this process for the second and third items, etc. At the end of this process, the largest itemwill have bubbled down to the end of the list Repeat this process for the unsorted portion of the list, etc.

3. set n to the length of the list while n > 1 bubble the elements from position 0 to position n - 1 decrement n Formalize the Strategy

4. set n to the length of the list while n > 1 for each position i from 1 to n - 1 if the elements at i and i - 1 are out of order swap them decrement n Refine the Strategy

5. def bubbleSort(lyst): n = len(lyst) while n > 1: # Do n - 1 bubbles #i = 1 # Start each bubble for i in range(1, n): if lyst[i] < lyst[i - 1]: # Swap if needed swap(lyst, i, i – 1) n -= 1 Implement bubbleSort Analysis: How many iterations does the outer loop perform? How many iterations does the inner loop perform?

6. def bubbleSort(lyst): n = len(lyst) while n > 1: isSorted = True i = 1 for i in range(n): if lyst[i] < lyst[i - 1]: swap(lyst, i, i – 1) isSorted = False if isSorted: break n -= 1 Improving bubbleSort Analysis: Best, worst, average cases?

7. Example: Exponentiation def ourPow(base, expo): if expo == 0: return 1 else: return base * ourPow(base, expo – 1) What is the best case performance? Worst case? Average case? Recursive definition: b n = 1, when n = 0 b n = b * b n-1 otherwise

8. Faster Exponentiation def fastPow(base, expo): if expo == 0: return 1 elif n % 2 == 1: return base * fastPow(base, expo – 1) else: result = fastPow(base, expo // 2) return result * result What is the best case performance? Worst case? Average case? Recursive definition: b n = 1, when n = 0 b n = b * b n-1, when n is odd b n = (b n/2 ) 2, when n is even

9. The Fibonacci Series def fib(n): if n == 1 or n == 2: return 1 else: return fib(n – 1) + fib(n – 2) fib(n) = 1, when n = 1 or n = 2 fib(n) = fib(n – 1) + fib(n – 2) otherwise 1 1 2 3 5 8 13...

10. Tracing fib(5) with a Call Tree fib(5) fib(4) fib(3) fib(2) fib(1) 1 1 1 1 1

11. Work Done – Function Calls fib(5) fib(4) fib(3) fib(2) fib(1) 1 1 1 1 1 Somewhere between 1 n and 2 n

12. Memoization def fib(n): if n == 1 or n == 2: return 1 else: return fib(n – 1) + fib (n – 2) Intermediate values returned by the function can be memoized, or saved in a cache, for subsequent access Then they don’t have to be recomputed!

13. Memoization def fib(n): cache = dict() def fastFib(n): if n == 1 or n == 2: return 1 elif n in cache: return cache[n] else: value = fastFib(n – 1) + fastFib(n – 2) cache[n] = value return value return fastFib(n) The cache is a dictionary whose keys are the arguments of fib and whose values are the values of fib at those keys

14. Improving on n 2 Sorting Selection sort uses a linear method within a linear method, so it ’ s an O(n 2 ) method Find a way of using a linear method with a method that ’ s better than linear

15. A Hint from Binary Search Binary search is better than linear, because we divide the problem size by 2 on each step Find a way of dividing the size of sorting problem by 2 on each step, even though each step will itself be linear This should produce an O(nlogn) algorithm

16. Quick Sort Select a pivot element (say, the element at the midpoint) Shift all of the smaller values to the left of the pivot, and all of the larger values to the right of the pivot (the linear part) Sort the values to the left and to the right of the pivot (ideally, done logn times)

17. 89566372413495 0 1 2 3 4 5 6 pivot Step 1: select the pivot (at the midpoint) 56634134728995 0 1 2 3 4 5 6 Step 2: shift the data pivot Trace of Quick Sort

18. 56634134728995 0 1 2 3 4 5 6 Step 3: sort to the left of the pivot pivot 34415663728995 0 1 2 3 4 5 6 Step 4: sort to the right of the pivot pivot Trace of Quick Sort

19. Design of Quick Sort: First Cut quickSort(lyst, left, right) if left < right pivotPosition = partition(lyst, left, right) quickSort (lyst, left, pivotPosition - 1); quickSort (lyst, pivotPosition + 1, right)

20. Design of Quick Sort: First Cut quickSort(lyst, left, right) if left < right pivotPosition = partition(lyst, left, right) quickSort (lyst, left, pivotPosition - 1); quickSort (lyst, pivotPosition + 1, right) This version selects the midpoint element as the pivot The position of the pivot might change during the shifting of data partition(lyst, left, right) pivotValue = lyst[(left + right) // 2] shift smaller values to left of pivotValue shift larger values to right of pivotValue return pivotPosition

21. Implementation of Partition def partition(lyst, left, right): # Find the pivot and exchange it with the last item middle = (left + right) // 2 pivot = lyst[middle] lyst[middle] = lyst[right] lyst[right] = pivot # Set boundary point to first position boundary = left # Move items less than pivot to the left for index in range(left, right): if lyst[index] < pivot: swap(lyst, index, boundary) boundary += 1 # Exchange the pivot item and the boundary item swap(lyst, right, boundary) return boundary The number of comparisons required to shift values in each sublist is equal to the size of the sublist.

22. def quickSort(lyst): def recurse(left, right): if left < right: pivotPosition = partition(lyst, left, right) recurse(left, pivotPosition - 1); recurse(pivotPosition + 1, right) def partition(lyst, left, right): # Find the pivot and exchange it with the last item middle = (left + right) // 2 pivot = lyst[middle] lyst[middle] = lyst[right] lyst[right] = pivot # Set boundary point to first position boundary = left # Move items less than pivot to the left for index in range(left, right): if lyst[index] < pivot: swap(lyst, index, boundary) boundary += 1 # Exchange the pivot item and the boundary item swap(lyst, right, boundary) return boundary recurse(0, len(lyst) – 1)

23. The number of comparisons in the top-level call is n Complexity Analysis The sum of the comparisons in the two recursive calls is also n The sum of the comparisons in the four recursive calls beneath these is also n, etc. Thus, the total number of comparisons equals n * the number of times the list must be subdivided

24. How Many Times Must the Array Be Subdivided? It depends on the data and on the choice of the pivot element Ideally, when the pivot is the median on each call, the list is subdivided log 2 n times Best-case behavior is O(nlogn)

25. Call Tree For a Best Case 34415663728995 344156728995 34567295 We select the midpoint element as the pivot. The median element happens to be at the midpoint on each call. But the list was already sorted!

26. Worst Case What if the value at the midpoint is near the largest value on each call? Or near the smallest value on each call? Then there will be approximately n subdivisions, and quick sort will degenerate to O(n 2 )

27. Call Tree For a Worst Case 34415663728995 We select the first element as the pivot. The smallest element happens to be the first one on each call. n subdivisions! 415663728995 5663728995 63728995 728995 8995

28. Other Methods of Selecting the Pivot Element Pick a random element Pick the median of the first three elements Pick the median of the first, middle, and last elements Pick the median element - not!! This is an O(n) algorithm

29. For Friday Working with the Array Data Structure Chapter 4