1. Recitation 9 Programming for Engineers in Python

2. Plan Global variables Sorting: Quick sort Insertion sort Selection sort Complexity 2

3. Global Variables 3 debug_mode = True def example1(): if debug_mode: print 'Running example1‘ Global variables are created outside the functions, Can be accessed from any function.

4. Global Variables 4 The following example is supposed to keep track of whether the function has been called: -------------------------------------------------- been_called = False def example2(): been_called = True # WRONG -------------------------------------------------- What is the value of been_called? >>> been_called False

5. been_called doesn’t change! Instead, example2 creates a new local variable named been_called. When the function ends: the local been_called goes away no effect on the global variable. Global Variables 5 __main__ been_called = False example2 been_called = True

6. Global Variables 6 To reassign a global variable inside a function, declare the global variable before using it: -------------------------------- been called = False def example2(): global been_called been_called = True -------------------------------- The global statement tells the interpreter something like, “In this function, when I say been_called, I mean the global variable; don’t create a local one.”

7. Global Variables 7 If the global value is mutable, you can modify it without declaring it: known = {0:0, 1:1} def example4(): known[2] = 1 Add, remove and replace elements of a global list or dictionary – works! Re-assignment – requires declaring it: def example5(): global known known = dict()

8. Sorting 8 In many queries we handle a list of values and want a specific value. We can go over all the list until we find the requested value. But better… We can sort the list and then find it more easily (e.g., by binary search)

9. Sorting 9 In class we saw several examples for sorting: Bubble Sort Merge Sort We will now see more sorting algorithms.

10. Quick Sort 10 Principle: Divide and conquer Divide the list into two sub-lists such that all elements from one list are greater than all elements from other list. Sort each of these sub-lists recursively

11. Quick Sort 11 Divide the list into two sub-lists: Pick a pivot element Put all elements > pivot to its right Put all elements < pivot to its left 8 39 15141367 13 815 679314

12. Quick Sort 12 At this stage, the pivot is in its final position 8 39 15141367 13 815 679314

13. How do we choose the pivot? 13 Ideally, we should take the median value However, it would take time to find what is the median, so we can take the: First Middle Last Any random element In practice, each such choice is relatively efficient.

14. Some visualization 14 http://www.youtube.com/watch?v=vxENKlcs2Tw http://www.youtube.com/watch?v=y_G9BkAm6B8 http://www.youtube.com/watch?v=ywWBy6J5gz8

15. Implementation 15 First, we look on the ‘backbone’ of the algorithm: Partitioning the list according to a pivot into two sub- lists. After this stage, the pivot value is in its final position. Recursively repeat this procedure to each of the two sublists.

16. Quick Sort (backbone) 16 def quicksort_help(lst, first, last): ''' a recursive function to sort a lst from index first to index last, in place''' print first, last,lst if first < last: pivot = partition(lst, first, last) # recursive calls quicksort_help(lst, first, pivot-1) quicksort_help(lst, pivot+1, last) def quicksort(lst): ''' sort the list''' quicksort_help(lst, 0, len(lst)-1) Where is the termination of the recursive function?

17. Partition 17 def partition(lst, first, last): pivot = choose_pivot(lst, first, last) pivot_val = lst[pivot] swap(lst, pivot, last) # pivot aside # go over the list and partition store_ind = first for i in range(first, last): if lst[i] < pivot_val: swap(lst, i, store_ind) store_ind += 1 swap(lst, store_ind, last) return store_ind

18. Auxiliary Functions 18 def choose_pivot(lst, first, last): ''' choose the index of the pivot. This simple implementation always returns the last index''' return first def swap(lst, i, j): ''' recieve a list and indices, i and j. Switch the the i-th and the j-th elements in the list ''' tmp = lst[i] lst[i] = lst[j] lst[j] = tmp

19. QuickSort 19 lst = [] for i in range(20): lst.append(randrange(30)) print lst quicksort(lst, 0, len(lst)-1) print lst [11, 14, 2, 10, 7, 5, 1, 6, 12, 17, 16, 13, 9, 11, 10, 10, 0, 17, 25, 21] [0, 1, 2, 5, 6, 7, 9, 10, 10, 10, 11, 11, 12, 13, 14, 16, 17, 17, 21, 25]

20. Insertion Sort 20 Another example for sorting algorithm is Insertion Sort: In average, It is less efficient compared to Quick Sort. For small lists it is quicker than other algorithms. Many people perform a similar algorithm when sorting elements (cards etc.) We will explain this algorithm in class, you will implement it in HW.

21. Insertion Sort 21 Input: lst – an unsorted list of elements The algorithm: Go over lst from the 2 nd element until the end. For the i’th element: - Assume that the sub-list lst[0:i-1] is sorted. - Put the element i in its position so that the sub-list lst[0:i] is sorted and contains elements 1, …,i. At this stage, the first i elements are sorted. When we reach the end of the list, the list is sorted.

22. Insertion Sort - Example 22

23. Comparing Sorting 23 Quick Sort Insertion Sort

24. Complexity of algorithms 24 We can compare the two algorithms in respect to time complexity. Here we shall introduce only the highlights of that complexity analysis.

25. Insertion Sort - Complexity 25 Best case scenario: The list is already sorted. In this case, the i’th element is compared only to the (i-1)’th element. Thus, we have ~n comparisons. Worst case scenario: The list is sorted backwards. In this case the i’th element needs to be shifted all the way back to the start of the list. Quadratic complexity: 1+2+3+..+n-1=~n 2 Average case (without proving): ~n 2 operations.

26. Quick Sort - Complexity 26 The partition step takes ~n operations - we go through the list and change the location of the elements according to the pivot. Best case scenario: In each partition, we divide the list into almost equal size lists, each containing (k-1)/2 elements. That means we have ~log(n) partitions. Therefore total time complexity is ~n*log(n). Worst case scenario: In each step we divide the list of k elements (starting from k=n) into k-1 and 1 elements. In this case each step takes ~n-k operations. Thus we have: (n-1)+(n-2)+(n-3)+…+1=~n 2.

27. Quick Sort - Complexity 27 Average case (without proving): In this case it takes ~1.39n*log(n) operations. Thus, Quick Sort is faster than Insertion Sort on average. In fact, any algorithm for sorting n elements using pairwise comparisons cannot do it faster than ~n*log(n) in the average case.

28. Selection Sort 28 Given a list of elements: Find the minimum and swap it with the first element. Repeat for the remainder of the list – swap the i’th smallest element with the i‘th element in the list.

29. Selection Sort - Example 29

30. Selection Sort - Complexity 30 In each step we look for the minimal element – thus we go through all the remaining elements. This is done n times. Time complexity is: (n)+(n-1)+(n-2)+…+1~=n 2 Done on all inputs, therefore best case=worst case=average case.

31. And the winner is… 31 Quick Sort Insertion Sort Selection Sort