2. Reynolds 2006 Complexity1 Complexity Analysis Algorithm: –A sequence of computations that operates on some set of inputs and produces a result in a finite period of time. Representing algorithms: –Pseudocode Characterizing performance of algorithms: –Theta, or “Big-O” notation

3. Reynolds 2006 Complexity2 Sequential Search Pseudocode Read all the names into array NAMES. index = 0 matchFound = false locationOfMatch = -1 while( matchFound == false and index < length_of_NAMES_array ) do if( NAMES[ index ] == "Debbie Drawe" ) then matchFound = true locationOfMatch = index else index = index + 1 endWhile if( matchFound == true ) then print( "Match found at name " & index ) else print( "Match not found" ) EndOfAlgorithm

4. Reynolds 2006 Complexity3 Pair up! Count the primitive operations in the sequential search pseudocode What is the worst case scenario? Develop a formula for the total number of operations in a sequential search (worst case).

5. Reynolds 2006 Complexity4 Sequential Search Performance The while loop will dominate execution time when the number of names in the list becomes large. –The number of iterations will increase proportionally to the size of the list. The “order of growth” of time to search will be n, the number of names in the list. –Θ( n ) –Sequential Search has a Theta of n.

6. Reynolds 2006 Complexity5 Sequential Search Performance What is best case performance? 1*(while loop time) + overhead What is average case performance? (n/2)*(while loop time) + overhead What is worst case performance? n*(while loop time) + overhead

7. Reynolds 2006 Complexity6 Insertion Sort Given a set to be sorted, use the card player’s approach: –Add each new item at the appropriate place in the set of already sorted items.

8. Reynolds 2006 Complexity7 Insertion Sort Pseudocode Read all the numbers into array NUMS. numberIndex = 1 sortedIndex = 0 while( numberIndex < length_of_NUMS_array ) do newNum = NUMS[ numberIndex ] sortedIndex = numberIndex - 1 /* From high to low, look for the place for the new number. If the previously sorted numbers are larger, move them up in the array NUMS. */ while( NUMS[ sortedIndex ] > newNum and sortedIndex >= 0 ) do NUMS[ sortedIndex + 1 ] = NUMS[ sortedIndex ] sortedIndex = sortedIndex - 1 endWhile /* We found the place for the new number, so insert it. */ NUMS[ sortedIndex + 1 ] = newNum numberIndex++ sortedIndex = numberIndex -1; endWhile EndOfAlgorithm

9. Reynolds 2006 Complexity8 Performance of Insertion Sort Execution time will increase with the size of the set to be sorted (outer while loop) Each element to be sorted must be compared one or many times with the elements already sorted (inner while loop).

10. Reynolds 2006 Complexity9 Insertion Sort Performance What is best case performance? Items already sorted n*(outerWhile) + 1*(innerWhile) + overhead What is average case performance? n*(outerWhile) + n*(n/2)*(innerWhile) + overhead What is worst case performance? Items already sorted in reverse order n*(outerWhile) + ((n 2 – n)/2)*(innerWhile) + overhead What is Theta? Θ(n 2 ).

11. Reynolds 2006 Complexity10 Merge Sort Divide the problem into many smaller problems before recombining the elements for the full solution. –Use recursion to create the smaller problems (stay tuned…) Imagine two piles of playing cards: –Each pile is sorted from smallest to largest, with the cards face up in two piles. –The two smallest cards are showing. –The merge routine compares the two cards that are showing, and places the smaller card face down in the merged pile.

12. Reynolds 2006 Complexity11 Merge Pseudocode The following pseudocode takes: –A single array NUMS[], which includes two sorted sub arrays. –The first sorted subarray is NUMS[first] through NUMS[second-1] –The second sorted subarray is NUMS[second] through NUMS[last] Merge is only the merge part of the merge sort.

13. Reynolds 2006 Complexity12 Merge Pseudocode Procedure merge( NUMS[ ], first, second, last ) if( second == first ) return index1 = first index2 = second while( index1 < index2 and index2 <= last ) do /* If the number in the first array is smaller, leave it in that position, and check the next number in the first array. */ if( NUMS[ index1 ] <= NUMS[ index2 ] ) then index1 = index1 + 1 else /* If the number in the second array is smaller, save it, move the array elements up one, and put the number from the second array in place. */ temp = NUMS[ index2 ] shiftIndex = index2 while( shiftIndex > index1 ) do NUMS[ shiftIndex ] = NUMS[ shiftIndex - 1 ] shiftIndex = shiftIndex - 1 endWhile /* Insert the smaller element in the hole */ NUMS[ index1 ] = temp index1 = index1 + 1 index2 = index2 + 1 endIf endWhile endProcedure

14. Reynolds 2006 Complexity13 Merge Sort Use recursion to divide the problem into subproblems. Merge sort repeatedly calls itself until the recursion "bottoms out" with lists whose lengths are one. Then the recursion "returns," reassembling the numbers in sorted order as it does.

15. Reynolds 2006 Complexity14 Merge Sort Pseudocode Procedure mergeSort( NUMS[ ], first, last ) if( first < last ) then middle = truncate( ( last + first ) / 2 ) call mergeSort( NUMS, first, middle ) call mergeSort( NUMS, middle + 1, last ) merge( NUMS, first, middle + 1, last ) endProcedure The NUMS[] array is the set of items to be sorted. The first subscript of the items to be sorted is first. The last subscript of the items to be sorted is last.

16. Reynolds 2006 Complexity15 Following the Merge Sort for NUMS = { 1, 6, 4, 2 } Call mergeSort passing NUMS, 0, and 3. Middle = (0+3)/2 = 1. Call mergeSort again passing NUMS, 0, & 1 Middle = (0+1)/2 = 0. Call mergeSort again passing NUMS, 0, and 0 Since 0 is not less than 0, mergeSort simply returns

17. Reynolds 2006 Complexity16 (cont.)Following the Merge Sort for NUMS = { 1, 6, 4, 2 } Call mergeSort passing NUMS, –middle + 1 (1) as first, and last (1) Since 1 is not less than 1, it simply returns Call procedure merge passing NUMS, –first (0) –middle + 1 (1) –and last (1) –This is a call to order the first two elements (1,6)

18. Reynolds 2006 Complexity17 (cont.)Following the Merge Sort for NUMS = { 1, 6, 4, 2 } Having completed ordering the front half of the list, the algorithm returns to call mergeSort passing NUMS, –middle + 1 (2) –last (3). mergeSort calculates middle as (2+3)/2 = 2 Call mergeSort passing NUMS, 2, and 2 Since 2 is not less than 2, it simply returns

19. Reynolds 2006 Complexity18 (cont.)Following the Merge Sort for NUMS = { 1, 6, 4, 2 } call mergeSort passing NUMS, –middle + 1 (3) as first, –last (3). Since 3 is not less than 3, it mergeSort returns call procedure merge passing NUMS, –first (2), –middle + 1 (3), and –last (3). –This is a call to order the last two elements ( 4, 2 ).

20. Reynolds 2006 Complexity19 (cont.)Following the Merge Sort for NUMS = { 1, 6, 4, 2 } Having returned from ordering the front and back halves of the list, the algorithm calls merge passing NUMS, –first (0), List 1 consists of elements 0 & 1 (1, 6) –middle + 1 (2), List 2 consists of elements 2 & 3 (2, 4) –and last (3). Merge completes by ordering the list {1, 2, 4, 6}.

21. Reynolds 2006 Complexity20 Merge Sort Performance Total time T consists of the time to recursively solve 2 problems of half the size, and then to combine the results: T = 2T(n/2) + merge Since the time for merge is Θ(n): T = 2T(n/2) + Θ(n) How can we characterize T in general?

22. Reynolds 2006 Complexity21 Merge Sort Recursion Tree Sum Θ(n) Θ(n) Θ(n/2) Θ(n/2) Θ(n) Θ(n/4) Θ(n/4) Θ(n/4) Θ(n/4) Θ(n)...

23. Reynolds 2006 Complexity22 Total Merge Sort Run Time For each level of recursion: –Θ(n) How many levels of recursion? –log 2 ( n ) –lg( n ) –A problem with 8 numbers will have 3 levels What is the Theta of merge sort? –Θ( n( lg n ) )

24. Reynolds 2006 Complexity23 What is the Significance of Θ( n( lg n ) ) vs. Θ( n 2 )? Suppose 1 million numbers must be sorted –insertion sort will require ~(10 6 ) 2, or 1,000,000,000,000 cycles –merge sort will require ~10 6 ( lg 10 6 ), or 10 6 ( 20 ), or 20,000,000 cycles –Merge sort is faster by 5 orders of magnitude!

25. Reynolds 2006 Complexity24 Binary Search Sequential Search performs with Θ(n) If we know the list is ordered, we can search much more efficiently. –The difference between a phone book with entries in sorted order and a phone book with entries in random order! Binary Search performs with Θ(lg n) –Works by repetitively dividing the list in half. –Search a list

26. 25 Binary Search Pseudocode Read all the names into array NAMES. begin = 0 end = length_of_NAMES_array - 1 matchFound = false locationOfMatch = -1 searchItem = "Mark Waldman" while( matchFound == false and begin <= end ) do midpoint = truncate( ( begin + end ) / 2 ) if( NAMES[ midpoint ] == searchItem ) then matchFound = true locationOfMatch = midpoint else if( searchItem < NAMES[ midpoint ] ) then end = midpoint - 1 else begin = midpoint + 1 endIf endWhile if( matchFound == true ) then print( "Match found at name " & locationOfMatch ) else print( "Match not found" ) EndOfAlgorithm

27. Reynolds 2006 Complexity26 Binary Search Performance With each iteration, the binary search reduces the size of the list to be searched by a factor of 2. So, the binary search will find the search item, or conclude that the search item is not in the list, when the algorithm has executed (lg n) iterations or fewer. If there are 7 items in the list, the algorithm will complete in 3 iterations or fewer. If there are 1,000,000 items in the list, the algorithm will complete in 20 iterations or fewer.