1. HEAPS Amihood Amir Bar Ilan University 2014

2. Sorting Bubblesort: Until no exchanges: For i=1 to n-1 if A[i]>A[i+1] then exchange their values end Time : O(n 2 )

3. "Bubbling Up" the Largest Element Traverse a collection of elements –Move from the front to the end –“Bubble” the largest value to the end using pair-wise comparisons and swapping 5 12 3542 77 101 1 2 3 4 5 6 Swap 4277

4. "Bubbling Up" the Largest Element Traverse a collection of elements –Move from the front to the end –“Bubble” the largest value to the end using pair-wise comparisons and swapping 5 12 3577 42 101 1 2 3 4 5 6 Swap 3577

5. "Bubbling Up" the Largest Element Traverse a collection of elements –Move from the front to the end –“Bubble” the largest value to the end using pair-wise comparisons and swapping 5 12 7735 42 101 1 2 3 4 5 6 Swap 1277

6. "Bubbling Up" the Largest Element Traverse a collection of elements –Move from the front to the end –“Bubble” the largest value to the end using pair-wise comparisons and swapping 5 77 1235 42 101 1 2 3 4 5 6 No need to swap

7. "Bubbling Up" the Largest Element Traverse a collection of elements –Move from the front to the end –“Bubble” the largest value to the end using pair-wise comparisons and swapping 5 77 1235 42 101 1 2 3 4 5 6 Swap 5101

8. "Bubbling Up" the Largest Element Traverse a collection of elements –Move from the front to the end –“Bubble” the largest value to the end using pair-wise comparisons and swapping 77 1235 42 5 1 2 3 4 5 6 101 Largest value correctly placed

9. Sorting Selection sort: For i=1 to n do For j=i+1 to n do If A[i]>A[j] then exchange them end Time: = O(n 2 )

10. Selecting the Smallest Element Traverse a collection of elements –Move from the front to the end –Select the smallest value using pair-wise comparisons and swapping 5 12 3542 77 101 1 2 3 4 5 6 Swap 4277

11. Selecting the Smallest Element Traverse a collection of elements –Move from the front to the end –Select the smallest value using pair-wise comparisons and swapping 5 12 3577 42 101 1 2 3 4 5 6 Swap 3542

12. Selecting the Smallest Element Traverse a collection of elements –Move from the front to the end –Select the smallest value using pair-wise comparisons and swapping 5 12 77 101 1 2 3 4 5 6 Swap 3542 12 35

13. Selecting the Smallest Element Traverse a collection of elements –Move from the front to the end –Select the smallest value using pair-wise comparisons and swapping 577 101 1 2 3 4 5 6 42 12 35 No need to swap

14. Selecting the Smallest Element Traverse a collection of elements –Move from the front to the end –Select the smallest value using pair-wise comparisons and swapping 577 101 1 2 3 4 5 6 42 12 35 Swap 512

15. Selecting the Smallest Element Traverse a collection of elements –Move from the front to the end –Select the smallest value using pair-wise comparisons and swapping 77 101 1 2 3 4 5 6 4235 512 Smallest value correctly placed

16. Sorting Insertion sort: For i=1 to n do Insert A[i+1] into appropriate (sorted) position in A[1],…,A[i] end Time: = O(n 2 )

17. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 5 12 3542 77 101 1 2 3 4 5 6

18. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 5 12 3577 42 101 1 2 3 4 5 6

19. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 5 12 3577 42 101 1 2 3 4 5 6

20. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 5 12 3577 42 101 1 2 3 4 5 6

21. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 5 12 7735 42 101 1 2 3 4 5 6

22. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 5 12 7742 35 101 1 2 3 4 5 6

23. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 5 12 7742 35 101 1 2 3 4 5 6

24. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 5 77 1242 35 101 1 2 3 4 5 6

25. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 5 77 4212 35 101 1 2 3 4 5 6

26. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 5 77 4235 12 101 1 2 3 4 5 6

27. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 5 77 4235 12 101 1 2 3 4 5 6

28. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 5 77 4235 12 101 1 2 3 4 5 6

29. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 5 77 4235 12 101 1 2 3 4 5 6

30. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 5 77 4235 12 101 1 2 3 4 5 6

31. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 101 77 4235 12 5 1 2 3 4 5 6

32. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 101 5 4235 12 77 1 2 3 4 5 6

33. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 101 42 535 12 77 1 2 3 4 5 6

34. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 101 42 35 5 12 77 1 2 3 4 5 6

35. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 101 42 35 12 5 77 1 2 3 4 5 6

36. Keeping Prefix Sorted Traverse a collection of elements –Move from the front to the end –Keep the prefix sorted throughout 101 42 35 12 5 77 1 2 3 4 5 6

37. Complexity The Time of all algorithms we saw: O(n 2 ) Can we do better?

38. Merge Sort Based on the Merging operation. Given two sorted arrays: A[1],…,A[n] and B[1],…,B[m]. merge them into one sorted array: C[1],…,C[n+m]

39. Merging Pa, Pb, Pc <- 1 While (Pa < n+1 and Pb < m+1) If A[Pa] ≤ B[Pb] thenC[Pc] <- A[Pa] Pa <- Pa+1 Pc <- Pc+1 elseC[Pc] <- B[Pb] Pb <- Pb+1 Pc <- Pc+1 end

40. Merging If Pa=n+1 and Pb < m+1 then for i=Pb to m do C[Pc] <- B[i] Pc <- Pc +1 end If Pb=n+1 and Pa < m+1 then for i=Pa to n do C[Pc] <- A[i] Pc <- Pc +1 end end Algorithm

41. Merging 3102354 152575 A:B: C:

42. Merging 3102354 52575 1 A:B: C:

43. Merging 102354 52575 13 A:B: C:

44. Merging 102354 2575 135 A:B: C:

45. Merging 2354 2575 13510 A:B: C:

46. Merging 54 2575 1351023 A:B: C:

47. Merging 54 75 135102325 A:B: C:

48. Merging 75 13510232554 A:B: C:

49. Merging 1351023255475 A:B: C: Time: O(n+m) Linear !!!

50. Merge Sort A recursive sorting algorithm: Mergesort(A) If n=1 then Return(A) else Split A[1],…,A[n] to two length n/2 arrays: A[1],…,A[n/2] and A[n/2+1],…,A[n] Mergesort(A[1],…,A[n/2]) Mergesort(A[n/2+1],…,A[n]) Merge(A[1],…,A[n/2], A[n/2+1],…,A[n], B) A <- B Return(A)

51. Merge Sort Example 996861558358640

52. Merge Sort Example 996861558358640 996861558358640

53. Merge Sort Example 996861558358640 996861558358640 1599658358640

54. Merge Sort Example 996861558358640 996861558358640 1599658358640 996861558358640

55. Merge Sort Example 996861558358640 996861558358640 1599658358640 996861558358640 40

56. Merge Sort Example 996861558358604 40 Merge

57. Merge Sort Example 158669958350486 996861558358604 Merge

58. Merge Sort Example 615869904355886 158669958350486 Merge

59. Merge Sort Example 04615355886 99 615869904355886 Merge

60. Merge Sort Example 04615355886 99

61. Merge Sort Time The recurrence: T M (n)=2T M (n/2)+n T M (1)=1 Closed Form of T M (n) : O(n log n)

62. A Data Structures Algorithm Sort array A using AVL Trees: For i=1 to n do: Insert A[i] into AVL tree For i=1 to n do: Extract smallest element from AVL tree and put in A[i]

63. Sorting Algorithm using AVL Can we extract the smallest element from AVL tree? -- Yes. Go all the way to the left. Time: O(log n) Therefore: Sorting Algorithm Time: O(n log n)

64. Priority Queue What property of the AVL Tree did we use? Only that we can find the minimum fast. A Priority Queue is a data structure that supports the following operations: INSERT (Q,k) EXTRACTMIN(Q,m)

65. Priority Queue Implementation Heap: A binary tree where every vertex has a key. For every vertex v, key(v) ≤ key(w), where w is a child of v. Difference between heap and binary search tree: 10 530 4030 HeapBinary search tree

66. Heap Additional Heap Property: All levels of the heap are full, except possibly the last level, which is left-filled. Example:

67. Heap Properties The keys on every path from the root to a leaf are nondecreasing. Example:

68. Heap Properties The height of a heap with n elements is O(log n). Example: h < log n ≤ h+1

69. Heap Insertion Insert 6 3 35232822 82010 21 74 256

70. Heap Insertion Insert 6 3 35232822 82010 21 74 25620 6

71. Heap Insertion Insert 6 3 35232822 810 21 74 2520 6 6 7 OK

72. Heap Deletion Delete Min 3 3 35232822 810 21 4 2520 6 7 Now fix heap

73. Heap Deletion Delete Min 3 35232822 810 21 4 25 6 7 20 4

74. Heap Deletion Delete Min 3 35232822 810 21 25 6 7 20 4 10 OK

75. Heap Operations Time O(log n) Conclude: Heap of n elements can be constructed in time: O(n log n) But: We can do a lot better!!!

76. Build Heap Put all elements in a tree where all levels are full and the last level is left-filled. Construct heaps from leaves up

77. Time Analysis For trees in level h : 1 For trees in level h-1 : 2 For trees in level h-i :i For trees in level 0 :log n

78. Time Analysis Number of trees in level h:n/2 Number of trees in level h-1:n/2 2 … Number of trees in level h-i:n/2 i+1 … Number of trees in level 0 :1

79. Total Time Calculate it: Note that n/2 trees do constant time work, and another quarter need to go to two levels. Only one tree needs to fix log n levels. Formally: Need to calculate this

80. Total Time We know: for x<1. Differentiate both sides: Multiply both sides by x:

81. Total Time In our case: x = 1/2. So total time: O(n).

82. In our case: Look Ma No Pointers Our tree is almost complete. All levels are full and the last one is left-filled. So write down all keys sequentially from top to bottom and left to right.

83. Getting rid of Pointers Example: 3,4,6,21,10,7,8,22,28,23,35,25,20 3 35232822 8710 21 64 2520

84. Calculate Location node i in level j = location 2 j -1+i in array. Location i in array = node in level. 3 35232822 8710 21 64 2520

85. Node j in row i is the 2 i -1+j’s element. Its left son is the 2 i+1 -1+2(j-1)+1’s element. But 2 i+1 -1+2(j-1)+1 = 2 i+1 +2(j-1) = 2 i+1 +2j-2 = 2(2 i -1+j). Location of children row i j

86. Conclude: Location of left son of i is 2i. Location of right son of i is 2i+1. Location of father of i is: i/2,if i is even (i-1)/2,if i is odd Location of Children

87. How? Given array A of n elements: Construct a heap H A in-place. For i=n downto 1 do EXTRACTMIN (H A,v) Put v in location A[i] Endfor Time: O(n) for heap construction and O(nlog n) for loop. Heap Sort

88. Note: Array A is now sorted in non-increasing order. What do we do if we want to sort in non- decreasing order? Options: Reverse order of A in linear time in- place. Use MAX rather than MIN priority Queue. Heap Sort