1. www.monash.edu.au 1 COMMONWEALTH OF AUSTRALIA Copyright Regulations 1969 WARNING This material has been reproduced and communicated to you by or on behalf of Monash University pursuant to Part VB of the Copyright Act 1968 (the Act). The material in this communication may be subject to copyright under the Act. Any further reproduction or communication of this material by you may be the subject of copyright protection under the Act. Do not remove this notice.

2. www.monash.edu.au FIT2004 Algorithms & Data Structures L2: Revision - Divide & Conquer Prepared by: Bernd Meyer including revision material from FIT1008 February 2007

3. www.monash.edu.au 3 Overview Divide and Conquer Merge Sort (Revision) Quick Sort (Revision) Heap Structures Heap Sort

4. www.monash.edu.au 4 Search Divide and Conquer Search Search Recall: Binary Search

5. www.monash.edu.au 5 Sort Divide and Conquer Sorting SortSort SortSortSortSort

6. www.monash.edu.au 6 Divide and Conquer Combine CombineCombine

7. www.monash.edu.au 7 Divide and Conquer method sort(array) { if (size of array > 1) { split(array, firstPart, secondPart) sort(firstPart) sort(secondPart) combine(firstPart, secondPart) }

8. www.monash.edu.au 8 Merge Sort Sort Merge Split

9. www.monash.edu.au 9 public void mergeSort(int[] array) { int[] tmp = new int[array.length]; int from = 0, to = array.length-1; mergeSort(array, from, to, tmp); } Merge Sort method

10. www.monash.edu.au 10 void mergeSort(int[] array, int from, int to, int[] tmp) { if (from!=to) { int mid = (from + to)/2; mergeSort(array, from, mid, tmp); mergeSort(array, mid+1, to, tmp); mergeArrays(array, from, mid, to, tmp); for (int i = from; i <= to; i++) {array[i] = tmp[i];} } Merge Sort method

11. www.monash.edu.au 11 351528301014224350 a: b: to: 10 tmp: Example: mergeArrays from: 0 mid: 5 32

12. www.monash.edu.au 12 515283014224350 a: tmp: ia=0 k=0 ib=6 3 Example: mergeArrays 32 b: 10

13. www.monash.edu.au 13 515283014224350 a: tmp: ia=0 k=0 3 ib=6 3 Example: mergeArrays 32 b: 10

14. www.monash.edu.au 14 315283014224350 a: tmp: ia=1ib=6 k=1 35 5 Example: mergeArrays 32 b: 10

15. www.monash.edu.au 15 35283014224350 a: 35 tmp: ia=2ib=6 k=2 10 15 Example: mergeArrays 32 10 b:

16. www.monash.edu.au 16 352830224350 a: 3510 tmp: ia=2ib=7 k=3 1510 14 Example: mergeArrays 32 14 b:

17. www.monash.edu.au 17 14 3528304350 a: 3510 tmp: ia=2ib=8 k=4 151014 15 Example: mergeArrays 32 22 b:

18. www.monash.edu.au 18 1514 3530144350 a: 3510 tmp: ia=3ib=8 k=5 151022 Example: mergeArrays 32 28 b:

19. www.monash.edu.au 19 1514 35301450 a: 3510 tmp: ia=3ib=9 k=6 151022 2822 28 Example: mergeArrays 32 43 b:

20. www.monash.edu.au 20 1514 35 50 a: 3510 tmp: ia=4ib=9 k=7 151022 3022 2843 28 Example: mergeArrays 32 30 b:

21. www.monash.edu.au 21 1514 35 50 a: 3510 tmp: ia=5ib=9 k=8 151022 3222 2843 28 30 Example: mergeArrays 32 b:

22. www.monash.edu.au 22 1514 35 50 a: 351032 tmp: ia=6ib=11 k=10 151022 2843 28 30 4350 Done! Example: mergeArrays 32 b:

23. www.monash.edu.au 23 protected void mergeArrays(int[] a, int from, int mid, int to, int[] tmp) { int ia = from, ib = mid+1; for (int k = from; k <= to; k++) { if (ia > mid) // a finished, copy b { tmp[k] = a[ib]; ib++;} else if (ib > to) // b finished, copy a { tmp[k] = a[ia]; ia++;} else if (a[ia] <= a[ib]) // a[ia] is the item to copy { tmp[k] = a[ia]; ia++;} else { tmp[k] = a[ib]; // b[ib] is the item to copy ib++;} } Example: mergeArrays

24. www.monash.edu.au 24 Merge Sort Analysis Most of the work is in the merging Takes O(n log(n)), where n is the number of elements in the array. Can you work out why? Uses more space than other sorts This is typically the method that you would naturally use when sorting a pile of books, CDs cards, etc.

25. www.monash.edu.au 25 Quick Sort x < ppp <= x Partition Sort

26. www.monash.edu.au 26 589351024153713201770 array: from:0 Example: Quick Sort to:10

27. www.monash.edu.au 27 58935142415371320770 array: “pivot element” 15 Example: Quick Sort mid = (from+to)/2 = 5

28. www.monash.edu.au 28 partition: 589351424371320770 array: 15 5893514243713207701571451315353789202470 index: 4 Example: Quick Sort

29. www.monash.edu.au 29 141337892024705357 141357378920247035 15 quickSort Example: Partition

30. www.monash.edu.au 30 void quickSort(int array[], int start, int size) { int index; if (from != to) { index = partition(array, from, to); quickSort(array, from, index); quickSort(array, index+1, to); } Quick Sort method

31. www.monash.edu.au 31 int partition(int[] array, int start, int to) { int mid = (from+to)/2; int index = from; swap(array,from,mid); for (int k = from+1; k <= to; k++) { if (array[k] < array[from]) { index++; swap(array,k,index); } swap(array,from,index); return index; } Partition method

32. www.monash.edu.au 32 Quick Sort Analysis Most of the work done in partitioning Need to be careful of choice of pivot –we will return to this in the algorithms’ analysis Average case takes O(n log(n)) time Worst case takes O(n 2 ) time

33. www.monash.edu.au 33 Summary Divide & Conquer Sorting Two Divide & Conquer Methods QuickSort –partition does (almost) all the work MergeSort –merge does (almost) all the work What is the general complexity of divide & conquer? >next lecture… Can we be any faster than this? –in terms of complexity classes? Big-Oh? >can we prove this? –how else?

34. www.monash.edu.au 34 Queues Recall: queues implement wait lists first-in-first-out principle easily implemented as lists ADT Queue sorts queue, elem, bool; ops empty:-> queue; isempty:queue-> bool; first: queue-> elem; enqueue:queue x elem -> queue; dequeue: queue-> queue; axioms … Note: this is parametric in the element type

35. www.monash.edu.au 35 Priority Queues Often you do not want FIFO but you have explicit priorities. Example: Reordering flight booking requests according to frequent flyer status New Operations “DeleteMin” extracts element with lowest priority ADT Queue sorts queue, elem, bool; ops empty:-> queue; isempty:queue-> bool; first: queue-> elem; enqueue:queue x elem -> queue; deleteMin: queue-> queue; axioms …

36. www.monash.edu.au 36 Priority Queues -- Trivial Implementation A Priority Queue can be implemented using a list and search. insert:O(1) deletemin = find + delete:O(n) can we be faster? We could use trees… you need to understand dynamic trees for this (later in unit), but it would be a bit of overkill here.

37. www.monash.edu.au 37 Heaps Use a Heap instead. A Heap is a binary tree that is completely filled with the possible exception fo the last level.

38. www.monash.edu.au 38 Heaps in Arrays As this structure is very regular, it can be easily represented in an array: parent a[I], left child a[2i], right child a[2i+1]

39. www.monash.edu.au 39 Heap Order Property To access the heap elements efficiently in priority order we need to maintain some sorted structure. Idea: DeleteMin - Minimum should be at root Order property: the parent p of every non-root node n has lower priority than n (or equal to).

40. www.monash.edu.au 40 Heap Order Property A Heap Not A Heap

41. www.monash.edu.au 41 Heap Class in Java … Why extends Comparable > ?

42. www.monash.edu.au 42 Heap Operations: Insert Idea: insert at first free position in bottom level, then let percolate upwards into correct position. (Example: insert 14)

43. www.monash.edu.au 43 Heap Operations: Insert (cont) Idea: insert at first free position in bottom level, then let percolate upwards into correct position.

44. www.monash.edu.au 44 Heap Operations: Insert

45. www.monash.edu.au 45 Heap Operations: DeleteMin Idea: Remove root, creating a whole. Re-insert last Element of heap by percolating down from root.

46. www.monash.edu.au 46 Heap Operations: DeleteMin (cont) Idea: Remove root, creating a whole. Re-insert last Element of heap by percolating down from root.

47. www.monash.edu.au 47 Heap Operations: DeleteMin (cont) Idea: Remove root, creating a whole. Re-insert last Element of heap by percolating down from root.

48. www.monash.edu.au 48 Heap Operations: DeleteMin

49. www.monash.edu.au 49 Heap Operations: DeleteMin (cont)

50. www.monash.edu.au 50 Heap Operations: DeleteMax It is easy to define a heap that has a deleteMax() operation instead of a deleteMin(). Exercise: Implement such a MaxHeap.

51. www.monash.edu.au 51 Heap Operations: Complexity Height of tree is O(log N) (complete tree has between 2 h and 2 h+1 -1 nodes) Insert: O(log N) worst case DeleteMin: O(log N) worst case

52. www.monash.edu.au 52 Heap Operations: re-ordering Other useful operations include decrease key increase key ie the re-assignment of priorities. How would you implement these? What are their complexities?

53. www.monash.edu.au 53 HeapSort A heap allows us to extract elements in sorted order. We can use it to sort a set of elements. Idea: 1.Build a heap of N elements 2.perform deleteMin() or deleteMax N times The deleted elements can be collected in a new array or (better) in the same array (“in situ”)

54. www.monash.edu.au 54 HeapSort - remove root element - store the deleted element (root=97) is stored in the position of the last heap element which has just been freed up (heap is one smaller) - percolate down the hole at the root

55. www.monash.edu.au 55 HeapSort in Java

56. www.monash.edu.au 56 HeapSort in Java

57. www.monash.edu.au 57 Building a Heap To create a heap from N items, we first create an arbitrary heap and then apply percolateDown() to each non-leaf node bottom- up. This effectively creaes small “sub-heaps” that are combined bottom-up. Bottom-up heap construction is more efficient than the naïve way (how?). Details see next lecture.

58. www.monash.edu.au 58 Building a Heap bottom-up

59. www.monash.edu.au 59 Building a Heap bottom-up

60. www.monash.edu.au 60 Building a Heap bottom-up

61. www.monash.edu.au 61 Building a Heap bottom-up

62. www.monash.edu.au 62 Reading Weiss, 6.1-6.3 (201-215), 254-256 Sedgewick 373-395 (part 1-4)