1. Adapted from instructor resource slides
Sorting
Chapter 13
6/11/15
Adapted from instructor resource slides
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

2. Today:
Exams are still being corrected. Will post grades over the weekend, return Tuesday.
Any questions on project #2?
Review Thursday’s material (hashing/sorting)
Heaps, priority queue
Code exercises in class
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

3. Hash Tables In some situations faster search is needed
Solution is to use a hash function
Value of key field given to hash function
Location in a hash table is calculated
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

4. Hash Functions
Simple function could be to mod the value of the key by the size of the table
H(x) = x % tableSize
Note that we have traded speed for wasted space
Table must be considerably larger than number of items anticipated
Suggested to be 1.5-2x larger
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

5. Hash Functions
Observe the problem with same value returned by h(x) for different values of x
Called collisions
A simple solution is linear probing
Empty slots marked with -1
Linear search begins at collision location
Continues until empty slot found for insertion
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

6. Hash Functions When retrieving a value linear probe until found
If empty slot encountered then value is not in table
If deletions permitted
Slot can be marked so it will not be empty and cause an invalid linear probe
Ex. -1 for unused slots, -2 for slots which used to contain data
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

7. Collision Reduction Strategies
Hash table capacity
Size of table must be 1.5 to 2 times the size of the number of items to be stored
Otherwise probability of collisions is too high
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

8. Collision Reduction Strategies
Linear probing can result in primary clustering
Consider quadratic probing
Probe sequence from location i is i + 1, i – 1, i + 4, i – 4, i + 9, i – 9, …
Secondary clusters can still form
Double hashing
Use a second hash function to determine probe sequence
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

9. Collision Reduction Strategies
Chaining
Table is a list or vector of head nodes to linked lists
When item hashes to location, it is added to that linked list
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

10. Improving the Hash Function
Ideal hash function
Simple to evaluate
Scatters items uniformly throughout table
Modulo arithmetic not so good for strings
Possible to manipulate numeric (ASCII) value of first and last characters of a name
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

11. Importance of Sorting
Once a set of items is sorted, many other problems become easy
Using O(nlgn) sorting algorithms leads to sub-quadratic runtimes
Large-scale data processing would be impossible if sorting took Ω(n2) time

12. Comparison Functions
The most common (and natural) way for sorting elements
Is “Jones” the same as “jones”? What about “Jones, John” and “Jones – John”?
The comparison function determines the results of the sort

13. Equal Elements
Michael Jordon the basketball player vs. Michael Jordan the statistician
Elements with equal keys will bunch up together
Sometimes the relative order matters
A sorting algorithm which maintains this order is said to be stable
Most fast sorting algorithms are not stable….

14. Categories of Sorting Algorithms
Selection sort
Make passes through a list
On each pass reposition correctly some element (largest or smallest)
Scan list to find smallest element, put in first location, then loop again, replace position 2

15. Categories of Sorting Algorithms
Exchange sort
Systematically interchange pairs of elements which are out of order
Bubble sort does this
Largest element sinks to the back, smaller elements “bubble up”
Out of order, exchange
In order, do not exchange

16. Bubble Sort Algorithm 1. Initialize numCompares to n - 1
2. While numCompares != 0, do following
a. Set last = 1 // location of last element in a swap
b. For i = 1 to numPairs if xi > xi Swap xi and xi + 1 and set last = i
c. Set numCompares = last – 1
End while
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

17. Categories of Sorting Algorithms
Insertion sort
Repeatedly insert a new element into an already sorted list
Note this works well with a linked list implementation
All these have computing time O(n2)

18. Algorithm for Linear Insertion Sort
For i = 2 to n do the following
a. set NextElement = x[i] and x[0] = nextElement b. set j = i c. While nextElement < x[j – 1] do following set x[j] equal to x[j – 1] increment j by 1 End while
d. set x[j] equal to nextElement
End for
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

19. Example of Insertion Sort
Given list to be sorted 67, 33, 21, 84, 49, 50, 75
Note sequence of steps carried out
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

20. Quicksort A more efficient exchange sorting scheme than bubble sort
A typical exchange involves elements that are far apart
Fewer interchanges are required to correctly position an element.
Quicksort uses a divide-and-conquer strategy
A recursive approach
The original problem partitioned into simpler sub-problems,
Each sub problem considered independently.
Subdivision continues until sub problems obtained are simple enough to be solved directly
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

21. Quicksort Choose some element called a pivot
Perform a sequence of exchanges so that
All elements that are less than this pivot are to its left and
All elements that are greater than the pivot are to its right.
Divides the (sub)list into two smaller sub lists,
Each of which may then be sorted independently in the same way.
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

22. Quicksort If the list has 0 or 1 elements,
return. // the list is sorted
Else do:
Pick an element in the list to use as the pivot.
  Split the remaining elements into two disjoint groups:
SmallerThanPivot = {all elements < pivot}
LargerThanPivot = {all elements > pivot}
 Return the list rearranged as:
Quicksort(SmallerThanPivot),
pivot,
Quicksort(LargerThanPivot).
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

23. Quicksort Example
Given to sort: 75, 70, 65, , 98, 78, 100, 93, 55, 61, 81,
Select, arbitrarily, the first element, 75, as pivot.
Search from right for elements <= 75, stop at first element <75
Search from left for elements > 75, stop at first element >=75
Swap these two elements, and then repeat this process 84 68
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

24. Quicksort Example When done, swap with pivot
75, 70, 65, 68, 61, 55, 100, 93, 78, 98, 81, 84
When done, swap with pivot
This SPLIT operation placed pivot 75 so that all elements to the left were <= 75 and all elements to the right were >75.
View code for split() template
75 is now placed appropriately
Need to sort sublists on either side of 75
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

25. Quicksort Example Need to sort (independently): 55, 70, 65, 68, 61 and
100, 93, 78, 98, 81, 84
Let pivot be 55, look from each end for values larger/smaller than 55, swap
Same for 2nd list, pivot is 100
Sort the resulting sublists in the same manner until sublist is trivial (size 0 or 1)
View quicksort() recursive function
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

26. Quicksort Note visual example of a quicksort on an array etc. …
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

27. Improvements to Quicksort
Quicksort is a recursive function
stack of activation records must be maintained by system to manage recursion.
The deeper the recursion is, the larger this stack will become.
The depth of the recursion and the corresponding overhead can be reduced
sort the smaller sublist at each stage first
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

28. Improvements to Quicksort
An arbitrary pivot gives a poor partition for nearly sorted lists (or lists in reverse)
Virtually all the elements go into either SmallerThanPivot or LargerThanPivot
all through the recursive calls.
Quicksort takes quadratic time to do essentially nothing at all.
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

29. Improvements to Quicksort
Better method for selecting the pivot is the median-of-three rule,
Select the median of the first, middle, and last elements in each sublist as the pivot.
Often the list to be sorted is already partially ordered
Median-of-three rule will select a pivot closer to the middle of the sublist than will the “first-element” rule.
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

30. Comparisons of Sorts Sort of a randomly generated list of 500 items
Note: times are on 1970s hardware
Algorithm
Type of Sort
Time (sec)
Simple selection
Heapsort
Bubble sort
2 way bubble sort
Quicksort
Linear insertion
Binary insertion
Shell sort
Selection
Exchange
Insertion 69 18
165
141 6 66 37 11

31. Heaps A heap is a binary tree with properties: It is complete
Each level of tree completely filled
Except possibly bottom level (nodes in left most positions)
The key in any node dominates the keys of its children
Min-heap: Node dominates by containing a smaller key than its children
Max-heap: Node dominates by containing a larger key than its children

32. Heaps Which of the following are heaps? A B C
A is, not B…not, C would be a heap, but does not satisfy heap order A B C

33. Implementing a Heap Use an array or vector
Number the nodes from top to bottom
Number nodes on each row from left to right
Store data in ith node in ith location of array (vector)
Why is an array or vector a good choice? Why not a linked list?

34. Implementing a Heap Note the placement of the nodes in the array
42 should be 41

35. Implementing a Heap
In an array implementation children of ith node are at myArray[2*i] and myArray[2*i+1]
Parent of the ith node is at myArray[i/2]
Note: array indexed at 1

36. Basic Heap Operations Construct an empty heap
Check if the heap is empty
Insert an item
Retrieve the largest/smallest element
Remove the largest/smallest element

37. Basic Heap Operations Constructor Empty Retrieve max/min item
Set size to 0, allocate array
Empty
Check value of size
Retrieve max/min item
Return root of the binary tree, myArray[1]

38. Basic Heap Operations Insert an item Place new item at end of array
“Bubble” it up to the correct place
Interchange with parent so long as it is greater/less than its parent

39. Basic Heap Operations Delete max/min item
Max/Min item is the root, swap with last node in tree
Delete last element
Bubble the top element down until heap property satisfied
Interchange with larger of two children

40. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

41. Percolate Down Algorithm
1. Set c = 2 * r
2. While r <= n do following
a. If c < n and myArray[c] < myArray[c + 1] Increment c by 1 b. If myArray[r] < myArray[c] i. Swap myArray[r] and myArray[c] ii. set r = c iii. Set c = 2 * c else Terminate repetition End while
Also called bubble down. Exercise that understand this algorithm. Max number of repetitions is the height of the tree. myArray[r],….myArray[n] stores a semi-heap, only the value in heap[r] mail fail the heap-order condition. C is the location of the left child
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

42. Basic Heap Operations Insert an item Amounts to a percolate up routine
Place new item at end of array
Interchange with parent so long as it is greater than its parent
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

43. Heapsort Given a list of numbers in an array Convert to a heap
Stored in a complete binary tree
Convert to a heap
Begin at last node not a leaf
Apply percolated down to this subtree
Continue
Called heapify
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

44. Heapsort
Algorithm for converting a complete binary tree to a heap – called "heapify" For r = n/2 down to 1: Apply percolate_down to the subtree in myArray[r] , … myArray[n] End for
Puts largest element at root
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

45. Heapsort Now swap element 1 (root of tree) with last element
This puts largest element in correct location
Use percolate down on remaining sublist
Converts from semi-heap to heap
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

46. Heapsort Again swap root with rightmost leaf
Continue this process with shrinking sublist
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

47. Heapsort Algorithm
1. Consider x as a complete binary tree, use heapify to convert this tree to a heap
2. for i = n down to 2: a. Interchange x[1] and x[i] (puts largest element at end) b. Apply percolate_down to convert binary tree corresponding to sublist in x[1] .. x[i-1]
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

48. Priority Queue A collection of data elements Basic Operations A heap
Items stored in order by priority
Max-heap == highest priority first
Higher priority items removed ahead of lower
Basic Operations
Constructor
Insert
Find, remove smallest/largest (priority) element
Change priority
Delete an item

49. Priority Queue Useful for many applications
Process scheduling (operating systems)
Simulation of airports and computer networks
More efficient to insert into a priorty queue than to re-sort everything on new arrival

50. Exercises: Selection Sort
Given the following array, show the output of selection sort after each iteration: i
x[i] 1 4 2 7 3 11 5
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

51. Exercises: Selection Sort pseduocode? (array)
For i = 1 to n-1
Create variables smallPos = i and smallest = x[smallPos]
For j = i+1 to n-1
If x[j] < smallest //smaller element found
Set smallPos = j and smallest = x[smallPos]
Set x[smallPos] = x[i] and set x[i] = smallest
Array: 4,7,2,11,3
Apply this to previous example
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

52. Exercises: Heap Given the same array: 4,7,2,11,3 Apply HeapSort
First create heap
Apply heapify
Swap largest element with last element
Percolate down
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

53. Percolate Down Algorithm
1. Set c = 2 * r
2. While r <= n do following
a. If c < n and myArray[c] < myArray[c + 1] Increment c by 1 b. If myArray[r] < myArray[c] i. Swap myArray[r] and myArray[c] ii. set r = c iii. Set c = 2 * c else Terminate repetition End while
Also called bubble down. Exercise that understand this algorithm. Max number of repetitions is the height of the tree. myArray[r],….myArray[n] stores a semi-heap, only the value in heap[r] mail fail the heap-order condition. C is the location of the left child
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved

54. Insertion Sort? https://www.youtube.com/watch?v=DFG-XuyPYUQ
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved