1. Algorithms and data structures Protected by http://creativecommons.org/licenses/by-nc-sa/3.0/hr/ 25.11.2015

2. Creative Commons n You are free to: share — copy and redistribute the material in any medium or format share — copy and redistribute the material in any medium or format adapt — remix, transform, and build upon the material adapt — remix, transform, and build upon the material n Under the following terms: Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. NonCommercial — You may not use the material for commercial purposes. NonCommercial — You may not use the material for commercial purposes. ShareAlike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original. ShareAlike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original. No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits. Text copied from http://creativecommons.org/licenses/by-nc-sa/3.0/ 25.11.2015Algorithms and data structures, FER Notices: You do not have to comply with the license for elements of the material in the public domain or where your use is permitted by an applicable exception or limitation. No warranties are given. The license may not give you all of the permissions necessary for your intended use. For example, other rights such as publicity, privacy, or moral rights may limit how you use the material. 2 / 18

3. Heap Heap sort Heap 25.11.2015

4. Basic terms Priority queue is a data structure resembling in some aspects to the ordinary queue Priority queue is a data structure resembling in some aspects to the ordinary queue Similarity to the ordinary queue - enqueue and dequeue operations are also defined Similarity to the ordinary queue - enqueue and dequeue operations are also defined Difference is in the fact that dequeued is not the element that was first enqueued, but the one with the extreme value of some attribute (i.e. highest priority) Difference is in the fact that dequeued is not the element that was first enqueued, but the one with the extreme value of some attribute (i.e. highest priority) n The priority queue can be presented in different ways: As a sorted linked list As a sorted linked list As a sorted binary tree As a sorted binary tree n The most natural and most efficient way to present it is using the structure heap 25.11.2015Algorithms and data structures, FER4 / 18

5. Heap n Heap is a complete binary tree where the nodes can be mutually compared using some ordering relation (e.g. >=) and where any node satisfies this relation toward its children (if existent). 6 75 46 13 26 31 25.11.2015Algorithms and data structures, FER5 / 18

6. Heap structure - implementation n elements to form a heap are stored like a complete binary tree. The simplest storage and access is achieved if an array is used. For semantic reasons the node numbering shall proceed from 1 to n. n elements to form a heap are stored like a complete binary tree. The simplest storage and access is achieved if an array is used. For semantic reasons the node numbering shall proceed from 1 to n. It is achieved by starting with an empty tree, satisfying the heap properties, and insertion of single members, while maintaining the heap properties It is achieved by starting with an empty tree, satisfying the heap properties, and insertion of single members, while maintaining the heap properties A new member is added to the “bottom” (leaf) of the heap and then it is compared and swapped with its parent, grandparent and so on until its compared attribute is found to be less or equal to the correposnding value of its predecessor A new member is added to the “bottom” (leaf) of the heap and then it is compared and swapped with its parent, grandparent and so on until its compared attribute is found to be less or equal to the correposnding value of its predecessor  GomiluStvori (CreateHeap) 25.11.2015Algorithms and data structures, FER6 / 18

7. Worst case analysis For the worst case scenario let us take n elements For the worst case scenario let us take n elements On the i- th level of a complete binary tree there are at most 2 i-1 nodes On the i- th level of a complete binary tree there are at most 2 i-1 nodes On all the lower levels until then there are a total of 2 i-1 - 1 nodes, for i > 1 On all the lower levels until then there are a total of 2 i-1 - 1 nodes, for i > 1 A tree with k levels contains at most 2 k -1 nodes A tree with k levels contains at most 2 k -1 nodes A tree with k-1 levels contains at most 2 k-1 -1 nodes A tree with k-1 levels contains at most 2 k-1 -1 nodes If the tree is complete, the last level has started if 2 k-1 - 1 < n ≤ 2 k - 1 If the tree is complete, the last level has started if 2 k-1 - 1 < n ≤ 2 k - 1 If follows: If follows: 2 k-1 < n + 1  (k – 1) log 2 < log (n + 1)  k < log 2 (n + 1) + 1 n + 1 ≤ 2 k  log (n+1) ≤ k log 2  log 2 (n+1) ≤ k log 2 (n+1) ≤ k < log 2 (n + 1) + 1 or k =  log 2 (n+1)  25.11.2015Algorithms and data structures, FER7 / 18

8. Example For n = 14 there are required  log 2 15  =  ln 15/ln 2  =  2.70805/0.693147  =  3.9  = 4 levels For n = 14 there are required  log 2 15  =  ln 15/ln 2  =  2.70805/0.693147  =  3.9  = 4 levels For n = 15 there are required  log 2 16  =  4  = 4 levels For n = 15 there are required  log 2 16  =  4  = 4 levels For n = 16 there are required  log 2 17  =  4.087  = 5 levels For n = 16 there are required  log 2 17  =  4.087  = 5 levels 25.11.2015Algorithms and data structures, FER8 / 18

9. Acceleration of the algorithm - I In the worst case, the while loop is repeated in proportion to the number of levels in the heap In the worst case, the while loop is repeated in proportion to the number of levels in the heap The worst case is caused by an ascending sequence of input data. The worst case is caused by an ascending sequence of input data. In that case, every new element inserted using the function insert, becomes the root, so that swapping is performed through k levels In that case, every new element inserted using the function insert, becomes the root, so that swapping is performed through k levels The execution time is then O(n log 2 n) The execution time is then O(n log 2 n) For average data, the execution time to form a heap is O(n), what is better for an order of magnitude For average data, the execution time to form a heap is O(n), what is better for an order of magnitude 25.11.2015Algorithms and data structures, FER9 / 18

10. Acceleration of the algorithm - II n To accelerate the forming of heap, an algorithm was devised starting from the leafs towards the root, level by level. n Only the root can violate the heap property, while the subtrees preserve it. n Therefore, it is necessary only to correct this single violation and the heap is restored. In the example, the function adjust performs this In the example, the function adjust performs this The heap property is satisfied for leafs; restoration of the heap property, done in the function create_heap, is necessary only for the root The heap property is satisfied for leafs; restoration of the heap property, done in the function create_heap, is necessary only for the root  GomiluPodesi (HeapAdjust) 25.11.2015Algorithms and data structures, FER10 / 18

11. n Creation of the heap for the input sequence: 10,63,18,71,7,51,32 Example - heap creation 10 63 18 7175132 10 63 51 7171832 10 71 51 6371832 71 63 51 1071832 25.11.2015Algorithms and data structures, FER11 / 18

12. Acceleration of the algorithm For n data, 2 k-1  n < 2 k, the number of levels is k =  log 2 (n+1)  For n data, 2 k-1  n < 2 k, the number of levels is k =  log 2 (n+1)  For the worst case, the number of iterations in adjust is k-i for a node at level i where can be at most 2 i-1 nodes For the worst case, the number of iterations in adjust is k-i for a node at level i where can be at most 2 i-1 nodes Execution time for create_heap is: Execution time for create_heap is: If the factor 0 is omitted and the summation order reversed, it follows: If the factor 0 is omitted and the summation order reversed, it follows: The exponent changes from 0 to k -1, while the factor changes from k -1 to 0 2 k-1  n The series sum tends towards 2 25.11.2015Algorithms and data structures, FER12 / 18

13. Acceleration of the algorithm In the worst case, the execution time of the algorithm create_heap is O(n), what is for an order of magnitude better than O(n log 2 n) for subsequent calls of insert In the worst case, the execution time of the algorithm create_heap is O(n), what is for an order of magnitude better than O(n log 2 n) for subsequent calls of insert The function create_heap requires all the elements to be known at creation time, while insert can introduce a new element at any time The function create_heap requires all the elements to be known at creation time, while insert can introduce a new element at any time The reason why heap has been designed are the functions that heap performs quickly: the insertion of new elements and removal of the largest one from the data set. The reason why heap has been designed are the functions that heap performs quickly: the insertion of new elements and removal of the largest one from the data set. Removal of the largest element is performed by deleting of the root and by calling of the function adjust, while insertion of new elements is performed with the function insert Removal of the largest element is performed by deleting of the root and by calling of the function adjust, while insertion of new elements is performed with the function insert – In this way both functions are performed in time O(log 2 n) 25.11.2015Algorithms and data structures, FER13 / 18

14. Variation of the algorithm n A heap can be formed for different relationships among their elements: The heap with relationship larger than is called max heap The heap with relationship larger than is called max heap The heap with relationship smaller than is called min heap The heap with relationship smaller than is called min heap 25.11.2015Algorithms and data structures, FER14 / 18

15. Heap sort n Heap sort: Element from the heap root is swapped with the last element in the array Element from the heap root is swapped with the last element in the array The array length is decreased by 1, it is adjusted and becomes a heap again The array length is decreased by 1, it is adjusted and becomes a heap again Adjustment complexity is O(log 2 n) Adjustment complexity is O(log 2 n) It is repeated n times so that the sort complexity is O(nlog 2 n) It is repeated n times so that the sort complexity is O(nlog 2 n) Difference in execution times for different orders of magnitude in complexity becomes significant for a large n Difference in execution times for different orders of magnitude in complexity becomes significant for a large n  Sortovi (Sorting) 25.11.2015Algorithms and data structures, FER15 / 18

16. Exercises n A sequence of input data is given: 12, 15, 5, 3, 7, 2 18, 11, 4, 10 a) Make a sketch of a complete binary tree created by consecutive input of the given sequence of numbers b) Make a sketch of the data structure heap containing the above input data c) What is the a priori execution time for transformation of a complete binary tree into the structure heap? 25.11.2015Algorithms and data structures, FER16 / 18

17. Exercises A sequence of input data of type int is given: A sequence of input data of type int is given: 12, 5, 4, 10, 7, 8, 11 a) Make a sketch of the tree representing the structure heap, such to enable the solution of the problem under b) b) Illustrate the functioning of descending sort using the structure heap ( heap sort ) c) What is the a priori execution time for sorting of n records? 25.11.2015Algorithms and data structures, FER17 / 18

18. Exercises Write a program to find the k -th largest array member in an integer array of n members. Write a program to find the k -th largest array member in an integer array of n members. a) Sort the input array in descending order and print the member with index k-1 (or k if heap sort was used). b) Input k array members, sort them in descending order. Start reading the remaining array members. If some member is smaller than the one with index k-1 (or k ), ignore it, if it is larger, insert it at the right place and throw out the member that would now have the index k (or k+1 ). c) Use different sorting methods and determine the corresponding a priori execution times and measure the a posteriori execution times. 25.11.2015Algorithms and data structures, FER18 / 18