Priority Queues An abstract data type (ADT) Similar to a queue Additionally, each element has a priority When one would get the next element off the queue, the highest(or lowest)-priority element is retrieved first [https://en.wikipedia.org/wiki/Priority_queue]

Two kinds of priority queues: Min priority queue. Max priority queue.

Min Priority Queue Collection of elements. Each element has a priority or key. Supports following operations: empty size insert an element into the priority queue (push) get element with min priority (top) remove element with min priority (pop)

Max Priority Queue Collection of elements. Each element has a priority or key. Supports following operations: empty size insert an element into the priority queue (push) get element with max priority (top) remove element with max priority (pop)

Complexity Of Operations Two good implementations are heaps and leftist trees. empty, size, and top => O(1) time insert (push) and remove (pop) => O(log n) time where n is the size of the priority queue Leftist trees are better than binary heaps in their ability to merge two of them into one quickly, leftist tree merge O(log n) time vs binary heap merge O(n) time

Applications Sorting use element key as priority insert elements to be sorted into a priority queue remove/pop elements in priority order if a min priority queue is used, elements are extracted in ascending order of priority (or key) if a max priority queue is used, elements are extracted in descending order of priority (or key)

Sorting Example Sort five elements whose keys are 6, 8, 2, 4, 1 using a max priority queue. Insert the five elements into a max priority queue. Do five remove max operations placing removed elements into the sorted array from right to left. (Assuming O(logn) time per each operation)

After Inserting Into Max Priority Queue 8 4 6 Max Priority Queue 1 2 Sorted Array

After First Remove Max Operation 4 6 Max Priority Queue 1 2 8 Sorted Array

After Second Remove Max Operation 4 Max Priority Queue 1 2 6 8 Sorted Array

After Third Remove Max Operation Max Priority Queue 1 2 4 6 8 Sorted Array

After Fourth Remove Max Operation Max Priority Queue 1 2 4 6 8 Sorted Array

After Fifth Remove Max Operation Max Priority Queue 1 2 4 6 8 Sorted Array

Complexity Of Sorting Sort n elements. n insert operations => O(n log n) time. n remove max operations => O(n log n) time. total time is O(n log n). compare with O(n2) for sort methods of Chapter 2. (e.g. insertion sort)

Heap Sort Uses a max priority queue that is implemented as a heap. Initial insert operations are replaced by a heap initialization step that takes O(n) time. In-place, not stable In-place: O(1) auxiliary space

Machine Scheduling m identical machines (drill press, cutter, sander, etc.) n jobs/tasks to be performed assign jobs to machines so that the time at which the last job completes is minimum Another example: m = number of cores , n = number of processes

Machine Scheduling Example 3 machines and 7 jobs job times are [6, 2, 3, 5, 10, 7, 14] possible schedule 6 13 A Example schedule is constructed by scheduling the jobs in the order they appear in the given job list (left to right); each job is scheduled on the machine on which it will complete earliest. 2 7 21 B 3 13 C time ----------->

Machine Scheduling Example 6 13 A 2 7 21 B 3 13 C time -----------> Finish time = 21 Objective: Find schedules with minimum finish time.

LPT Schedules Longest Processing Time first. Jobs are scheduled in the order 14, 10, 7, 6, 5, 3, 2 Each job is scheduled on the machine on which it finishes earliest.

LPT Schedule [14, 10, 7, 6, 5, 3, 2] Finish time is 16! 14 16 A 10 15 B 7 13 16 C Finish time is 16!

LPT Schedule LPT rule does not guarantee minimum finish time schedules. (LPT Finish Time)/(Minimum Finish Time) <= 4/3 - 1/(3m) where m is number of machines. Usually LPT finish time is much closer to minimum finish time. Minimum finish time scheduling is NP-hard.

NP-hard Problems Infamous class of problems for which no one has developed a polynomial time algorithm. That is, no algorithm whose complexity is O(nk) for any constant k is known for any NP-hard problem. The class includes thousands of real-world problems. Highly unlikely that any NP-hard problem can be solved by a polynomial time algorithm. NP-hard = non-deterministic polynomial-time hard (at least as hard as the hardest problems in NP) Given the answer, it can be verified in polynomial time. (quickly checkable) e.g. Subset sum problem is NP-complete [https://en.wikipedia.org/wiki/P_versus_NP_problem]

NP-hard Problems Since even polynomial time algorithms with degree k > 3 (say) are not practical for large n, we must change our expectations of the algorithm that is used. Usually develop fast heuristics for NP-hard problems. Algorithm that gives a solution close to best. Approximation algorithms Runs in acceptable amount of time. LPT rule is a good heuristic for minimum finish time scheduling.

Complexity Of LPT Scheduling Sort jobs into decreasing order of task time. O(n log n) time (n is number of jobs) Schedule jobs in this order. assign job to machine that becomes available first must find minimum of m (m is number of machines) finish times takes O(m) time using simple strategy So need O(mn) time to schedule all n jobs. O(n log n + mn) = O(mn) (mn dominates n log n)

Using A Min Priority Queue Min priority queue has the finish times of the m machines. Initial finish times are all 0. To schedule a job remove machine with minimum finish time from the priority queue. Update the finish time of the selected machine and insert the machine back into the priority queue.

Using A Min Priority Queue m put operations to initialize priority queue 1 remove min and 1 insert to schedule each job each insert and remove min operation takes O(log m) time time to schedule is O(n log m) overall time is O(n log n + n log m) = O(n log (mn)) If m is big, the new time is much faster

Huffman Codes A particular type of optimal prefix code. Useful in lossless compression. May be used in conjunction with LZW method. Read from text.

Min Tree Definition Each tree node has a value. Heap property: Value in any node is the minimum value in the subtree for which that node is the root. Equivalently, no descendent has a smaller value.

Root has minimum element. Min Tree Example 2 4 9 3 4 8 7 9 9 Root has minimum element.

Root has maximum element. Max Tree Example 9 4 9 8 4 2 7 3 1 Root has maximum element.

Min (binary) Heap Definition complete binary tree min tree

Complete binary tree with 9 nodes. Min Heap With 9 Nodes Complete binary tree with 9 nodes.

Complete binary tree with 9 nodes that is also a min tree. Min Heap With 9 Nodes 2 4 6 7 9 3 8 Complete binary tree with 9 nodes that is also a min tree.

Complete binary tree with 9 nodes that is also a max tree. Max Heap With 9 Nodes 9 8 6 7 2 5 1 Complete binary tree with 9 nodes that is also a max tree.

Heap Height Since a heap is a complete binary tree, the height of an n node heap is log2 (n+1). Since height must be an integer, the precise expression for height is ceiling(log2 (n+1)).

A Heap Is Efficiently Represented As An Array 9 8 6 7 2 5 1 9 8 7 6 7 2 6 5 1 1 2 3 4 5 6 7 8 9 10

Moving Up And Down A Heap 9 8 6 7 2 5 1 3 4

Inserting An Element Into A Max Heap 9 8 6 7 2 5 1 7 Complete binary tree with 10 nodes.

Inserting An Element Into A Max Heap 9 8 7 6 7 2 6 5 1 5 7 New element is 5.

Inserting An Element Into A Max Heap 9 8 7 6 7 2 6 5 Bubble-up [https://en.wikipedia.org/wiki/Binary_heap#Insert] 1 7 7 New element is 20.

Inserting An Element Into A Max Heap 9 8 7 6 2 6 5 1 7 7 7 New element is 20.

Inserting An Element Into A Max Heap 9 7 6 8 2 6 5 1 7 7 7 New element is 20.

Inserting An Element Into A Max Heap 20 9 7 6 8 2 6 5 1 7 7 7 New element is 20.

Inserting An Element Into A Max Heap 20 9 7 6 8 2 6 5 1 7 7 7 Complete binary tree with 11 nodes.

Inserting An Element Into A Max Heap 20 9 7 6 8 2 6 5 1 7 7 7 New element is 15.

Inserting An Element Into A Max Heap 20 9 7 6 2 6 5 1 7 7 8 7 8 New element is 15.

Inserting An Element Into A Max Heap 20 15 7 6 9 2 6 5 1 7 7 8 7 8 New element is 15.

Complexity Of Insert Following insertion, max element ‘bubbles up’ 8 6 7 2 5 1 20 9 15 Following insertion, max element ‘bubbles up’ Complexity is O(log n), where n is heap size.

Change Key Use the same procedure for insertion to: Increase Key of an element (MAX HEAP only) Decrease Key of an element (MIN HEAP only) The changed key ‘bubbles up’

Removing The Max Element 8 6 7 2 5 1 20 9 15 Max element is in the root.

Removing The Max Element 15 7 6 9 2 6 5 1 7 7 8 7 8 After max element is removed. Replace with last element.

Removing The Max Element 8 15 7 6 9 2 6 5 1 7 7 7 Heap property violated  Heapify! Swap with MAX of its children

Removing The Max Element 8 15 7 6 9 2 6 5 1 7 7 7 Heap property violated  Heapify! Swap with MAX of its children

Removing The Max Element 15 7 8 6 9 2 6 5 1 7 7 7 Heap property violated  Heapify! Swap with MAX of its children

Removing The Max Element 15 9 7 6 8 2 6 5 1 7 7 7 8 is bigger than its children  Done!

Removing The Max Element 15 9 7 6 8 2 6 5 1 7 7 7 Max element is 15.

Complexity Of Remove Max Element 6 2 5 1 7 9 8 Heapify is a top-down approach Complexity is O(log n).