CSC 380: Design and Analysis of Algorithms Dr. Curry Guinn

Quick Info Dr. Curry Guinn CIS 2015 guinnc@uncw.edu www.uncw.edu/people/guinnc 962-7937 Office Hours: MWF: 10:00am-11:00m and by appointment

Outline of today Heaps (Priority Queues) Greedy Algorithms Shortest path algorithms Dijkstra

Priority Queue A kind of queue Dequeue gets element with the highest priority Priority is based on a comparable value (key) of each object (smaller value higher priority, or higher value higher priority) Example Applications: printer -> print (dequeue) the shortest document first operating system -> run (dequeue) the shortest job first

Priority Queue (Heap) Operations deleteMin Priority Queue insert insert (enqueue) deleteMin (dequeue) smaller value higher priority find, return, and remove the minimum element peek Look at the smallest value

Implementation using Linked List Unsorted linked list insert takes O(1) time deleteMin takes O(N) time Sorted linked list insert takes O(N) time deleteMin takes O(1) time

Heap properties heap: a tree with the following two properties: 1. completeness complete tree: every level is full except possibly the lowest level, which must be filled from left to right with no leaves to the right of a missing node (i.e., a node may not have any children until all of its possible siblings exist) Heap shape:

Heap properties 2 2. heap ordering a tree has heap ordering if P < X for every element X with parent P In other words, in heaps, parents' element values are always smaller than those of their children Implies that minimum element is always the root Is every a heap a BST? Are any heaps BSTs? Are any BSTs heaps? Answer: no, it is not a BST.

Which are min-heaps? wrong! 10 20 10 20 80 10 80 20 80 40 60 85 99 40 30 15 50 700 50 700 10 10 20 80 10 40 60 85 99 20 80 20 80 50 700 40 60 99 40 60

Heap height and runtime Height of a complete tree is always log n, because it is always balanced This suggests that searches, adds, and removes will have O(log n) worst-case runtime Because of this, if we implement a priority queue using a heap, we can provide the O(log n) runtime required for the add and remove operations n-node complete tree of height h: h = log n 2h  n  2h+1 - 1

Adding to a heap When an element is added to a heap, it should be initially placed as the rightmost leaf (to maintain the completeness property) heap ordering property becomes broken! 10 10 20 80 20 80 40 60 85 99 40 60 85 99 50 700 65 50 700 65 15

Adding to a heap, cont'd. To restore heap ordering property, the newly added element must be shifted upward ("bubbled up") until it reaches its proper place Bubble up (book: "percolate up") by swapping with parent How many bubble-ups could be necessary, at most? 10 10 20 80 15 80 40 60 85 99 40 20 85 99 50 700 65 15 50 700 65 60

Heap practice problem Draw the state of the min-heap tree after adding the following elements to it: 6, 50, 11, 25, 42, 20, 104, 76, 19, 55, 88, 2

The peek operation peek on a min-heap is trivial; because of the heap properties, the minimum element is always the root peek is O(1) 10 20 80 40 60 85 99 50 700 65

Removing from a min-heap Min-heaps only support remove of the min element (the root) Must remove the root while maintaining heap completeness and ordering properties Intuitively, the last leaf must disappear to keep it a heap Initially, just swap root with last leaf (we'll fix it) 10 65 20 80 20 80 40 60 85 99 40 60 85 99 700 50 65 700 50 65

Removing from heap, cont'd. Must fix heap-ordering property; root is out of order Shift the root downward ("bubble down") until it's in place Swap it with its smaller child each time What happens if we don't always swap with the smaller child? 65 20 20 80 40 80 40 60 85 99 50 60 85 99 700 50 700 65

Implementation with a list!

Array Implementation If Parent is i, If Child is i, Left child is 2*i Right child is 2*i + 1 If Child is i, Parent is Floor(i/2) =

Optimization problems An optimization problem is one in which you want to find, not just a solution, but the best solution A “greedy algorithm” sometimes works well for optimization problems A greedy algorithm works in phases. At each phase: You take the best you can get right now, without regard for future consequences You hope that by choosing a local optimum at each step, you will end up at a global optimum 2

Shortest Path Algorithms Single-Source Shortest Path Problem Given an input a weighted graph, G = (V, E), and a distinguished vertex, s, find the shortest weighted path from s to every other vertex in G. Assume there is no edge with a negative cost (which could create negative cost cycles).

Shortest Path Algorithms Dijkstra’s Algorithm (for single-source weighted graphs) A greedy algorithm, solving a problem by stages by doing what appears to be the best thing at each stage. Select a vertex v, which has the smallest dv among all the unknown vertices, and declare that the shortest path from s to v is known.

Shortest Path Algorithms For each adjacent vertex, w, update dw = dv + cv,w if this new value for dw is an improvement.

Stages of Dijkstra's algorithm

Stages of Dijkstra's algorithm

An Animation https://www.cs.usfca.edu/~galles/visualization/Dijkstra.html

Pseudocode (not Levitin)

Runtime of Dijkstra's alg. The simplest implementation of the Dijkstra's algorithm stores vertices of set Q in an ordinary linked list or array Operation Extract-Min(Q) is simply a linear search through all vertices in Q In this case, the total running time is O(V2) For sparse graphs, that is, graphs with much less than V2 edges: Dijkstra's algorithm can be implemented more efficiently by using a priority queue to implement the removeMin function algorithm improves to O((E+V)logV) time

For Next Class, Monday (after Spring Break) Huffman coding! A compression technique Homework 5 due Tuesday after Spring Break