Theory of Algorithms: Greedy Techniques James Gain and Edwin Blake {jgain | Department of Computer Science University of Cape Town August - October 2004

Objectives lTo introduce the mind set of greedy techniques lTo show some examples of greedy algorithms:  Change Making  Huffman Coding lTo discuss the strengths and weaknesses of greedy techniques l“Greed, for lack of a better word, is good! Greed is right! Greed works!” Gordon Grecko (Michael Douglas) in Wall Street

Greedy Algorithms lOptimization problems solved through a sequence of choices that are: 1.Feasible - satisfy problem constraints 2.Locally Optimal - best choice among all feasible options for that step 3.Irrevocable - no backing out lA Greedy grab of the best alternative, hoping that a sequence of locally optimal steps will lead to a globally optimal solution lEven if not optimal, sometimes an approximation is acceptable lNot all optimization problems can be approached in this manner!

Applications of the Greedy Strategy lOptimal solutions:  Change making  Minimum Spanning Trees (MST) - Prim’s and Kruskal’s Algorithms  Single-source shortest paths - Dijkstra’s Algorithm  Simple scheduling problems  Huffman codes lApproximations:  Traveling Salesman Problem (TSP)  Knapsack problem  Other combinatorial optimization problems

Change Making lProblem: give change for a specific amount n, with the least number of coins of the denominations d 1 > d 2 > … > d m lExample:  Smallest change for R2.54 using R5, R2, R1, 50c, 20c, 10c, 5c, 2c, 1c  R2 + 50c + 2c + 2c = R2.54 lAlgorithm:  At any step choose the coin of the largest denomination that doesn’t exceed the remaining total; Repeat lNote: Optimal for reasonable sets of coins

Minimum Spanning Tree Problem lProblem: Given n points, connect them in the cheapest way so that there is a path between any pair of points lSpanning Tree (of a connected graph G ):  A connected acyclic subgraph of G that includes all of G’s vertices. lMinimum Spanning Tree (of a weighted graph G ):  A spanning tree of G with minimum total weight lAlgorithms: Prim’s and Kruskal’s Algorithms lExample: 

Shortest Paths Problem lSingle Source Shortest Paths Problem:  Given a weighted graph G, find the shortest paths from a source vertex s to each of the other vertices lSolution:  Dijkstra’s Algorithm (a relative of Prim’s MST) for positive weights

Text Compression lVariable length encoding:  Compresses text by assigning codes of different length to characters based on their probability of occurrence  Used by Samuel Morse in constructing telegraph codes lPrefix-free Codes:  Codes are unique in that no code is a prefix for a code of another character lTree for Binary Codes:  Leaves are characters  Left edge codes 0, right edge codes 1  The code of a character is a simple walk from root to leaf

Algorithm for Huffman Coding lInvented by David Huffman as part of a class assignment while he was an undergraduate lAlgorithm:  Construct a Huffman Tree that assigns shorter strings to higher frequencies. This defines a Huffman Code 1.Initialize n one-node trees labeled with the characters of the alphabet. Record the frequency (weight) of each character in the root 2.REPEAT until a single tree is obtained: Find two trees with the smallest weight Make them the left and right sub-tree of a new tree and record the sum of their weights in the root

Example: Huffman Coding 0.55 C 0.15 A 0.4 B 0.45 C 0.15 A 0.4 B C 0.15 A 0.4 B CharCode A01 B1 C00

Notes on Huffman Coding lCompression Ratio:  Standard measure of compression  CR = (x - y) / y * 100%, where x is compressed and y is uncompressed  Typically, 20-80% in the case of Huffman Coding lYields optimal compression provided:  The probabilities of character occurrences are independent  Probabilities are known in advance

Strengths and Weaknesses of Greedy Techniques Strengths:  Intuitively simple and appealing ûWeaknesses:  Only applicable to optimization problems  Doesn’t always produce an optimal solution

Summary To-Date Introduction Fundamentals of the Analysis of Algorithm Efficiency , 2.6, 2.7 Brute Force Divide-and-Conquer , 4.5, 4.6 Decrease-and-Conquer Transform-and-Conquer6.1, 6.5, 6.6 Space and Time Tradeoffs Greedy Techniques9.4