 Greedy Algorithms

Greedy Algorithm  Greedy Algorithm - Makes locally optimal choice at each stage. - For optimization problems.  If the local optimum is a part of the global optimum, we get the global optimum.

Greedy Algorithm vs Dynamic Programming Dynamic Programming Greedy Algorithm

Knapsack Problem n items a thief’s knapsack of size W

Knapsack Problem  0-1 knapsack problem - Each item must be either taken or left behind.  Fractional knapsack problem - The thief can take fractions of items.

Knapsack Problem $60 $100$120 $135 knapsack ($6/unit)($5/unit)($4/unit)($3/unit) n = 4, W = 50

Fractional Knapsack Problem $60 $100 $120 $135 knapsack ($6/unit) ($5/unit) ($4/unit) ($3/unit) Greedy algorithm: greatest value per unit 50 $240

0-1 Knapsack Problem $60 $100 $120 $135 optimal ($6/unit) ($5/unit) ($4/unit) ($3/unit) 50 value per unit $160 $135 $220

0-1 Knapsack Problem optimal 50 value per unit $160 $135 $220 Difficult to get the optimal solution with a greedy strategy. Dynamic Programming :

Optimal Substructure vs Subproblem Solution T 1,5 T 1,6 T 2,5 T 2,6 T i,j : the solution of a subproblem A subproblem solution? A local optimum?

Greedy Algorithm vs Dynamic Programming Dynamic ProgrammingGreedy Algorithm Computes all subproblemsFind a local optimum Always finds the optimal solution May not be able to find the optimal solution Less efficientMore efficient

Optimal Substructure vs Subproblem Solution  Subproblem solution - From all subproblem solutions  Optimal substructure - With only constant number of parameters - Without subproblems or future choices Usually top-down Usually Bottom-up

Huffman Codes  A lossless data compression algorithm.  It uses variable-length code

Variable-Length Code Six characters : a, b, c, d, e, f How can we represent them with binary strings? abcdef Fixed-length

Variable-Length Code Six characters : a, b, c, d, e, f How can we represent them with binary strings? abcdef Variable-length What 0010 menas? = a a b a = a d a = c e …

Prefix Code Six characters : a, b, c, d, e, f How can we represent them with binary strings? abcdef Variable-length No codeword is a prefix of another codeword.

Prefix Code Six characters : a, b, c, d, e, f How can we represent them with binary strings? abcdef Variable-length No codeword is a prefix of another codeword.

Variable-Length Code abcdefTota l Frequency Fixed-length Variable-length Is it the optimal way?

Variable-Length Code abcdefTota l Frequency Fixed-length Variable-length Alternative?

Huffman Tree charcodefreq a00045 b00113 c01012 d01116 e1009 f1015 Fixed-length code

Huffman Tree charcodefreq a045 b10113 c10012 d11116 e11019 f11005 Variable-length code

Huffman’s Algorithm 0 0 Every non-leaf node has two children. 0 Observation 1 The longest code : at least 2 characters

Huffman’s Algorithm Observation 1 Observation 2 The longest code : at least 2 characters The longest 2 codes : the least frequent two characters 1 0

Huffman’s Algorithm Observation 1 Observation 2 Observation 3 The longest code : at least 2 characters The longest 2 codes : the least frequent two characters A non-leaf node : handled like a leaf node T T ’

Huffman’s Algorithm A non-leaf node : handled like a leaf node Observation T T ’ 1 0 Total length in T = Total length in T ‘ + 16 * 1 + (9 + 5) * 2

Huffman’s Algorithm Observation 1 Observation 2 Observation 3 The longest code : at least 2 characters The longest 2 codes : the least frequent two characters A non-leaf node : handled like a leaf node T T ’

Huffman’s Algorithm Merging two least frequent nodes.

Huffman’s Algorithm Merging two least frequent nodes. 1 0

Huffman’s Algorithm Merging two least frequent nodes

Huffman’s Algorithm Merging two least frequent nodes

Huffman’s Algorithm Merging two least frequent nodes

Huffman’s Algorithm Merging two least frequent nodes charcode a0 b101 c100 d111 e1101 f1100

Greedy Algorithm  Optimization Algorithms : finds a proper local optimum.  Not as powerful as Dynamic Programming, but simpler.  Greedy algorithms - Knapsack Problem - Huffman Code

 Graphs

What is a graph? V = { 1, 2, 3, 4, 5 } E = { {1,2}, {1,3}, {2,3}, {2,4}, {2,5}, {3,4} } G = ( V, E )

Directed and Undirected Directed graphUndirected graph

Representations of Graphs Directed graph Adjacency-list

Representations of Graphs Directed graphAdjacency-Matrix

Representations of Graphs Adjacency-Matrix Adjacency-list Adjacency ListAdjacency Matrix space|V| + |E||V| 2 Finding all edges|V| + |E||V| 2 Finding one edgenum of edges1

Adjacency Matrix of Graphs Directed graphAdjacency-Matrix

Adjacency Matrix of Graphs A = A T =

Weighted Graphs