Introduction to Algorithms Jiafen Liu Sept. 2013.

Slides:



Advertisements
Similar presentations
Introduction to Algorithms
Advertisements

CS Section 600 CS Section 002 Dr. Angela Guercio Spring 2010.
Comp 122, Spring 2004 Greedy Algorithms. greedy - 2 Lin / Devi Comp 122, Fall 2003 Overview  Like dynamic programming, used to solve optimization problems.
CSCE 411H Design and Analysis of Algorithms Set 8: Greedy Algorithms Prof. Evdokia Nikolova* Spring 2013 CSCE 411H, Spring 2013: Set 8 1 * Slides adapted.
Greedy Algorithms Greed is good. (Some of the time)
Analysis of Algorithms
Greed is good. (Some of the time)
Chapter 4 The Greedy Approach. Minimum Spanning Tree A tree is an acyclic, connected, undirected graph. A spanning tree for a given graph G=, where E.
Introduction to Algorithms 6.046J/18.401J L ECTURE 16 Greedy Algorithms (and Graphs) Graph representation Minimum spanning trees Optimal substructure Greedy.
Lecture 7: Greedy Algorithms II Shang-Hua Teng. Greedy algorithms A greedy algorithm always makes the choice that looks best at the moment –My everyday.
CPSC 411, Fall 2008: Set 4 1 CPSC 411 Design and Analysis of Algorithms Set 4: Greedy Algorithms Prof. Jennifer Welch Fall 2008.
Greedy Algorithms Reading Material: –Alsuwaiyel’s Book: Section 8.1 –CLR Book (2 nd Edition): Section 16.1.
Greedy Algorithms Reading Material: Chapter 8 (Except Section 8.5)
Greedy Algorithms CIS 606 Spring Greedy Algorithms Similar to dynamic programming. Used for optimization problems. Idea – When we have a choice.
Greedy Algorithms Like dynamic programming algorithms, greedy algorithms are usually designed to solve optimization problems Unlike dynamic programming.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2008 Lecture 2 Tuesday, 9/16/08 Design Patterns for Optimization.
Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at
Lecture 7: Greedy Algorithms II
CPSC 411, Fall 2008: Set 4 1 CPSC 411 Design and Analysis of Algorithms Set 4: Greedy Algorithms Prof. Jennifer Welch Fall 2008.
16.Greedy algorithms Hsu, Lih-Hsing. Computer Theory Lab. Chapter 16P An activity-selection problem Suppose we have a set S = {a 1, a 2,..., a.
David Luebke 1 8/23/2015 CS 332: Algorithms Greedy Algorithms.
Analysis of Algorithms
CS 3343: Analysis of Algorithms Lecture 21: Introduction to Graphs.
Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure.
1 Greedy algorithm 叶德仕 2 Greedy algorithm’s paradigm Algorithm is greedy if it builds up a solution in small steps it chooses a decision.
Greedy Algorithms Dr. Yingwu Zhu. Greedy Technique Constructs a solution to an optimization problem piece by piece through a sequence of choices that.
Data Structures and Algorithms A. G. Malamos
 2004 SDU Lecture 7- Minimum Spanning Tree-- Extension 1.Properties of Minimum Spanning Tree 2.Secondary Minimum Spanning Tree 3.Bottleneck.
David Luebke 1 10/24/2015 CS 332: Algorithms Greedy Algorithms Continued.
CSC 413/513: Intro to Algorithms Greedy Algorithms.
Greedy Algorithms and Matroids Andreas Klappenecker.
Introduction to Algorithms Jiafen Liu Sept
CSC 201: Design and Analysis of Algorithms Greedy Algorithms.
COSC 3101A - Design and Analysis of Algorithms 8 Elements of DP Memoization Longest Common Subsequence Greedy Algorithms Many of these slides are taken.
November 13, Algorithms and Data Structures Lecture XII Simonas Šaltenis Aalborg University
 2004 SDU Lecture 6- Minimum Spanning Tree 1.The Minimum Spanning Tree Problem 2.Greedy algorithms 3.A Generic Algorithm 4.Kruskal’s Algorithm.
Greedy Algorithms Z. GuoUNC Chapel Hill CLRS CH. 16, 23, & 24.
Greedy Algorithms BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)1.
CS 3343: Analysis of Algorithms Lecture 19: Introduction to Greedy Algorithms.
Greedy Algorithms Chapter 16 Highlights
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Spring, 2010 Lecture 2 Tuesday, 2/2/10 Design Patterns for Optimization.
Data Structures and Algorithms (AT70.02) Comp. Sc. and Inf. Mgmt. Asian Institute of Technology Instructor: Prof. Sumanta Guha Slide Sources: CLRS “Intro.
Greedy Algorithms Lecture 10 Asst. Prof. Dr. İlker Kocabaş 1.
Lecture 12 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea.
Divide and Conquer. Problem Solution 4 Example.
6/13/20161 Greedy A Comparison. 6/13/20162 Greedy Solves an optimization problem: the solution is “best” in some sense. Greedy Strategy: –At each decision.
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 17.
Greedy Algorithms. p2. Activity-selection problem: Problem : Want to schedule as many compatible activities as possible., n activities. Activity i, start.
Greedy Algorithms Many of the slides are from Prof. Plaisted’s resources at University of North Carolina at Chapel Hill.
Greedy Algorithms Many of the slides are from Prof. Plaisted’s resources at University of North Carolina at Chapel Hill.
Greedy Algorithms General principle of greedy algorithm
Algorithm Analysis Fall 2017 CS 4306/03
CSCE 411 Design and Analysis of Algorithms
Introduction to Algorithms
Introduction to Algorithms`
Lecture 12 Algorithm Analysis
CS 3343: Analysis of Algorithms
Graph Algorithms Using Depth First Search
Algorithm Design Methods
CS200: Algorithm Analysis
ICS 353: Design and Analysis of Algorithms
Greedy Algorithms Many optimization problems can be solved more quickly using a greedy approach The basic principle is that local optimal decisions may.
Lecture 12 Algorithm Analysis
Advanced Algorithms Analysis and Design
Advanced Algorithms Analysis and Design
Greedy Algorithms Comp 122, Spring 2004.
Introduction to Algorithms: Greedy Algorithms (and Graphs)
The Greedy Approach Young CS 530 Adv. Algo. Greedy.
Lecture 12 Algorithm Analysis
Advance Algorithm Dynamic Programming
Presentation transcript:

Introduction to Algorithms Jiafen Liu Sept. 2013

Today’s Tasks Graphs and Greedy Algorithms –Graph representation –Minimum spanning trees –Optimal substructure –Greedy choice –Prim’s greedy MST algorithm

Graphs(for review) A directed graph (digraph) G= (V, E) is consisting of –a set V of vertices(singular: vertex), –a set E ⊆ V×V of ordered edges. In an undirected graph G= (V, E),the edge set E consists of unordered pairs of vertices. In either case, we have |E|= O(|V| 2 ). if G is connected, then |E|≥|V|–1 Both of the above 2 imply that lg|E|= Θ(lg|V|). (Review Appendix B.)

How to store a graph in computer The adjacency matrix of a graph G= (V, E), where V= {1, 2, …, n}, is the matrix A[1.. n, 1.. n] given by An adjacency list of a vertex v ∈ V is the list Adj[v] of vertices adjacent to v.

Example What’s the representation of this graph with matrix and list?

Analysis of adjacency matrix We say that, vertices are adjacent, but edges are incident on vertices. With |V| nodes, how much space the matrix takes? –Θ(|V| 2 )storage It applies to? –Dense representation

Analysis of adjacency list For undirected graphs, |Adj[v]|=degree(v). For digraphs, |Adj[v]|= out-degree(v).

Handshaking Lemma Handshaking Lemma: Σ v ∈ V degree(v)=2|E| for undirected graphs Under Handshaking Lemma, adjacency lists use how much storage? –Θ(V+E) a sparse representation (for either type of graph).

Minimum spanning trees Input: A connected, undirected graph G= (V, E) with weight function w: E→R. –For simplicity, assume that all edge weights are distinct. Output: A spanning tree T —a tree that connects all vertices —of minimum weight:

Example of MST

MST and dynamic programming MST T(Other edges of G are not shown.) What’s the connection of these two problem?

MST and dynamic programming MST T(Other edges of G are not shown.) Remove any edge (u, v) ∈ T.

MST and dynamic programming MST T(Other edges of G are not shown.) Remove any edge (u, v) ∈ T.

MST and dynamic programming MST T(Other edges of G are not shown.) Remove any edge (u, v) ∈ T. Then, T is partitioned into two subtrees T 1 and T 2.

Theorem of optimal substructure Theorem. The subtree T 1 is an MST of G 1 = (V 1, E 1 ), G 1 is a subgraph of G induced by the vertices of T 1 : V 1 =vertices of T 1 E 1 = { (x, y) ∈ E: x, y ∈ V 1 }. Similarly for T 2. How to prove? –Cut and paste

Proof of optimal substructure Proof. w(T) = w(u, v) + w(T 1 ) + w(T 2 ). If T 1 ′ were a lower-weight spanning tree than T 1 for G 1, then T′= {(u, v)} ∪ T 1 ′ ∪ T 2 would be a lower-weight spanning tree than T for G. Another hallmark of dynamic programming? Do we also have overlapping subproblems? –Yes.

MST and dynamic programming Great, then dynamic programming may work! Yes, but MST exhibits another powerful property which leads to an even more efficient algorithm.

Hallmark for “greedy” algorithms Theorem. Let T be the MST of G= (V, E), and let A ⊆ V. Suppose that (u, v) ∈ E is the least-weight edge connecting A to V–A. Then (u, v) ∈ T.

Proof of theorem Proof. Suppose (u, v) ∉ T. Cut and paste.

Proof of theorem Proof. Suppose (u, v) ∉ T. Cut and paste. Consider the unique simple path from u to v in T.

Proof of theorem Proof. Suppose (u, v) ∉ T. Cut and paste. Consider the unique simple path from u to v in T. Swap (u, v) with the first edge on this path that connects a vertex in A to a vertex in V–A.

Proof of theorem Proof. Suppose (u, v) ∉ T. Cut and paste. Consider the unique simple path from u to v in T. Swap (u, v) with the first edge on this path that connects a vertex in A to a vertex in V–A. A lighter-weight spanning tree than T !

Prim’s algorithm IDEA: Maintain V–A as a priority queue Q. Key each vertex in Q with the weight of the least-weight edge connecting it to a vertex in A. At the end, forms the MST.

Example of Prim’s algorithm

Analysis of Prim

Handshaking Lemma ⇒ Θ(E) implicit DECREASE-KEY’s.

Analysis of Prim

MST algorithms Kruskal’s algorithm (see the book): –Uses the disjoint-set data structure –Running time = O(ElgV). Best to date: –Karger, Klein, and Tarjan [1993]. –Randomized algorithm. –O(V+ E)expected time.

More Applications Activity-Selection Problem 0-1 knapsack

Activity-Selection Problem Problem: Given a set A = {a 1, a 2, …, a n } of n activities with start and finish times (s i, f i ), 1 ≤ i ≤ n, select maximal set S of non- overlapping activities.

Activity Selection Here is the problem from the book: –Activity a 1 starts at s 1 = 1, finishes at f 1 = 4. –Activity a 2 starts at s 2 = 3, finishes at f 2 = 5. Got the idea? –The set S is sorted in monotonically increasing order of finish time.The subsets of {a 3, a 9, a 11 } and {a 1, a 4, a 8, a 11 }are mutually compatible.

Software School of XiDian University Activity Selection

Objective: to create a set of maximum activities a i that are compatible. –Modeling the subproblems: Create a set of activities that can start after a i finishes, and finish before activity a j starts. –Let S ij be that set of activities. S ij = { a k ∈ S : f i ≤ s k < f k ≤ s j } ⊆ S where S is the complete set of activities, only one of which can be scheduled at any particular time. (They share a resource, which happens, in a way, to be a room).

Property of S ij We add fictitious activities a 0 and a n+1 and adopt the conventions that f 0 = 0, s n+1 = ∞. Assume activities are sorted by increasing finishing times:f 0 ≤ f 1 ≤ f 2 ≤... ≤ f n < f n+1. Then S = S 0, n+1, for 0 ≤ i, j ≤ n+1. Property 1. S ij = φ if i ≥ j. –Proof Suppose i ≥ j and S ij ≠ φ, then there exists a k such that f i ≤ s k < f k ≤ s j < f j, therefore f i < f j. But i ≥ j implies that f i ≥ f j, that’s a contradiction.

Optimal Substructure The optimal substructure of this problem is as follows: –Suppose we have an optimal solution A ij to S ij include a k, i.e., a k ∈ S ij. Then the activities that make up S ij can be represented by A ij = A ik ∪ {a k } ∪ A kj We apply cut-and-paste argument to prove the optimal substructure property.

A recursive solution Define c[i, j] as the number of activities in a maximum-size subset of mutually compatible activities. Then Converting a dynamic-programming solution to a greedy solution. –We may still write a tabular, bottom-up, dynamic programming algorithm based on recurrence. –But we can obtain a simplified solution by transforming a dynamic-programming solution into a greedy solution.

Greedy choice property Theorem 16.1 Consider any nonempty subproblem S ij, and let a m be the activity in S ij with the earliest finish time: f m = min{ f k : a k ∈ S ij }. – Activity a m is used in some maximum-size subset of mutually compatible activities of S ij. –The subproblem S im is empty, so that choosing a m leaves the subproblem S mj as the only one that may be nonempty.

A recursive greedy algorithm The procedure R ECURSIVE -A CTIVITY -S ELECTION is almost “tail recursive”: it ends with a recursive call to itself followed by a union operation. It is a straightforward task to transform a tail-recursive procedure to an iterative form.

An iterative greedy algorithm This procedure schedules a set of n activities in Θ(n) time, assuming that the activities were already sorted initially by their finish times.

0-1 knapsack A thief robs a store and finds n items, with item i being worth $v i and having weight w i pounds, The thief can carry at most W ∈ N in his knapsack but he wants to take as valuable a load as possible. Which item should he take? i.e., is maximized and s.t.

Fractional knapsack problem The setup is the same, but the thief can take fractions of items.

Optimal substructure Do they exhibit the optimal substructure property? –For 0-1 knapsack problem, if we remove item j from this load, the remaining load must be at most valuable items weighing at most W-w j from n -1 items. –For fractional one, if we remove a weight w of one item j from the optimal load, then the remaining load must be the most valuable load weighing at most W -w that the thief can take from the n-1 original items plus w j –w pounds of item j.

Different solving method Can Fractional knapsack problem use Greedy Choice? –Yes How?

Can Fractional knapsack problem use Greedy Choice? –No WHY? Then, How? Different solving method

Optimal Substructure (0/1sack) Objectives: –Let c[i, j] represents the total value that can be taken from the first i items when the knapsack can hold j. –Our Objective is to get the maximum value for c[n, W] where n is the number of given items and W is the maximum weight of items that the thief can put it in his knapsack. Optimal Substructure –If an optimal solution contains item n, the remaining choices must constitute an optimal solution to similar problem on items 1, 2, …,n –1 with bound W–w n. –If an optimal solution does not contain item n, the solution must also be an optimal solution to similar problem on items 1, 2, …, n –1 with bound W.

0-1 Knapsack problem Let c[i, j] denote the maximum value of the objects that fit in the knapsack, selecting objects from 1 through i with the sack’s weight capacity equal to j. An application of the principle of optimality we have derived the following recurrence for c[i, j]: c[i, j] = max(c[i –1, j], c[i –1, j – w i ] + vi) The boundary conditions are c[0, j] = 0 if j ≥ 0, and c[i, j] = −∞ when j < 0.

0-1 Knapsack problem There are n = 5 objects with integer weights w[1..5] = {1, 2, 5, 6, 7}, and values v[1..5] = {1, 6, 18, 22, 28}. The following table shows the computations leading to c[5,11] (i.e., assuming a knapsack capacity of 11). Sack’s Capacity wiwi vivi c[4, 8] =max (c[3,8], 22 + c[3, 2]) c[3, 8] c[3, 2]

Further Thought Connection and Difference : Divide and Conquer –Subproblems are independent Dynamic Programming Greedy Algorithm –The same direction –Optimal Substructure –Overlapping Subproblems

When Greedy Algorithm Works? There is no way in general. If we can demonstrate the following properties, then it is probable to use greedy algorithm: Greedy-choice property –a global optimal solution can be arrived at by making local optimal (greedy) choice Optimal substructure (the same with that of dynamic programming) –if an optimal solution to the problem contains within it optimal solutions to subproblems