Chapter 8 Dynamic Programming Copyright © 2007 Pearson Addison-Wesley. All rights reserved.

Slides:



Advertisements
Similar presentations
Dynamic Programming Dynamic Programming is a general algorithm design technique for solving problems defined by recurrences with overlapping subproblems.
Advertisements

Lecture 21: Matrix Operations and All-pair Shortest Paths Shang-Hua Teng.
Discussion #36 Spanning Trees
3 -1 Chapter 3 The Greedy Method 3 -2 The greedy method Suppose that a problem can be solved by a sequence of decisions. The greedy method has that each.
Spanning Trees.
Chapter 8 Dynamic Programming Copyright © 2007 Pearson Addison-Wesley. All rights reserved.
Chapter 9: Greedy Algorithms The Design and Analysis of Algorithms.
© 2004 Goodrich, Tamassia Directed Graphs1 JFK BOS MIA ORD LAX DFW SFO.
Greedy Algorithms Reading Material: Chapter 8 (Except Section 8.5)
Design and Analysis of Algorithms - Chapter 81 Dynamic Programming Dynamic Programming is a general algorithm design technique Dynamic Programming is a.
Lecture 22: Matrix Operations and All-pair Shortest Paths II Shang-Hua Teng.
Chapter 9 Greedy Technique Copyright © 2007 Pearson Addison-Wesley. All rights reserved.
1 Directed Graphs CSC401 – Analysis of Algorithms Lecture Notes 15 Directed Graphs Objectives: Introduce directed graphs and weighted graphs Present algorithms.
Chapter 8 Dynamic Programming Copyright © 2007 Pearson Addison-Wesley. All rights reserved.
Directed Graphs1 JFK BOS MIA ORD LAX DFW SFO. Directed Graphs2 Outline and Reading (§6.4) Reachability (§6.4.1) Directed DFS Strong connectivity Transitive.
Minimum Spanning Trees1 JFK BOS MIA ORD LAX DFW SFO BWI PVD
© 2004 Goodrich, Tamassia Minimum Spanning Trees1 Minimum Spanning Trees (§ 13.7) Spanning subgraph Subgraph of a graph G containing all the vertices of.
Greedy Algorithms Like dynamic programming algorithms, greedy algorithms are usually designed to solve optimization problems Unlike dynamic programming.
Minimum Spanning Trees1 JFK BOS MIA ORD LAX DFW SFO BWI PVD
Dynamic Programming A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 8 ©2012 Pearson Education, Inc. Upper Saddle River,
1 Minimum Spanning Trees Longin Jan Latecki Temple University based on slides by David Matuszek, UPenn, Rose Hoberman, CMU, Bing Liu, U. of Illinois, Boting.
Minimum Spanning Tree By Jonathan Davis.
1 Minimum Spanning Trees Longin Jan Latecki Temple University based on slides by David Matuszek, UPenn, Rose Hoberman, CMU, Bing Liu, U. of Illinois, Boting.
Minimum Spanning Tree in Graph - Week Problem: Laying Telephone Wire Central office.
Data Structures and Algorithms Graphs Minimum Spanning Tree PLSD210.
© The McGraw-Hill Companies, Inc., Chapter 3 The Greedy Method.
1 Shortest Path Problem Topic 11 ITS033 – Programming & Algorithms C B A E D F Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program,
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 8 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.
Chapter 2 Graph Algorithms.
CSCE350 Algorithms and Data Structure Lecture 17 Jianjun Hu Department of Computer Science and Engineering University of South Carolina
Dynamic Programming Dynamic Programming Dynamic Programming is a general algorithm design technique for solving problems defined by or formulated.
Introduction DATA STRUCTURE – Graph electronic circuits networks (roads, flights, communications) CS16 LAX JFK LAX DFW STL HNL FTL.
Minimum-Cost Spanning Tree CS 110: Data Structures and Algorithms First Semester,
Prim's Algorithm This algorithm starts with one node. It then, one by one, adds a node that is unconnected to the new graph to the new graph, each time.
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 9 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.
Union-find Algorithm Presented by Michael Cassarino.
1 Greedy Technique Constructs a solution to an optimization problem piece by piece through a sequence of choices that are: b feasible b locally optimal.
Trees Dr. Yasir Ali. A graph is called a tree if, and only if, it is circuit-free and connected. A graph is called a forest if, and only if, it is circuit-free.
Minimum- Spanning Trees
1 Week 3: Minimum Spanning Trees Definition of MST Generic MST algorithm Kruskal's algorithm Prim's algorithm.
1 Digraphs Reachability Connectivity Transitive Closure Floyd-Warshall Algorithm JFK BOS MIA ORD LAX DFW SFO v 2 v 1 v 3 v 4 v 5 v 6 v 7.
Directed Graphs1 JFK BOS MIA ORD LAX DFW SFO. Directed Graphs2 Outline and Reading (§12.4) Reachability (§12.4.1) Directed DFS Strong connectivity Transitive.
Directed Graphs Directed Graphs Shortest Path 12/7/2017 7:10 AM BOS
Minimum-cost spanning tree
Directed Graphs 12/7/2017 7:15 AM Presentation for use with the textbook Data Structures and Algorithms in Java, 6th edition, by M. T. Goodrich, R. Tamassia,
Minimum-Cost Spanning Tree and Kruskal’s Algorithm
Minimum Spanning Trees
Chapter 9 : Graphs Part II (Minimum Spanning Trees)
Greedy Technique.
May 12th – Minimum Spanning Trees
Minimum Spanning Trees
Shortest Paths C B A E D F Shortest Paths 1
Dynamic Programming Copyright © 2007 Pearson Addison-Wesley. All rights reserved.
Dynamic Programming Dynamic Programming is a general algorithm design technique for solving problems defined by recurrences with overlapping subproblems.
Chapter 8 Dynamic Programming
Kruskal’s Algorithm Elaicca Ronna Ordoña. A cable company want to connect five villages to their network which currently extends to the market town of.
Directed Graphs 9/20/2018 1:45 AM Presentation for use with the textbook Data Structures and Algorithms in Java, 6th edition, by M. T. Goodrich, R. Tamassia,
Directed Graphs 5/1/15 12:25:22 PM
Directed Graphs Directed Graphs 1 Shortest Path Shortest Path
CSCE350 Algorithms and Data Structure
Minimum Spanning Tree.
Minimum Spanning Tree.
Minimum Spanning Tree.
Chapter 13 Graph Algorithms
Minimum Spanning Trees
Directed Graphs Directed Graphs Directed Graphs 2/23/ :12 AM BOS
Minimum Spanning Tree.
Minimum spanning trees
Lecture 21: Matrix Operations and All-pair Shortest Paths
Presentation transcript:

Chapter 8 Dynamic Programming Copyright © 2007 Pearson Addison-Wesley. All rights reserved.

Dynamic Programming Dynamic Programming is a general algorithm design technique for solving problems defined by or formulated as recurrences with overlapping subinstances Invented by American mathematician Richard Bellman in the 1950s to solve optimization problems and later assimilated by CS “Programming” here means “planning” Main idea: -set up a recurrence relating a solution to a larger instance to solutions of some smaller instances - solve smaller instances once -record solutions in a table -extract solution to the initial instance from that table

Examples of DP algorithms Fibonacci numbers Longest common subsequence Spanning tree Warshall’s algorithm for transitive closure Floyd’s algorithm for all-pairs shortest paths Constructing an optimal binary search tree Some instances of difficult discrete optimization problems: - traveling salesman - - knapsack

A spanning tree of a graph is just a subgraph that contains all the vertices and is a tree. A graph may have many spanning trees. or Some Spanning Trees from Graph AGraph A Spanning Trees

All 16 of its Spanning TreesComplete Graph

Minimum Spanning Trees The Minimum Spanning Tree for a given graph is the Spanning Tree of minimum cost for that graph Complete GraphMinimum Spanning Tree

Algorithms for Obtaining the Minimum Spanning Tree Kruskal's Algorithm Prim's Algorithm Boruvka's Algorithm

Kruskal's Algorithm This algorithm creates a forest of trees. Initially the forest consists of n single node trees (and no edges). At each step, we add one edge (the cheapest one) so that it joins two trees together. If it were to form a cycle, it would simply link two nodes that were already part of a single connected tree, so that this edge would not be needed.

The steps are: 1. The forest is constructed - with each node in a separate tree. 2. The edges are placed in a priority queue. 3. Until we've added n-1 edges, 1. Extract the cheapest edge from the queue, 2. If it forms a cycle, reject it, 3. Else add it to the forest. Adding it to the forest will join two trees together. Every step will have joined two trees in the forest together, so that at the end, there will only be one tree in T.

A BC D E F G H I J Complete Graph

A BC D E F G H I J AABD BB B CD JC C E F D DH JEG FFGI GGIJ HJJI

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Sort Edges (in reality they are placed in a priority queue - not sorted - but sorting them makes the algorithm easier to visualize)

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Cycle Don’t Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Cycle Don’t Add Edge

A BC D E F G H I J B B D J C C E F D DH J EG F F G I G G I J HJ JI 1 AD 4 BC 4 AB Add Edge

A BC D E F G H I J A BC D E F G H I J Minimum Spanning TreeComplete Graph

Analysis of Kruskal's Algorithm Your First Part of Assignment: Can the Running Time of Kruskal's Algorithm be O(m log n) where (m = edges, n = nodes)? Why and why not? Testing if an edge creates a cycle can be slow unless a complicated data structure called a “union-find” data structure is used? Show how such data structure can be used to efficiently implement Kruskal's Algorithm ?

Prim's Algorithm This algorithm starts with one node. It then, one by one, adds a node that is unconnected to the new graph, each time selecting the node whose connecting edge has the smallest weight out of the available nodes’ connecting edges.

The steps are: 1. The new graph is constructed - with one node from the old graph. 2. While new graph has fewer than n nodes, 1. Find the node from the old graph with the smallest connecting edge to the new graph, 2. Add it to the new graph Every step will have joined one node, so that at the end we will have one graph with all the nodes and it will be a minimum spanning tree of the original graph.

A BC D E F G H I J Complete Graph

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Old GraphNew Graph A BC D E F G H I J

A BC D E F G H I J Complete GraphMinimum Spanning Tree A BC D E F G H I J

Analysis of Prim's Algorithm Unlike Kruskal’s, it doesn’t need to see all of the graph at once. It can deal with it one piece at a time. It also doesn’t need to worry if adding an edge will create a cycle since this algorithm deals primarily with the nodes, and not the edges. For this algorithm the number of nodes needs to be kept to a minimum in addition to the number of edges.

Analysis of Prim's Algorithm Your Second Part of Assignment: What is the Running Time of Prim's Algorithm? When it can be O(m + n log n) where m = edges, n = nodes and when it could be O(n^2)? Show in details in both cases, with example, the data structure can be used to reach the desired complexity?

Boruvka's Algorithm This algorithm is similar to Prim’s, but nodes are added to the new graph in parallel all around the graph. It creates a list of trees, each containing one node from the original graph and proceeds to merge them along the smallest-weight connecting edges until there’s only one tree, which is, of course, the MST. It works rather like a merge sort.

The steps are: 1. Make a list of n trees, each containing a single node 2. While list has more than one tree, 1. For each tree in the list, find the node not connected to the tree with the smallest connecting edge to that tree, 2. Add all the edges found to the new graph, thus creating a new set of trees Every step will have joined groups of trees, until only one tree remains.

A BC D E F G H I J Complete Graph

A B C D E F G H I J A BC D E F G H I J Trees of the Graph at Beginning of Round 1 List of Trees

A BC D E F G H I J Round A B D Tree A

A BC D E F G H I J Round A B D Edge A-D

A BC D E F G H I J A BC D J Round 1Tree B

A BC D E F G H I J A BC D J Round 1Edge B-A

A BC D E F G H I J Round BC E F Tree C

A BC D E F G H I J Round BC E F Edge C-F

A BC D E F G H I J Round A B D H J Tree D

A BC D E F G H I J Round A B D H J Edge D-A

A BC D E F G H I J Round C E G Tree E

A BC D E F G H I J Round C E G Edge E-C

A BC D E F G H I J Round C F G I Tree F

A BC D E F G H I J Round C F G I Edge F-C

A BC D E F G H I J Round E F G I J Tree G

A BC D E F G H I J Round E F G I J Edge G-E

A BC D E F G H I J Round D H J Tree H

A BC D E F G H I J Round D H J Edge H-J

A BC D E F G H I J Round F G I J Tree I

A BC D E F G H I J Round F G I J Edge I-G

A BC D E F G H I J Round B D G H I J Tree J

A BC D E F G H I J Round B D G H I J Edge J-H

A-D B-A C-F D-A E-C F-C G-E H-J I-G J-H A BC D E F G H I J Round 1 Ends - Add Edges List of Edges to Add

D-A-B F-C-E-G-I H-J List of Trees A BC D E F G H I J Trees of the Graph at Beginning of Round 2

A BC D E F G H I J Round A BC D E F G H I J Tree D-A-B

A BC D E F G H I J Round A BC D E F G H I J Edge B-C

A BC D E F G H I J Round 2Tree F-C-E-G-I A BC D E F G H I J

A BC D E F G H I J Round 2Edge I-J A BC D E F G H I J

A BC D E F G H I J A BC D E F G H I J Round 2Tree H-J

A BC D E F G H I J A BC D E F G H I J Round 2Edge J-I

B-C I-J J-I A BC D E F G H I J List of Edges to Add Round 2 Ends - Add Edges

A BC D E F G H I J A BC D E F G H I J Minimum Spanning TreeComplete Graph

Analysis of Boruvka's Algorithm Although this algorithm is difficult to explain, unlike the two preceding algorithms, it does not require a complicated data structure. Like Prim’s, it does not need to worry about detecting cycles. It does, however, need to see the whole graph, but it only examines pieces of it at a time, not all of it at once. Like Kruskal’s it is best if edges are kept to a minimum (though it doesn’t hurt to keep the nodes to a minimum as well).

Analysis of Boruvka's Algorithm Your Third Part of Assignment: What is the Running Time of Boruvka's Algorithm? Why?

The Problem: Transitive Closure B A D C E G Given a graph G, Compute the Reachability Degree of each node ? Reachability degree means : the number of reachable node from each and single node. Given a transportation network G, Is there any path from station A to station E?

Directed Graphs The Problem: Transitive Closure bGiven a digraph G, the transitive closure of G is the digraph G* such that: –G* has the same vertices as G –if G has a directed path from u to v (u  v), G* has a directed edge from u to v B A D C E B A D C E G G* bThe transitive closure provides reachability information about a digraph

Think of a method to do so?

Directed Graphs81 Computing the Transitive Closure bWe can perform DFS starting at each vertex –O(n(n+m)) If there's a way to get from A to B and from B to C, then there's a way to get from A to C. Alternatively... Use dynamic programming: the Floyd-Warshall Algorithm

Directed Graphs82 Floyd-Warshall Transitive Closure bIdea #1: Number the vertices 1, 2, …, n. bIdea #2: Consider paths that use only vertices numbered 1, 2, …, k, as intermediate vertices: k j i Uses only vertices numbered 1,…,k-1 Uses only vertices numbered 1,…,k-1 Uses only vertices numbered 1,…,k (add this edge if it’s not already in)

Directed Graphs83 Floyd-Warshall Example JFK BOS MIA ORD LAX DFW SFO v 2 v 1 v 3 v 4 v 5 v 6

Directed Graphs84 Floyd-Warshall, Iteration 1 JFK BOS MIA ORD LAX DFW SFO v 2 v 1 v 3 v 4 v 5 v 6 e.g v5 is connected to v1 and v1 is connected to v4,then connect v5 to v4

Directed Graphs85 Floyd-Warshall, Iteration 1 JFK BOS MIA ORD LAX DFW SFO v 2 v 1 v 3 v 4 v 5 v 6 In a paper complete such transitive closure graph and show it to me ?

Directed Graphs86 Floyd-Warshall, Iteration 2 JFK BOS MIA ORD LAX DFW SFO v 2 v 1 v 3 v 4 v 5 v 6

Directed Graphs87 Floyd-Warshall, Iteration 3 JFK BOS MIA ORD LAX DFW SFO v 2 v 1 v 3 v 4 v 5 v 6

Directed Graphs88 Floyd-Warshall, Iteration 4 JFK BOS MIA ORD LAX DFW SFO v 2 v 1 v 3 v 4 v 5 v 6

Directed Graphs89 Floyd-Warshall, Iteration 5 JFK MIA ORD LAX DFW SFO v 2 v 1 v 3 v 4 v 5 v 6 BOS

Directed Graphs90 Floyd-Warshall, Iteration 6 JFK MIA ORD LAX DFW SFO v 2 v 1 v 3 v 4 v 5 v 6 BOS

Directed Graphs91 Floyd-Warshall, Conclusion JFK MIA ORD LAX DFW SFO v 2 v 1 v 3 v 4 v 5 v 6 BOS

Part four of your assignment : This is the pseudo code for Floyd-Warshall algorithm. In a step by step apply it on the previous graph and report its complexity? Algorithm FloydWarshall(G) Input digraph G Output transitive closure G* of G i  1 for all v  G.vertices() denote v as v i i  i + 1 G 0  G for k  1 to n do G k  G k  1 for i  1 to n (i  k) do for j  1 to n (j  i, k) do if G k  1.areAdjacent(v i, v k )  G k  1.areAdjacent(v k, v j ) if  G k.areAdjacent(v i, v j ) G k.insertDirectedEdge(v i, v j, k) return G n