Algorithm Design Methods

Slides:



Advertisements
Similar presentations
Introduction to Algorithms
Advertisements

Unit-iv.
Algorithm Design Methods (I) Fall 2003 CSE, POSTECH.
Algorithm Design Methods Spring 2007 CSE, POSTECH.
 Review: The Greedy Method
MCS 312: NP Completeness and Approximation algorithms Instructor Neelima Gupta
Greedy Algorithms.
Chapter 3: Planning and Scheduling Lesson Plan
1.1 Data Structure and Algorithm Lecture 6 Greedy Algorithm Topics Reference: Introduction to Algorithm by Cormen Chapter 17: Greedy Algorithm.
Chapter 5 Fundamental Algorithm Design Techniques.
Greedy Algorithms Clayton Andrews 2/26/08. What is an algorithm? “An algorithm is any well-defined computational procedure that takes some value, or set.
Analysis of Algorithms
Greed is good. (Some of the time)
Chapter 4 The Greedy Approach. Minimum Spanning Tree A tree is an acyclic, connected, undirected graph. A spanning tree for a given graph G=, where E.
S. J. Shyu Chap. 1 Introduction 1 The Design and Analysis of Algorithms Chapter 1 Introduction S. J. Shyu.
Lecture 7: Greedy Algorithms II Shang-Hua Teng. Greedy algorithms A greedy algorithm always makes the choice that looks best at the moment –My everyday.
Week 2: Greedy Algorithms
Fundamental Techniques
Lecture 7: Greedy Algorithms II
Bold Stroke January 13, 2003 Advanced Algorithms CS 539/441 OR In Search Of Efficient General Solutions Joe Hoffert
IT 60101: Lecture #201 Foundation of Computing Systems Lecture 20 Classic Optimization Problems.
Introduction to Job Shop Scheduling Problem Qianjun Xu Oct. 30, 2001.
Algorithmics - Lecture 101 LECTURE 10: Greedy technique.
1 Short Term Scheduling. 2  Planning horizon is short  Multiple unique jobs (tasks) with varying processing times and due dates  Multiple unique jobs.
For Wednesday No reading No homework There will be homework for Friday, as well the program being due – plan ahead.
CSC 201: Design and Analysis of Algorithms Greedy Algorithms.
CSE 589 Part VI. Reading Skiena, Sections 5.5 and 6.8 CLR, chapter 37.
Greedy Algorithms. What is “Greedy Algorithm” Optimization problem usually goes through a sequence of steps A greedy algorithm makes the choice looks.
December 14, 2015 Design and Analysis of Computer Algorithm Pradondet Nilagupta Department of Computer Engineering.
Algorithm Design Methods 황승원 Fall 2011 CSE, POSTECH.
Greedy Algorithms BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)1.
© The McGraw-Hill Companies, Inc., Chapter 1 Introduction.
CS 361 – Chapter 10 “Greedy algorithms” It’s a strategy of solving some problems –Need to make a series of choices –Each choice is made to maximize current.
1 Ch18. The Greedy Methods. 2 BIRD’S-EYE VIEW Enter the world of algorithm-design methods In the remainder of this book, we study the methods for the.
The Theory of NP-Completeness
8.3.2 Constant Distance Approximations
Some Topics in OR.
Hard Problems Some problems are hard to solve.
Dynamic Programming Sequence of decisions. Problem state.
Lecture on Design and Analysis of Computer Algorithm
Priority Queues An abstract data type (ADT) Similar to a queue
Algorithm Design Methods
Design & Analysis of Algorithm Greedy Algorithm
Design and Analysis of Algorithm
Approximation Algorithms
Greedy Algorithms / Interval Scheduling Yin Tat Lee
Algorithm Design Methods
Heuristics Definition – a heuristic is an inexact algorithm that is based on intuitive and plausible arguments which are “likely” to lead to reasonable.
ICS 353: Design and Analysis of Algorithms
CS 3343: Analysis of Algorithms
CS4335 Design and Analysis of Algorithms/WANG Lusheng
Greedy Algorithm Enyue (Annie) Lu.
Exam 2 LZW not on syllabus. 73% / 75%.
Analysis & Design of Algorithms (CSCE 321)
Advanced Algorithms Analysis and Design
Lecture 6 Topics Greedy Algorithm
Priority Queues An abstract data type (ADT) Similar to a queue
Branch and Bound Searching Strategies
Algorithm Design Methods
Topic 15 Job Shop Scheduling.
Lecture 6 Greedy Algorithms
The Theory of NP-Completeness
The Greedy Approach Young CS 530 Adv. Algo. Greedy.
Major Design Strategies
Chapter 3: Planning and Scheduling Lesson Plan
Advance Algorithm Dynamic Programming
Chapter 1. Formulations.
Approximation Algorithm
Department of Computer Science & Engineering
Algorithm Design Methods
Presentation transcript:

Algorithm Design Methods Greedy method. Divide and conquer. Dynamic Programming. Backtracking. Branch and bound.

Some Methods Not Covered Linear Programming. Integer Programming. Simulated Annealing. Neural Networks. Genetic Algorithms. Tabu Search.

Optimization Problem A problem in which some function (called the optimization or objective function) is to be optimized (usually minimized or maximized) subject to some constraints.

Machine Scheduling Find a schedule that minimizes the finish time. optimization function … finish time constraints each job is scheduled continuously on a single machine for an amount of time equal to its processing requirement no machine processes more than one job at a time

Bin Packing Pack items into bins using the fewest number of bins. optimization function … number of bins constraints each item is packed into a single bin the capacity of no bin is exceeded

Min Cost Spanning Tree Find a spanning tree that has minimum cost. optimization function … sum of edge costs constraints must select n-1edges of the given n vertex graph the selected edges must form a tree

Feasible And Optimal Solutions A feasible solution is a solution that satisfies the constraints. An optimal solution is a feasible solution that optimizes the objective/optimization function. e.g. Earliest finish time, fewest number of bins, lowest cost, etc.

Greedy Method Solve problem by making a sequence of decisions. Decisions are made one by one in some order. Each decision is made using a greedy criterion. A decision, once made, is (usually) not changed later.

Greedy Method Make locally optimal choice at each stage Greedy algorithms are “short-sighted” Hoping to find the global optimum Can result in locally optimal solutions: Source: Public Domain, https://commons.wikimedia.org/w/index.php?curid=193687 Start1 results in sub-optimal solution Start1 end1 Start2 end2

Greedy Method Optimal substructure: A problem exhibits optimal substructure if an optimal solution to the problem contains optimal solutions to the sub-problems. * Greedy algorithms usually fail to find the optimal solution But still Useful: Usually fast and simple Often produce good approximations Source *: Introduction to Algorithms (Cormen, Leiserson, Rivest, and Stein) 2001, Chapter 16 "Greedy Algorithms". Source: https://en.wikipedia.org/wiki/Greedy_algorithm

Machine Scheduling LPT Scheduling. Schedule jobs one by one and in decreasing order of processing time. Each job is scheduled on the machine on which it finishes earliest. Scheduling decisions are made serially using a greedy criterion (minimize finish time of this job). LPT scheduling is an application of the greedy method.

LPT Schedule LPT rule does not guarantee minimum finish time schedules. (LPT Finish Time)/(Minimum Finish Time) <= 4/3 - 1/(3m) where m is number of machines Minimum finish time scheduling is NP-hard. In this case, the greedy method does not work. The greedy method does, however, give us a good heuristic for machine scheduling.

Container Loading Ship has capacity c. m containers are available for loading. Weight of container i is wi. Each weight is a positive number. Sum of container weights > c. Load as many containers as is possible without sinking the ship. Similar to truck loading.

Greedy Solution Load containers in increasing order of weight until we get to a container that doesn’t fit. Does this greedy algorithm always load the maximum number of containers? Yes. May be proved using a proof by induction (see text).

Container Loading With 2 Ships Can all containers be loaded into 2 ships whose capacity is c (each)? Same as bin packing with 2 bins. Are 2 bins sufficient for all items? Same as machine scheduling with 2 machines. Can all jobs be completed by 2 machines in c time units? NP-hard.

0/1 Knapsack Problem

0/1 Knapsack Problem Hiker wishes to take n items on a trip. The weight of item i is wi. The items are to be carried in a knapsack whose weight capacity is c. When sum of item weights <= c, all n items can be carried in the knapsack. When sum of item weights > c, some items must be left behind. Which items should be taken/left?

0/1 Knapsack Problem Hiker assigns a profit/value pi to item i. All weights and profits are positive numbers. Hiker wants to select a subset of the n items to take. The weight of the subset should not exceed the capacity of the knapsack. (constraint) Cannot select a fraction of an item. (constraint) The profit/value of the subset is the sum of the profits of the selected items. (optimization function) The profit/value of the selected subset should be maximum. (optimization criterion)

0/1 Knapsack Problem Let xi = 1 when item i is selected and let xi = 0 when item i is not selected. i = 1 n pi xi maximize i = 1 n wi xi <= c subject to Same as ship loading to maximize earnings. Each container has a weight and profit. and xi = 0 or 1 for all i

Greedy Attempt 1 Be greedy on capacity utilization. n = 2, c = 7 Select items in increasing order of weight. n = 2, c = 7 w = [3, 6] p = [2, 10] only item 1 is selected profit (value) of selection is 2 not best selection!

Greedy Attempt 2 Be greedy on profit earned. n = 3, c = 7 Select items in decreasing order of profit. n = 3, c = 7 w = [7, 3, 2] p = [10, 8, 6] only item 1 is selected profit (value) of selection is 10 not best selection!

Greedy Attempt 3 Be greedy on profit density (p/w). n = 2, c = 7 Select items in decreasing order of profit density. n = 2, c = 7 w = [1, 7] p = [10, 20] only item 1 is selected profit (value) of selection is 10 not best selection!

Greedy Attempt 3 Be greedy on profit density (p/w). n = 2, c = 7 Works when selecting a fraction of an item is permitted Select items in decreasing order of profit density, if next item doesn’t fit take a fraction so as to fill knapsack. n = 2, c = 7 w = [1, 7] p = [10, 20] item 1 and 6/7 of item 2 are selected

0/1 Knapsack Greedy Heuristics Select a subset with <= k items. If the weight of this subset is > c, discard the subset. If the subset weight is <= c, fill as much of the remaining capacity as possible by being greedy on profit density. Try all subsets with <= k items and select the one that yields maximum profit.

0/1 Knapsack Greedy Heuristics (best value - greedy value)/(best value) <= 1/(k+1) Number of solutions (out of 600) within x% of best.

0/1 Knapsack Greedy Heuristics First sort into decreasing order of profit density. There are O(nk) subsets with at most k items. Trying a subset takes O(n) time. Total time is O(nk+1) when k > 0. (best value - greedy value)/(best value) <= 1/(k+1)