Analysis of Algorithms CS 477/677

Slides:



Advertisements
Similar presentations
5/5/20151 Analysis of Algorithms Lecture 6&7: Master theorem and substitution method.
Advertisements

Comp 122, Spring 2004 Divide and Conquer (Merge Sort)
September 12, Algorithms and Data Structures Lecture III Simonas Šaltenis Nykredit Center for Database Research Aalborg University
Divide-and-Conquer Recursive in structure –Divide the problem into several smaller sub-problems that are similar to the original but smaller in size –Conquer.
11 Computer Algorithms Lecture 6 Recurrence Ch. 4 (till Master Theorem) Some of these slides are courtesy of D. Plaisted et al, UNC and M. Nicolescu, UNR.
The master method The master method applies to recurrences of the form T(n) = a T(n/b) + f (n), where a  1, b > 1, and f is asymptotically positive.
A little bit of recurrences here.. From CLRS. Dr. M. Sakalli, Marmara University Picture May 5 th 2006, RPI, NY.
Data Structures, Spring 2004 © L. Joskowicz 1 Data Structures – LECTURE 3 Recurrence equations Formulating recurrence equations Solving recurrence equations.
Analysis of Algorithms CS 477/677 Midterm Exam Review Instructor: George Bebis.
Data Structures, Spring 2006 © L. Joskowicz 1 Data Structures – LECTURE 3 Recurrence equations Formulating recurrence equations Solving recurrence equations.
CS 253: Algorithms Chapter 4 Divide-and-Conquer Recurrences Master Theorem Credit: Dr. George Bebis.
Updates HW#1 has been delayed until next MONDAY. There were two errors in the assignment Merge sort runs in Θ(n log n). Insertion sort runs in Θ(n2).
4.Recurrences Hsu, Lih-Hsing. Computer Theory Lab. Chapter 4P.2 Recurrences -- Substitution method Recursion-tree method Master method.
Recurrences Part 3. Recursive Algorithms Recurrences are useful for analyzing recursive algorithms Recurrence – an equation or inequality that describes.
Recurrence Examples.
Recurrences The expression: is a recurrence. –Recurrence: an equation that describes a function in terms of its value on smaller functions Analysis of.
Chapter 4: Solution of recurrence relationships
Analysis of Algorithms CS 477/677 Recurrences Instructor: George Bebis (Appendix A, Chapter 4)
Recurrences The expression: is a recurrence. –Recurrence: an equation that describes a function in terms of its value on smaller functions BIL741: Advanced.
October 1, Algorithms and Data Structures Lecture III Simonas Šaltenis Nykredit Center for Database Research Aalborg University
Analysis of Algorithms
Analyzing Recursive Algorithms A recursive algorithm can often be described by a recurrence equation that describes the overall runtime on a problem of.
MCA 202: Discrete Mathematics Instructor Neelima Gupta
1 Computer Algorithms Lecture 7 Master Theorem Some of these slides are courtesy of D. Plaisted, UNC and M. Nicolescu, UNR.
Project 2 due … Project 2 due … Project 2 Project 2.
DR. NAVEED AHMAD DEPARTMENT OF COMPUTER SCIENCE UNIVERSITY OF PESHAWAR LECTURE-5 Advance Algorithm Analysis.
10/25/20151 CS 3343: Analysis of Algorithms Lecture 6&7: Master theorem and substitution method.
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 3.
Introduction to Algorithms Chapter 4: Recurrences.
Chapter 4. Recurrences. Outline Offers three methods for solving recurrences, that is for obtaining asymptotic bounds on the solution In the substitution.
Recurrences David Kauchak cs161 Summer Administrative Algorithms graded on efficiency! Be specific about the run times (e.g. log bases) Reminder:
Divide-and-Conquer UNC Chapel HillZ. Guo. Divide-and-Conquer It’s a technique instead of an algorithm Recursive in structure – Divide the problem into.
1 Algorithms CSCI 235, Fall 2015 Lecture 6 Recurrences.
Recurrences – II. Comp 122, Spring 2004.
Solving Recurrences with the Substitution Method.
Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master.
Divide and Conquer. Recall Divide the problem into a number of sub-problems that are smaller instances of the same problem. Conquer the sub-problems by.
1Computer Sciences. 2 GROWTH OF FUNCTIONS 3.2 STANDARD NOTATIONS AND COMMON FUNCTIONS.
1Computer Sciences Department. Objectives Recurrences.  Substitution Method,  Recursion-tree method,  Master method.
COSC 3101A - Design and Analysis of Algorithms 3 Recurrences Master’s Method Heapsort and Priority Queue Many of the slides are taken from Monica Nicolescu’s.
Introduction to Algorithms (2 nd edition) by Cormen, Leiserson, Rivest & Stein Chapter 4: Recurrences.
Master Method Some of the slides are from Prof. Plaisted’s resources at University of North Carolina at Chapel Hill.
Recurrences (in color) It continues…. Recurrences When an algorithm calls itself recursively, its running time is described by a recurrence. When an algorithm.
BY Lecturer: Aisha Dawood. A recurrence is a function is defined in terms of:  one or more base cases, and  itself, with smaller arguments. 2.
Chapter 4: Solution of recurrence relationships Techniques: Substitution: proof by induction Tree analysis: graphical representation Master theorem: Recipe.
Recurrences It continues… Jeff Chastine. Recurrences When an algorithm calls itself recursively, its running time is described by a recurrence. A recurrence.
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 4.
Equal costs at all levels
Analysis of Algorithms CS 477/677
Recursion Ali.
Mathematical Foundations (Solving Recurrence)
Analysis of Algorithms CS 477/677
Introduction to Algorithms: Recurrences
Lecture 11. Master Theorem for analyzing Recursive relations
Chapter 4: Divide and Conquer
CS 3343: Analysis of Algorithms
Advance Analysis of Lecture No 9 Institute of Southern Punjab Multan
Introduction to Algorithms 6.046J
Algorithms and Data Structures Lecture III
Richard Anderson Lecture 11 Recurrences
CS 3343: Analysis of Algorithms
Divide and Conquer (Merge Sort)
Using The Master Method Case 1
Analysis of Algorithms
Algorithms Recurrences.
CS200: Algorithm Analysis
Algorithms CSCI 235, Spring 2019 Lecture 6 Recurrences
Analysis of Algorithms CS 477/677
Richard Anderson Lecture 12 Recurrences
Algorithms and Data Structures Lecture III
Presentation transcript:

Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 4

Methods for Solving Recurrences Iteration method Substitution method Recursion tree method Master method CS 477/677 - Lecture 4

The recursion-tree method Convert the recurrence into a tree: Each node represents the cost incurred at that level of recursion Sum up the costs of all levels Used to “guess” a solution for the recurrence CS 477/677 - Lecture 4

Example 1 W(n) = 2W(n/2) + n2 Subproblem size at level i is: n/2i Subproblem size hits 1 when 1 = n/2i  i = lgn Cost of the problem at level i = (n/2i)2 No. of nodes at level i = 2i Total cost:  W(n) = O(n2) CS 477/677 - Lecture 4

Example 2 E.g.: T(n) = 3T(n/4) + cn2 Subproblem size at level i is: n/4i Subproblem size hits 1 when 1 = n/4i  i = log4n Cost of a node at level i = c(n/4i)2 Number of nodes at level i = 3i  last level has 3log4n = nlog43 nodes Total cost:  T(n) = O(n2) CS 477/677 - Lecture 4

Example 2 - Substitution T(n) = 3T(n/4) + cn2 Guess: T(n) = O(n2) Induction goal: T(n) ≤ dn2, for some d and n ≥ n0 Induction hypothesis: T(n/4) ≤ d (n/4)2 Proof of induction goal: ≤ 3d (n/4)2 + cn2 = (3/16) d n2 + cn2 = d n2 + cn2 + (3/16) d n2 - d n2 = d n2 + cn2 - (13/16) d n2 ≤ d n2 if: cn2 - (13/16) d n2 ≤ 0 d ≥ (16/13)c Therefore: T(n) = O(n2) CS 477/677 - Lecture 4

Example 3 W(n) = W(n/3) + W(2n/3) + n The longest path from the root to a leaf is: n  (2/3)n  (2/3)2 n  …  1 Subproblem size hits 1 when 1 = (2/3)in  i=log3/2n Cost of the problem at level i = n Total cost:  W(n) = O(nlgn) for all levels CS 477/677 - Lecture 4

Example 3 - Substitution W(n) = W(n/3) + W(2n/3) + n Guess: W(n) = O(nlgn) Induction goal: W(n) ≤ dnlgn, for some d and n ≥ n0 Induction hypothesis: W(k) ≤ d klgk for any k < n (n/3, 2n/3) Proof of induction goal: Try it out as an exercise!! CS 477/677 - Lecture 4

Idea: compare f(n) with nlogba Master’s method “Cookbook” for solving recurrences of the form: where, a ≥ 1, b > 1, and f(n) > 0 Idea: compare f(n) with nlogba f(n) is asymptotically smaller or larger than nlogba by a polynomial factor n f(n) is asymptotically equal with nlogba CS 477/677 - Lecture 4

Master’s method “Cookbook” for solving recurrences of the form: where, a ≥ 1, b > 1, and f(n) > 0 Case 1: if f(n) = O(nlogba -) for some  > 0, then: T(n) = (nlogba) Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn) Case 3: if f(n) = (nlogba +) for some  > 0, and if af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then: T(n) = (f(n)) regularity condition CS 477/677 - Lecture 4

Why nlogba? Assume n = bk  k = logbn At the end of iterations, i = k: Case 1: If f(n) is dominated by nlogba: T(n) = (nlogbn) Case 3: If f(n) dominates nlogba: T(n) = (f(n)) Case 2: If f(n) = (nlogba): T(n) = (nlogba logn) Assume n = bk  k = logbn At the end of iterations, i = k: CS 477/677 - Lecture 4

Examples T(n) = 2T(n/2) + n a = 2, b = 2, log22 = 1 Compare nlog22 with f(n) = n  f(n) = (n)  Case 2  T(n) = (nlgn) CS 477/677 - Lecture 4

Examples a = 2, b = 2, log22 = 1 Compare n with f(n) = n2 T(n) = 2T(n/2) + n2 a = 2, b = 2, log22 = 1 Compare n with f(n) = n2  f(n) = (n1+) Case 3  verify regularity cond. a f(n/b) ≤ c f(n)  2 n2/4 ≤ c n2  c = ½ is a solution (c<1)  T(n) = (n2) CS 477/677 - Lecture 4

Examples (cont.) a = 2, b = 2, log22 = 1 Compare n with f(n) = n1/2 T(n) = 2T(n/2) + a = 2, b = 2, log22 = 1 Compare n with f(n) = n1/2  f(n) = O(n1-) Case 1  T(n) = (n) CS 477/677 - Lecture 4

Examples T(n) = 3T(n/4) + nlgn a = 3, b = 4, log43 = 0.793 Compare n0.793 with f(n) = nlgn f(n) = (nlog43+) Case 3 Check regularity condition: 3(n/4)lg(n/4) ≤ (3/4)nlgn = c f(n), c=3/4 T(n) = (nlgn) CS 477/677 - Lecture 4

Examples T(n) = 2T(n/2) + nlgn a = 2, b = 2, log22 = 1 Compare n with f(n) = nlgn seems like case 3 should apply f(n) must be polynomially larger by a factor of n In this case it is only larger by a factor of lgn CS 477/677 - Lecture 4

Readings Chapter 2 CS 477/677 - Lecture 4