Trevor Brown trevor.brown@uwaterloo.ca CS 341: Algorithms Trevor Brown trevor.brown@uwaterloo.ca.

Slides:



Advertisements
Similar presentations
Divide-and-Conquer 7 2  9 4   2   4   7
Advertisements

5/5/20151 Analysis of Algorithms Lecture 6&7: Master theorem and substitution method.
Comp 122, Spring 2004 Divide and Conquer (Merge Sort)
September 12, Algorithms and Data Structures Lecture III Simonas Šaltenis Nykredit Center for Database Research Aalborg University
Divide-and-Conquer Recursive in structure –Divide the problem into several smaller sub-problems that are similar to the original but smaller in size –Conquer.
Divide and Conquer. Recall Complexity Analysis – Comparison of algorithm – Big O Simplification From source code – Recursive.
Algorithms Recurrences. Definition – a recurrence is an equation or inequality that describes a function in terms of its value on smaller inputs Example.
Nattee Niparnan. Recall  Complexity Analysis  Comparison of Two Algos  Big O  Simplification  From source code  Recursive.
Divide-and-Conquer1 7 2  9 4   2  2 79  4   72  29  94  4.
Divide-and-Conquer1 7 2  9 4   2  2 79  4   72  29  94  4.
Data Structures, Spring 2004 © L. Joskowicz 1 Data Structures – LECTURE 3 Recurrence equations Formulating recurrence equations Solving recurrence equations.
Data Structures, Spring 2006 © L. Joskowicz 1 Data Structures – LECTURE 3 Recurrence equations Formulating recurrence equations Solving recurrence equations.
Updates HW#1 has been delayed until next MONDAY. There were two errors in the assignment Merge sort runs in Θ(n log n). Insertion sort runs in Θ(n2).
Recurrences The expression: is a recurrence. –Recurrence: an equation that describes a function in terms of its value on smaller functions Analysis of.
Unit 1. Sorting and Divide and Conquer. Lecture 1 Introduction to Algorithm and Sorting.
Recurrences The expression: is a recurrence. –Recurrence: an equation that describes a function in terms of its value on smaller functions BIL741: Advanced.
Analysis of Recursive Algorithms October 29, 2014
October 1, Algorithms and Data Structures Lecture III Simonas Šaltenis Nykredit Center for Database Research Aalborg University
Divide-and-Conquer 7 2  9 4   2   4   7
Analyzing Recursive Algorithms A recursive algorithm can often be described by a recurrence equation that describes the overall runtime on a problem of.
Divide-and-Conquer1 7 2  9 4   2  2 79  4   72  29  94  4.
Project 2 due … Project 2 due … Project 2 Project 2.
Divide and Conquer Andreas Klappenecker [based on slides by Prof. Welch]
10/25/20151 CS 3343: Analysis of Algorithms Lecture 6&7: Master theorem and substitution method.
CompSci 102 Discrete Math for Computer Science April 17, 2012 Prof. Rodger.
Introduction to Algorithms Chapter 4: Recurrences.
Recurrences David Kauchak cs161 Summer Administrative Algorithms graded on efficiency! Be specific about the run times (e.g. log bases) Reminder:
Divide and Conquer Andreas Klappenecker [based on slides by Prof. Welch]
1 Algorithms CSCI 235, Fall 2015 Lecture 6 Recurrences.
Foundations II: Data Structures and Algorithms
Design & Analysis of Algorithms COMP 482 / ELEC 420 John Greiner
Lecture 5 Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master.
Divide and Conquer. Recall Divide the problem into a number of sub-problems that are smaller instances of the same problem. Conquer the sub-problems by.
1Computer Sciences Department. Objectives Recurrences.  Substitution Method,  Recursion-tree method,  Master method.
Recurrences (in color) It continues…. Recurrences When an algorithm calls itself recursively, its running time is described by a recurrence. When an algorithm.
Divide and Conquer Faculty Name: Ruhi Fatima Topics Covered Divide and Conquer Matrix multiplication Recurrence.
Advanced Algorithms Analysis and Design By Dr. Nazir Ahmad Zafar Dr Nazir A. Zafar Advanced Algorithms Analysis and Design.
Recurrences It continues… Jeff Chastine. Recurrences When an algorithm calls itself recursively, its running time is described by a recurrence. A recurrence.
Equal costs at all levels
Introduction to Algorithms: Divide-n-Conquer Algorithms
Recursion Ali.
Divide-and-Conquer 6/30/2018 9:16 AM
Unit 1. Sorting and Divide and Conquer
CS 3343: Analysis of Algorithms
Lecture 4 Divide-and-Conquer
Chapter 4 Divide-and-Conquer
Divide and Conquer.
CS 3343: Analysis of Algorithms
CSCE 411 Design and Analysis of Algorithms
CS 3343: Analysis of Algorithms
Algorithms and Data Structures Lecture III
Divide-and-Conquer 7 2  9 4   2   4   7
Topic: Divide and Conquer
Richard Anderson Lecture 11 Recurrences
CS 3343: Analysis of Algorithms
CSE 2010: Algorithms and Data Structures
Divide and Conquer (Merge Sort)
Divide-and-Conquer 7 2  9 4   2   4   7
At the end of this session, learner will be able to:
Introduction To Algorithms
Algorithms Recurrences.
Algorithms CSCI 235, Spring 2019 Lecture 6 Recurrences
Divide-and-Conquer 7 2  9 4   2   4   7
Richard Anderson Lecture 12, Winter 2019 Recurrences
Quicksort Quick sort Correctness of partition - loop invariant
Divide and Conquer Merge sort and quick sort Binary search
Richard Anderson Lecture 12 Recurrences
Algorithms and Data Structures Lecture III
Divide-and-Conquer 7 2  9 4   2   4   7
Presentation transcript:

Trevor Brown trevor.brown@uwaterloo.ca CS 341: Algorithms Trevor Brown trevor.brown@uwaterloo.ca

This time Divide and conquer paradigm (intro to…) Merge sort Recurrence relations Tree recursion method: merge sort as an example Guess and check method [time permitting] Master theorem

Divide and conquer A useful design strategy / paradigm

Divide-and-Conquer Design Strategy divide: Given a problem instance I, construct one or more smaller problem instances I1, …, Ia These are called subproblems Usually, want subproblems to be small compared to the size of I (e.g., half the size) conquer: For 1 ≤ j ≤ a, solve instance Ij recursively, obtaining solutions S1, …, Sa combine: Given solutions S1, …, Sa, use an appropriate combining function to find the solution S to the problem instance I i.e., S = Combine(S1, …, Sa).

Example: Design of Mergesort

divide 105 7 13 8 14 1 19 11 4 10 98 16 31 5 21 12 105 7 13 8 14 1 19 11 4 10 98 16 31 5 21 12 105 7 13 8 14 1 19 11 4 10 98 16 31 5 21 12 105 7 13 8 14 1 19 11 4 10 98 16 31 5 21 12 105 7 13 8 14 1 19 11 4 10 98 16 31 5 21 12

Merge: Conquer and combine 1 4 5 7 8 10 11 12 13 14 19 21 31 96 98 105 1 7 8 11 13 14 19 105 4 5 10 12 21 31 96 98 7 8 13 105 1 11 14 19 4 10 96 98 5 12 21 31 7 105 8 13 1 14 11 19 4 10 16 98 5 31 12 21 105 7 13 8 14 1 19 11 4 10 98 16 31 5 21 12

Merge simulation 1 L R 5 31 12 21 O 5 12 5 12 21 31 5 5 12 21

Another example merge step 1 4 5 7 8 10 11 12 13 14 19 21 31 96 98 105 1 7 8 11 13 14 19 105 4 5 10 12 21 31 96 98 7 8 13 105 1 11 14 19 4 10 96 98 5 12 21 31 7 105 8 13 1 14 11 19 4 10 16 98 5 31 12 21 105 7 13 8 14 1 19 11 4 10 98 16 31 5 21 12

Merge simulation 2 R L 4 10 96 98 5 12 21 31 O 4 5 10 12 21 31 96 98 4 5 10 12 21 31 96 4 5 10 4 5 4 5 10 12 4 5 10 12 21 4 5 10 12 21 31 4

Pseudocode for mergesort

Pseudocode for merge While both arrays still contain elements to be merged Handle case where the right array empties first (and left has some elements) Handle case where the left array empties first

Analysis of mergesort MergeSort(A, n) takes O(n) time, plus the time for its two recursive calls! O(1) O(1) O(1) O(n) O(n) ??? ??? O(n)

T(n) is a function of T(…). T is a recurrence relation More formally T(n) is a function of T(…). T is a recurrence relation How can we compute T(n)?

Analysis of Mergesort msort(n) cn + = cn + We use the recursion tree method This is informal! We ignore floor/ceil by assuming n=2k, and skip lots of math msort(n/2) 2(cn/2) + = cn + msort(n/2) 4(cn/4) + = cn + msort(n/4) msort(n/4) msort(n/4) msort(n/4) …. …. …. …. …. …. msort(1) msort(1) msort(1) msort(1) n(c) = cn Total = cn*#levels Total = cn log2(n) MergeSort takes O(n log n) time on an input of size n.

Recurrence relations Analysis tool for recursive algorithms

Recurrence Relations

Guess-and-check Method Suppose we have the following recurrence 𝑇 0 =4 ; 𝑇 𝑛 =𝑇 𝑛−1 +6𝑛−5 Guess the form of the solution any way you like My approach Recursively substitute the formula into itself Try to identify patterns to guess the final closed form Check that the guess was correct

Worked example Recurrence: 𝑇 0 =4 ; 𝑇 𝑛 =𝑇 𝑛−1 +6𝑛−5 𝑇 𝑛 = 𝑇 𝑛−2 +6 𝑛−1 −5 +6𝑛−5 (substitute) =𝑇 𝑛−2 +6𝑛−6−5+6𝑛−5 =𝑇 𝑛−2 +2 6𝑛−5 −6 = 𝑇 𝑛−3 +6 𝑛−2 −5 +2 6𝑛−5 −6 (substitute) =𝑇 𝑛−3 +6𝑛−2 6 −5+2 6𝑛−5 −6 =𝑇 𝑛−3 +3 6𝑛−5 −6(1+2) … identify patterns and guess what happens in the limit =𝑇 𝑛−𝑛 +𝑛 6𝑛−5 −6 1+2+3+…+ 𝑛−1 =𝑔𝑢𝑒𝑠𝑠(𝑛) Compare: new terms? +(6n-5) -6 Compare: new terms? +(6n-5) -2(6)

Recurrence: 𝑇 0 =4 ; 𝑇 𝑛 =𝑇 𝑛−1 +6𝑛−5 g𝑢𝑒𝑠𝑠 𝑛 =𝑇 𝑛−𝑛 +𝑛 6𝑛−5 −6 1+2+3+…+ 𝑛−1 =𝑇 0 +6 𝑛 2 −5𝑛−6 𝑖=1 𝑛−1 𝑖 (simplify) =4+6 𝑛 2 −5𝑛−6𝑛(𝑛−1)/2 =4+6 𝑛 2 −5𝑛−3 𝑛 2 +3𝑛 =3 𝑛 2 −2𝑛+4 Are we done? No! This is a guess. Must prove it is correct using induction

Proof Recurrence: 𝑇 0 =4 ; 𝑇 𝑛 =𝑇 𝑛−1 +6𝑛−5 Our guess: 𝑔𝑢𝑒𝑠𝑠 𝑛 =3 𝑛 2 −2𝑛+4 Want to prove: 𝑔𝑢𝑒𝑠𝑠 𝑛 =𝑇 𝑛 for all 𝑛 Base case: guess 0 =3 0 2 −2 0 +4=𝑇 0 Inductive case: assume 𝑔𝑢𝑒𝑠𝑠 n =𝑇 𝑛 , and show 𝑔𝑢𝑒𝑠𝑠 n+1 =𝑇 𝑛+1 . 𝑇 𝑛+1 =𝑇 𝑛 +6(𝑛+1)−5 (by definition) =𝑔𝑢𝑒𝑠𝑠 𝑛 +6 𝑛+1 −5 (by inductive hypothesis) = 3 𝑛 2 −2𝑛+4 +6 𝑛+1 −5=3 𝑛 2 +4𝑛+5 𝑔𝑢𝑒𝑠𝑠 𝑛+1 =3 𝑛+1 2 −2 𝑛+1 +4 (by definition) =3 𝑛 2 +2𝑛+1 −2𝑛−2+4=3 𝑛 2 +4𝑛+5=𝑇 𝑛+1 . QED.

Another approach Instead of substituting and carefully identifying the pattern, and computing exact constants in our quadratic function, you might simply guess that the result is a quadratic function 𝒂 𝒏 𝟐 +𝒃𝒏+𝒄 for some unknown constants a, b, c And then carry the unknown constants into the proof! In the inductive step, you can determine what the constants must be, for the proof to work out

Recursion tree method formalized Sample recurrence for two recursive calls on problem size n/2 Step 1 Step 2 Step 3 Step 4

Master theorem for recurrences Provides a formula for solving many recurrence relations We start with a simplified version

Proof of simplified master theorem Must sum the values over all levels!

Summing over all levels

The three cases for 𝑟

Math Details for each case Not covering in detail! Included just in case you are curious, and for your notes.

Worked Examples Recall: simplified master theorem b=2; y=1; x=1 Recall: simplified master theorem Θ 𝑛 𝑥 log 𝑛 =Θ 𝑛 log 𝑛 a=3; b=2; y=1; x=log2 3 Θ 𝑛 𝑥 =Θ 𝑛 log 2 3 a=4; b=2; y=1; x=log2 4 Θ 𝑛 𝑥 =Θ 𝑛 2 Questions: a=? b=? y=? x=? which Θ function? a=2; b=2; y=3/2; x=1 Θ 𝑛 𝑦 =Θ 𝑛 3/2

General master theorem Example recurrence: Arbitrary function of 𝒏 (not just 𝑐 𝑛 𝑦 ) Must reason about relationship between 𝑓(𝑛) and 𝑛 𝑥

Revisiting the recursion tree method Some recurrences with complex f(n) functions (such as f(n) = log n) can still be solved “by hand” Example: Let n= 2 j ; 𝑇 1 =1 ; 𝑇 𝑛 =2𝑇 𝑛 2 +𝑛 𝑙𝑜𝑔 𝑛 Must sum the values over all levels!

Revisiting the recursion tree method Recall: n= 2 j ; 𝑇 1 =1 ; 𝑇 𝑛 =2𝑇 𝑛 2 +𝑛 𝑙𝑜𝑔 𝑛

Next time More divide and conquer problems Non-dominated points Multi-precision multiplication (very likely) Selection (somewhat likely) Closest pair (unlikely)