Foundations II: Data Structures and Algorithms Topic 2: -- Analyzing algorithms -- Solving recurrences CSE 2331 / 5331

What if i (or j) starts from 1000 ? Example: for Loop (1) What if i (or j) starts from 1000 ? What if i starts from n/2 ? What if j ends with i2 ? What if j ends with 𝑖 ? Textbook A.1 for summation formulas CSE 2331 / 5331

What if we change the first line to n > 100000 ? Example: for Loop (2) What if we change the first line to n > 100000 ? CSE 2331 / 5331

Evaluating Summations Evaluate 𝑖=𝑚 𝑛 𝑓 𝑖 No fixed technique Be familiar with some basics Textbook Appendix A.1and A.2 CSE 2331 / 5331

Two Useful Techniques Splitting summations Using integral: E.g, 𝑖=1 𝑛 𝑖 , 𝑖=2 𝑛 lg 𝑖 Using integral: 𝑖=1 𝑛 𝑓 𝑖 ≈ 1 𝑛 𝑓 𝑥 𝑑𝑥 If f is monotonically increasing, 𝑚−1 𝑛 𝑓 𝑥 𝑑𝑥 ≤ 𝑖=𝑚 𝑛 𝑓 𝑖 ≤ 𝑚 𝑛+1 𝑓 𝑥 𝑑𝑥 E.g, 𝑖=1 𝑛 𝑖 CSE 2331 / 5331

Example: while Loop What if we change the 𝑖←𝑖+3 to 𝑖←𝑖+100 ? CSE 2331 / 5331

Changes to 𝑖←𝑖∗𝑐 since Θ log 𝑐 𝑛 =Θ( lg 𝑛) for any constant c. Analysis of while Loop It does not matter if Changes to 𝑖←𝑖∗𝑐 since Θ log 𝑐 𝑛 =Θ( lg 𝑛) for any constant c. Stops when 3 𝑘 ≈𝑛 , that is, after 𝑘=Θ log 3 𝑛 =Θ( lg 𝑛) iterations. CSE 2331 / 5331

What if we change the 𝑗≤𝑛 to 𝑗≤𝑖 ? More Examples (0) What if we change the 𝑗≤𝑛 to 𝑗≤𝑖 ? CSE 2331 / 5331

More Examples (1) CSE 2331 / 5331

What if we change the 𝑖←𝑖+1 to 𝑖←𝑖∗2 ? More Examples (2) What if we change the 𝑖←𝑖+1 to 𝑖←𝑖∗2 ? CSE 2331 / 5331

More Examples (3) CSE 2331 / 5331

Recursive Algorithms CSE 2331 / 5331

Sorting Revisited (1) CSE 2331 / 5331

T(n): worst case time complexity for any input of size n Recurrence: T(n) = T(n-1) + cn Solving recurrence: = T(n-2) + c(n-1) + cn = T(n-k) + c(n-k+1) + .. + cn = T(1) + c[2 + … + n-1 + n]= Θ (n2) Expansion into a series. Usual assumption: T(n) is constant for small n, say n = 1, 2, 3 CSE 2331 / 5331

More examples (1) CSE 2331 / 5331

More Examples (2) CSE 2331 / 5331

Sorting Revisited (2) Can we do better than Θ( 𝑛 2 ) ? Yes. Many ways. We will talk about Merge-sort CSE 2331 / 5331

Merge Sort Divide ? ? Conquer CSE 2331 / 5331

Conquer Step Merge(B, C) Input: Given two sorted arrays B and C Output: Merge into a single sorted array 2 3 4 5 7 2 3 4 5 7 8 10 16 16 10 8 3 7 5 4 2 2 3 4 5 2 3 4 2 2 3 If size of B and C are s and t: Running time: O ( s + t ) CSE 2331 / 5331

Pseudocode MergeSort ( A, r, s ) if ( r ≥ s) return; m = (r+s) / 2; A1 = MergeSort ( A, r, m ); A2 = MergeSort ( A, m+1, s ); Merge (A1, A2); Recurrence relation for MergeSort(A, 1, n) T(n) = 2T(n/2) + cn CSE 2331 / 5331

Solve Recurrence Expansion into a series Recursion tree Substitution method CSE 2331 / 5331

Expansion T(n) = 2 T(n/2) + n = 4 (2 T(n/8) + n/8)) + n + n ……. = 2 (2 T (n/4) + n/2) + n = 4 T(n/4) + n + n = 4 (2 T(n/8) + n/8)) + n + n ……. = 2k T(n/2k) + n + … + n Let 𝑛 2 𝑘 =1⇒𝑘= log 2 𝑛 . Then: 𝑇 𝑛 =Θ(𝑛 lg 𝑛) CSE 2331 / 5331

Recursion-tree Method Solve T(n) = 2 T(n/2) + n n T(n/2) T(n) = n n/2 n/4  (1) n n/2 T(n/4) n T(n/2) n n/2+n/2 4  (n/4) n   (1) T(n) T(n) ≤ n  height(tree) = n lg n CSE 2331/5331

Substitution Method Guess a solution Verify its correctness using induction CSE 2331 / 5331

Substitution Method Guess T(n) = O ( n lg n ) i.e, T(n) ≤ c n lg n Induction: Assume true for T(n/2) T(n) = 2 T(n/2) + n ≤ 2 c n/2 lg (n/2) + n = c n lg n - (c-1) n ≤ c n lg n if c > 1 Base case: T(1) …. no But true for T(2) for sufficiently large c Verification Solve c CSE 2331/5331

Remarks Lower bound similar => T(n) = (n lg n) One can use recursion tree / expansion to get an intuition, and then use substitution method to prove it. CSE 780 Algorithms

Another MergeSort MergeSort ( A, r, s ) if ( r ≥ s) return; m1 = r+(s-r) / 3; m2 = r+2(s-r) / 3; A1 = MergeSort ( A, r, m1 ); A2 = MergeSort ( A, m1 +1, m2 ); A3 = MergeSort ( A, m2 +1, s ); Merge (A1, A2, A3); What if we change 3 to k ? Recurrence relation for MergeSort(A,1,n) T(n) = 3T(n/3) + cn CSE 2331 / 5331

Another MergeSort MergeSort ( A, r, s ) if ( r ≥ s) return; m = r+(s-r) / 3; A1 = MergeSort ( A, r, m ); A2 = MergeSort ( A, m+1, s ); Merge (A1, A2); Recurrence relation for MergeSort(A,1,n) T(n) = T(n/3) + T(2n/3) + cn CSE 2331 / 5331

Recursion Tree log 3 𝑛 full levels and log 3 2 𝑛 total levels Total cost of each level is at most cn Upper bound 𝑂 𝑛 log 𝑛 Lower bound Ω(𝑛 log 𝑛) 𝑇 𝑛 =Θ(𝑛 log 𝑛) CSE 2331 / 5331

More Examples of Recurrences (1) 𝑇 𝑛 =𝑇 𝑛−1 +𝑐 𝑇 𝑛 =Θ 𝑛 𝑇 𝑛 =𝑇 𝑛−1 +𝑐𝑛 𝑇 𝑛 =Θ 𝑛 2 𝑇 𝑛 =𝑇 𝑛−1 +𝑐 𝑛 2 𝑇 𝑛 =Θ 𝑛 3 T(n) = T(n-10) + c ? T(n) = T(n-10) + n2 + 3n ? CSE 2331 / 5331

More Examples (2) 𝑇 𝑛 =𝑇 𝑛 2 +𝑐 𝑇 𝑛 =𝑇 𝑛 3 +𝑐𝑛 𝑇 𝑛 =2𝑇 𝑛 2 +𝑐 𝑛 2 𝑇 𝑛 =𝑇 𝑛 2 +𝑐 𝑇 𝑛 =Θ lg 𝑛 𝑇 𝑛 =𝑇 𝑛 3 +𝑐𝑛 𝑇 𝑛 =Θ 𝑛 𝑇 𝑛 =2𝑇 𝑛 2 +𝑐 𝑛 2 𝑇 𝑛 =Θ 𝑛 2 𝑇 𝑛 =4𝑇 𝑛 2 +𝑐𝑛 𝑇 𝑛 =4𝑇 𝑛 2 +𝑐 𝑛 2 𝑇 𝑛 =Θ n 2 lg 𝑛 𝑇(𝑛) = 𝑇( 𝑛 4 ) + 𝑇( 𝑛 2 ) + 𝑐𝑛 ? CSE 2331 / 5331

More Examples (3) 𝑇 𝑛 =2𝑇 𝑛−1 +𝑐 𝑇 𝑛 =Θ( 2 𝑛 ) Sometimes, it is hard to get tight bound 𝑇 𝑛 =𝑇 𝑛−1 +𝑇 𝑛−2 +𝑐 Easy to prove that 𝑇 𝑛 =𝑂 2 𝑛 and 𝑇 𝑛 =Ω ( 2 𝑛 ) CSE 2331 / 5331

Summary No good formula for solving recurrences Practice! Expansion and recursion tree are two most intuitive These two are quite similar Substitution method more rigorous and powerful But require good intuition and use of induction Other methods: Master Theorem (not covered) CSE 2331 / 5331