1. Foundations II: Data Structures and Algorithms
Topic 2:
-- Analyzing algorithms
-- Solving recurrences
CSE 2331 / 5331

2. What if i (or j) starts from 1000 ?
Example: for Loop (1)
What if i (or j) starts from ?
What if i starts from n/2 ?
What if j ends with i2 ?
What if j ends with 𝑖 ?
Textbook A.1 for summation formulas
CSE 2331 / 5331

3. What if we change the first line to n > 100000 ?
Example: for Loop (2)
What if we change the first line to n > ?
CSE 2331 / 5331

4. Evaluating Summations
Evaluate 𝑖=𝑚 𝑛 𝑓 𝑖
No fixed technique
Be familiar with some basics
Textbook Appendix A.1and A.2
CSE 2331 / 5331

5. Two Useful Techniques Splitting summations Using integral:
E.g, 𝑖=1 𝑛 𝑖 , 𝑖=2 𝑛 lg 𝑖
Using integral:
𝑖=1 𝑛 𝑓 𝑖 ≈ 1 𝑛 𝑓 𝑥 𝑑𝑥
If f is monotonically increasing,
𝑚−1 𝑛 𝑓 𝑥 𝑑𝑥 ≤ 𝑖=𝑚 𝑛 𝑓 𝑖 ≤ 𝑚 𝑛+1 𝑓 𝑥 𝑑𝑥
E.g, 𝑖=1 𝑛 𝑖
CSE 2331 / 5331

6. Example: while Loop What if we change the 𝑖←𝑖+3 to 𝑖←𝑖+100 ?
CSE 2331 / 5331

7. Changes to 𝑖←𝑖∗𝑐 since Θ log 𝑐 𝑛 =Θ( lg 𝑛) for any constant c.
Analysis of while Loop
It does not matter if
Changes to 𝑖←𝑖∗𝑐 since Θ log 𝑐 𝑛 =Θ( lg 𝑛) for any constant c.
Stops when 3 𝑘 ≈𝑛 , that is, after 𝑘=Θ log 3 𝑛 =Θ( lg 𝑛) iterations.
CSE 2331 / 5331

8. What if we change the 𝑗≤𝑛 to 𝑗≤𝑖 ?
More Examples (0)
What if we change the 𝑗≤𝑛 to 𝑗≤𝑖 ?
CSE 2331 / 5331

9. More Examples (1)
CSE 2331 / 5331

10. What if we change the 𝑖←𝑖+1 to 𝑖←𝑖∗2 ?
More Examples (2)
What if we change the 𝑖←𝑖+1 to 𝑖←𝑖∗2 ?
CSE 2331 / 5331

11. More Examples (3)
CSE 2331 / 5331

12. Recursive Algorithms
CSE 2331 / 5331

13. Sorting Revisited (1)
CSE 2331 / 5331

14. T(n): worst case time complexity for any input of size n Recurrence:
T(n) = T(n-1) + cn
Solving recurrence:
= T(n-2) + c(n-1) + cn
= T(n-k) + c(n-k+1) cn
= T(1) + c[2 + … + n-1 + n]= Θ (n2)
Expansion into a series.
Usual assumption:
T(n) is constant for small n, say n = 1, 2, 3
CSE 2331 / 5331

15. More examples (1)
CSE 2331 / 5331

16. More Examples (2)
CSE 2331 / 5331

17. Sorting Revisited (2) Can we do better than Θ( 𝑛 2 ) ? Yes. Many ways.
We will talk about Merge-sort
CSE 2331 / 5331

18. Merge Sort
Divide ? ?
Conquer
CSE / 5331

19. Conquer Step Merge(B, C)
Input: Given two sorted arrays B and C
Output: Merge into a single sorted array 16 10 8 3 7 5 4 2 2
2 3
If size of B and C are s and t:
Running time:
O ( s + t )
CSE 2331 / 5331

20. Pseudocode MergeSort ( A, r, s )
if ( r ≥ s) return;
m = (r+s) / 2;
A1 = MergeSort ( A, r, m );
A2 = MergeSort ( A, m+1, s );
Merge (A1, A2);
Recurrence relation for MergeSort(A, 1, n)
T(n) = 2T(n/2) + cn
CSE 2331 / 5331

21. Solve Recurrence Expansion into a series Recursion tree
Substitution method
CSE 2331 / 5331

22. Expansion T(n) = 2 T(n/2) + n = 4 (2 T(n/8) + n/8)) + n + n …….
= 2 (2 T (n/4) + n/2) + n = 4 T(n/4) + n + n
= 4 (2 T(n/8) + n/8)) + n + n
…….
= 2k T(n/2k) + n + … + n
Let 𝑛 2 𝑘 =1⇒𝑘= log 2 𝑛 . Then:
𝑇 𝑛 =Θ(𝑛 lg 𝑛)
CSE 2331 / 5331

23. Recursion-tree Method
Solve T(n) = 2 T(n/2) + n n
T(n/2)
T(n) = n
n/2
n/4
 (1) n
n/2
T(n/4) n
T(n/2) n
n/2+n/2
4  (n/4)
n   (1)
T(n)
T(n) ≤ n  height(tree) = n lg n
CSE 2331/5331

24. Substitution Method Guess a solution
Verify its correctness using induction
CSE 2331 / 5331

25. Substitution Method Guess T(n) = O ( n lg n ) i.e, T(n) ≤ c n lg n
Induction:
Assume true for T(n/2)
T(n) = 2 T(n/2) + n
≤ 2 c n/2 lg (n/2) + n
= c n lg n - (c-1) n
≤ c n lg n if c > 1
Base case:
T(1) …. no
But true for T(2) for sufficiently large c
Verification
Solve c
CSE 2331/5331

26. Remarks Lower bound similar => T(n) = (n lg n)
One can use recursion tree / expansion to get an intuition, and then use substitution method to prove it.
CSE 780 Algorithms

27. Another MergeSort MergeSort ( A, r, s ) if ( r ≥ s) return;
m1 = r+(s-r) / 3;
m2 = r+2(s-r) / 3;
A1 = MergeSort ( A, r, m1 );
A2 = MergeSort ( A, m1 +1, m2 );
A3 = MergeSort ( A, m2 +1, s );
Merge (A1, A2, A3);
What if we change
3 to k ?
Recurrence relation for MergeSort(A,1,n)
T(n) = 3T(n/3) + cn
CSE 2331 / 5331

28. Another MergeSort MergeSort ( A, r, s )
if ( r ≥ s) return;
m = r+(s-r) / 3;
A1 = MergeSort ( A, r, m );
A2 = MergeSort ( A, m+1, s );
Merge (A1, A2);
Recurrence relation for MergeSort(A,1,n)
T(n) = T(n/3) + T(2n/3) + cn
CSE 2331 / 5331

29. Recursion Tree log 3 𝑛 full levels and log 3 2 𝑛 total levels
Total cost of each level is at most cn
Upper bound 𝑂 𝑛 log 𝑛
Lower bound Ω(𝑛 log 𝑛)
𝑇 𝑛 =Θ(𝑛 log 𝑛)
CSE 2331 / 5331

30. More Examples of Recurrences (1)
𝑇 𝑛 =𝑇 𝑛−1 +𝑐
𝑇 𝑛 =Θ 𝑛
𝑇 𝑛 =𝑇 𝑛−1 +𝑐𝑛
𝑇 𝑛 =Θ 𝑛 2
𝑇 𝑛 =𝑇 𝑛−1 +𝑐 𝑛 2
𝑇 𝑛 =Θ 𝑛 3
T(n) = T(n-10) + c ?
T(n) = T(n-10) + n2 + 3n ?
CSE 2331 / 5331

31. More Examples (2) 𝑇 𝑛 =𝑇 𝑛 2 +𝑐 𝑇 𝑛 =𝑇 𝑛 3 +𝑐𝑛 𝑇 𝑛 =2𝑇 𝑛 2 +𝑐 𝑛 2
𝑇 𝑛 =𝑇 𝑛 2 +𝑐
𝑇 𝑛 =Θ lg 𝑛
𝑇 𝑛 =𝑇 𝑛 3 +𝑐𝑛
𝑇 𝑛 =Θ 𝑛
𝑇 𝑛 =2𝑇 𝑛 2 +𝑐 𝑛 2
𝑇 𝑛 =Θ 𝑛 2
𝑇 𝑛 =4𝑇 𝑛 2 +𝑐𝑛
𝑇 𝑛 =4𝑇 𝑛 2 +𝑐 𝑛 2
𝑇 𝑛 =Θ n 2 lg 𝑛
𝑇(𝑛) = 𝑇( 𝑛 4 ) + 𝑇( 𝑛 2 ) + 𝑐𝑛 ?
CSE 2331 / 5331

32. More Examples (3) 𝑇 𝑛 =2𝑇 𝑛−1 +𝑐
𝑇 𝑛 =Θ( 2 𝑛 )
Sometimes, it is hard to get tight bound
𝑇 𝑛 =𝑇 𝑛−1 +𝑇 𝑛−2 +𝑐
Easy to prove that 𝑇 𝑛 =𝑂 2 𝑛 and 𝑇 𝑛 =Ω ( 2 𝑛 )
CSE 2331 / 5331

33. Summary No good formula for solving recurrences
Practice!
Expansion and recursion tree are two most intuitive
These two are quite similar
Substitution method more rigorous and powerful
But require good intuition and use of induction
Other methods:
Master Theorem (not covered)
CSE 2331 / 5331