1. Lecture 4

2. kf(n) is O(f(n)) for any positive constant k n r is O(n p ) if r  p since lim n  n r /n p = 0, if r < p = 1 if r = p f(n) is O(g(n)), g(n) is O(h(n)), Is f(n) O(h(n)) ? kf(n)  kf(n), for all n, k > 0 f(n)  cg(n), for some c > 0, n  m g(n)  dh(n), for some d > 0, n  p f(n)  (cd)h(n), for some cd > 0, n  max(p,m) n r is O(exp(n)) for any r > 0 since lim n  n r /exp(n) = 0,

3. f(n) + g(n) is O(h(n)) if f(n), g(n) are O(h(n)) Is kn O(n 2 ) ? log n is O (n r ) if r  0, since lim n  log(n)/ n r = 0, kn is O(n) n is O(n 2 ) f(n)  ch(n), for some c > 0, n  m g(n)  dh(n), for some d > 0, n  p f(n) + g(n)  ch(n) + dh(n), n  max(m,p)  (c+d)h(n), c + d > 0, n  max(m,p)

4. T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n) is O(f(n)g(n)) T 1 (n)  cf(n), for some c > 0, n  m T 2 (n)  dg(n), for some d > 0, n  p T 1 (n) T 2 (n)  (cd)f(n)g(n), for some cd > 0, n  max(p,m) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) + T 2 (n) is O(max(f(n),g(n))) Let h(n) = max(f(n),g(n)), T 1 (n) is O(f(n)), f(n) is O(h(n)), so T 1 (n) is O(h(n)), T 2 (n) is O(g(n)), g(n) is O(h(n) ), so T 2 (n) is O(h(n)), Thus T 1 (n) + T 2 (n) is O(h(n))

5. Algorithm Complexity Analysis diff = sum = 0; For (k=0: k < N; k++) sum  sum + 1; diff  diff - 1; For (k=0: k < 3N; k++) sum  sum - 1;

6. First line takes 2 basic steps Every iteration of first loop takes 2 basic steps. First loop runs N times Every iteration of second loop takes 1 basic step Second loop runs for 3N times Overall, 2 + 2N + 3N steps This is O(N)

7. Rules Complexity of a loop: O(Number of iterations in a loop * maximum complexity of each iteration) Nested Loops: Analyze the innermost loop first, complexity of next outer loop = number of iterations in this loop * complexity of inner loop, etc….. sum = 0; For (i=0; i < N; i++) For (j=0; j < N; j++) sum  sum + 1;

8. If (Condition) S1 Else S2 Maximum of the two If (yes) print(1,2,….1000N) Else print(1,2,….N 2 ) Inner loop: O(N) Outer loop: N iterations Overall: O(N 2 )

9. Maximum Subsequence Problem There is an array of N elements Need to find i, j such that the sum of all elements between the ith and jth position is maximum for all such sums Maxsum = 0; For (i=0; i < N; i++) For (j=i; j < N; j++) { Thissum = sum of all elements between ith and jth positions; Maxsum = max(Thissum, Maxsum);}

10. Analysis Inner loop:  j=i N-1 (j-i + 1) = (N – i + 1)(N-i)/2 Outer Loop:  i=0 N-1 (N – i + 1)(N-i)/2 = (N 3 + 3N 2 + 2N)/6 Overall: O(N 3 )

11. Maxsum = 0; For (i=0; i < N; i++) For (Thissum=0;j=i; j < N; j++) { Thissum = Thissum + A[j]; Maxsum = max(Thissum, Maxsum);} Complexity?  i=0 N-1 (N-i) = N 2 – N(N+1)/2 = (N 2 – N)/2 O(N 2 )

12. Divide and Conquer Break a big problem into two small sub-problems Solve each of them efficiently. Combine the two solutions

13. Maximum subsequence sum by divide and conquer Divide the array into two parts: left part, right part Max. subsequence lies completely in left, or completely in right or spans the middle. If it spans the middle, then it includes the max subsequence in the left ending at the last element and the max subsequence in the right starting from the center

14. 4 –3 5 –2 -1 2 6 -2 Max subsequence sum for first half = 6 second half = 8 Max subsequence sum for first half ending at the last element is 4 Max subsequence sum for sum second half starting at the first element is 7 Max subsequence sum spanning the middle is ? Max subsequence spans the middle

15. Maxsubsum(A[], left, right) { if left = right, maxsum = max(A[left], 0); Center =  (left + right)/2  maxleftsum = Maxsubsum(A[],left, center); maxrightsum = Maxsubsum(A[],center+1,right); maxleftbordersum = 0; leftbordersum = 0; for (i=center; i>=left; i--) leftbordersum+=A[i]; Maxleftbordersum=max(maxleftbordersum, leftbordersum);

16. Find maxrightbordersum….. return(max(maxleftsum, maxrightsum, maxrightbordersum + maxleftbordersum);

17. Complexity Analysis T(1)=1 T(n) = 2T(n/2) + cn = 2.cn/2 + 4T(n/4) + cn = 4T(n/4) + 2cn = 8T(n/8) + 3cn =………….. = 2 i T(n/2 i ) + icn =………………… (reach a point when n = 2 i i=log n = n.T(1) + cnlog n

18. n + cnlogn = O(nlogn)

19. Linear Complexity Algorithm Maxsum = 0; Thissum = 0; For (j=0; j<N; j++) { Thissum = Thissum + A[j]; If (Thissum  0), Thissum = 0; If (Maxsum  Thissum), Maxsum = Thissum; } O(N) complexity

20. Master Theorem T(1)=p T(n) = aT(n/b) + cn k Case (1): a  b k then T(n) is O(n log b a ) Case(2): a = b k then T(n) is O(n k logn) Case(3): a < b k then T(n) is O(n k ) Cormen, Leiserson, Rivest

21. Binary Search You have a sorted list of numbers You need to search the list for the number If the number exists find its position. If the number does not exist you need to detect that

22. Search(num, A[],left, right) { if (left = right) { if (A[left ]=num) return(left) and exit; else conclude NOT PRESENT and exit; } center =  (left + right)/2  ; If (A[center]  num) Search(num,A[],center + 1,right); If (A[center]>num) Search(num,A[],left,center ); If (A[center]=num) return(center) and exit; }

23. Complexity Analysis T(n) = T(n/2) + c O(log n) complexity

24. Reading Assignment Whole of chapter 2