CS420 lecture two Fundamentals wim bohm, cs, CSU

Asymptotics Asymptotics show the relative growth of functions by comparing them to other functions. There are different notations: f(x) ~ g(x): f(x) = o(g(x)): where f and g are positive functions

Big O, big omega, big theta f(x) = O(g(x)) iff there are positive integers c and n0 such that f(x) < c.g(x) for all x > n0 f(x) = Ω(g(x)) iff f(x) > c.g(x) for all x > n0 f(x) = Θ(g(x)) iff f(x) = O(g(x)) and f(x) = Ω(g(x))

Big O etc. Big O used in upper bounds, ie the worst or average case complexity of an algorithm. Big theta is a tight bound, eg stronger than upper bound. Big omega used in lower bounds, ie the complexity of a problem. (So we were sloppy in the last lecture.)

Closed / open problems A closed problem has a lower bound Ω(f(x)) and an algorithm with upper bound O(f(x)) eg searching, sorting what about matrix multiply? An open problem has lower bound < upper bound

lower bounds An easy lower bound on a problem is the size of the output it needs to produce, or the number of inputs it has to access Generate all permutations of size n, lower bound? Towers of Hanoi, lower bound? Sum n input integers, lower bound? but... sum integers 1 to n?

growth rates f(n) = O(1) constant Scalar operations (+,-,*,/) when input size not measured in #bits Straight line code of simple assignments (x= simple expresssion) and conditionals with simple sub-expressions Function calls, discuss

f(n) = log(n) definition: bx = a  x = logba, eg 23=8, log28=3 log(x*y) = log x + log y because bx . by = bx+y log(x/y) = log x – log y log xa= a log x log x is a 1to1 monotonically growing function log x = log y  x=y

more log stuff logax = logbx / logba because

and more log stuff

log n and algorithms In algorithm analysis we often use log n when we should use floor(log(n)). Is that OK? When in each step of an algorithm we halve the size of the problem (or divide it by k) then it takes log n steps to get to the base case Notice that logb1n = O(logb2n) for any b1 and b2, so the base does not matter in O analysis Does that work for exponents too? Is 2n = O(3n) ? Is 3n = O(2n)?

log n and algorithms In algorithm analysis we often use log n when we should use floor(log(n)). That's OK floor(log(n)) = O(log(n)) When in each step of an algorithm we halve the size of the problem (or divide it by k) then it takes log n steps to get to the base case Notice that logb1n = O(logb2n) for any b1 and b2, so the base does not matter in O analysis Does that work for exponents too? Is 2n = O(3n) ? Is 3n = O(2n)?

log n and algorithms DivCo(P){ Algorithms with O(log n) complexity are often of a simple divide and conquer variety (base, solve-direct, and split are O(1)): DivCo(P){ if base(P) solve-direct(P) else split(P)  P1 ... Pn DivCo(Pi) } eg Binary Search

log n and algorithms DivCo(P){ Algorithms with O(log n) complexity are often of a simple divide and conquer variety (base, solve-direct, and split are O(1)): DivCo(P){ if base(P) solve-direct(P) else split(P)  P1 ... Pn DivCo(Pi) } Solve f(n) = f(n/2) + 1 f(1)=1 by repeated substitution

General Divide and Conquer DivCo(P) { if base(P) solve-direct(P) else split(P)  P1 ... Pn combine(DivCo(P1) ... DivCo(Pn)) } Depending on the costs of base, solve direct, split and combine we get different complexities (later).

f(n)=O(n) Linear complexity eg Linear Search in an unsorted list also: Polynomial evaluation A(x) = anxn+ an-1xn-1+...+ a1x+a0 (an!=0) Evaluate A(x0) How not to do it: an * exp(x,n)+ an-1*exp(x,n-1)+...+ a1*x+a0 why not?

How to do it: Horner's rule y=a[n] for (i=n-1;i>=0;i--) y = y *x + a[i]

Horner complexity Lower bound: Ω(n) because we need to access each a[i] at least once Upper bound: O(n) Closed problem But what if A(x) = xn Horner not optimal for xn

A(x)=xn Recurrence: x2n=xn.xn x2n+1=x.x2n y=1; Complexity? while(n!=0){ if(odd(n)) y=y*x; x=x*x; n = n/2; } Complexity?

O(n log(n)) Often resulting from divide and conquer algorithms where split & combine are O(n) and we divide in nearly equal halves. mergesort(A){ if size(A) <= 1 return A else return merge(mergesort(left half(A)), mergesort(right half(A)))

Merge Sort - Split {7,3,2,9,1,6,4,5} {7,3,2,9} {1,6,4,5} {7,3} {2,9} {1,6} {4,5} Give out cards, order suits: hearts, diamonds, spades, clubs Ask them to sort this way. Point out physical ease. {7} {3} {2} {9} {1} {6} {4} {5}

Merge Sort - Merge {1,2,3,4,5,6,7,9} {2,3,7,9} {1,4,5,6} {3,7} {2,9} {1,6} {4,5} {7} {3} {2} {9} {1} {6} {4} {5}

Merge sort complexity Time Space Total cost of all splits? Cost of each merge level in the tree? How many merge levels in the tree? Space Is extra space needed? (outside the array A) If so, how much?

Series

Geometric Series

Harmonic series why?

Products why?

Using integrals to bound sums If f is a monotonically growing function, then:

f(x1)+f(x2)+f(x3)+f(x4)<=f(x2)+f(x3)+f(x4)+f(x5) x1 x2 x3 x4 x5

some formula manipulation....

Example of use

using the integral bounds

concluding...

Sorting lower bound We will use in proving a lower bound on sorting