CS420 lecture three Lowerbounds wim bohm, cs CSU

Big O, big omega, big theta f(x) = O(g(x)) iff there are positive integers c and n 0 such that f(x) n 0 f(x) = Ω(g(x)) iff there are positive integers c and n 0 such that f(x) > c.g(x) for all x > n 0 f(x) = Θ(g(x)) iff f(x) = O(g(x)) and f(x) = Ω(g(x))

Big O etc. Big O used in upper bounds, ie the worst or average case complexity of an algorithm. Big theta is a tight bound, eg stronger than upper bound. Big omega used in lower bounds, ie the complexity of a problem.

lower bounds An easy lower bound on a problem is the size of the output it needs to produce, or the number of inputs it has to access – Generate all permutations of size n, lower bound? – Towers of Hanoi, lower bound? – Sum n input integers, lower bound? but... sum integers 1 to n?

some Ω(n) problems Given an unsorted array A and a number p, rearrange A such that all A[i] x

some Ω(n) problems Given an unsorted array A and a number x, rearrange A such that all A[i] =x L=1; R=n; while(L<R){ while((A[L]<=x)&&(L<R)L++ while((A[R]>=x)&&(L<R))R- - if(L<R)swap(A,L,R) }

Dutch National Flag

flag

Comparison problems We have seen search.

Comparison problems We have seen search. Why is it closed?

Comparison problems We have seen search. Why is it Ω(logn)?

Comparison problems We have seen search. What are the allowed operations?

Comparison problems We have seen search. Why is the decision tree binary?

Comparison problems We have seen search. At least how many leaves?

Comparison problems We have seen search. At least how high? Why?

Comparison problems Let's try the same for sort we have an O(n logn) upper bound what would be the allowed steps?

Comparison problems Let's try the same for sort we have an O(n logn) upper bound comparison, array element assignment, index arithmetic, establish sorted

Comparison problems Let's try the same for sort we have an O(n logn) upper bound is the decision tree binary again?

Comparison problems Let's try the same for sort we have an O(n logn) upper bound how many leaves this time?

Comparison problems Let's try the same for sort we have an O(n logn) upper bound every leaf represents a specific permutation of the array

Comparison problems Let's try the same for sort we have an O(n logn) upper bound how many different permutations?

Comparison problems Let's try the same for sort we have an O(n logn) upper bound n! how high will the decision tree at least be?

Comparison problems Let's try the same for sort we have an O(n logn) upper bound the decision tree will be at least log(n!) high log(n!) = O(nlogn) so sort is Ω(nlogn), and thus closed.

Tournaments Selecting the maximum element of an array. View it as a tournament. Comparison is a match, winning is transitive: – A>B and B>C implies A>C, so A does not need to play C, and B and C can never be the winner of the tournament Many possible O(n) algorithms tennis(S): if (|S|==1) S 1 else max(tennis(topHalf(S)), tennis(bottomHalf(S))) sweep(S): winner= S 1 for i=2 to n winner = max(winner,S i )

Runner up a b c d e f g h b a d c f e h g tennis sweep which gives more info about runner up?

Assumptions about initial input Transitive and total ordering – transitive: a>b and b>c implies a>c (no roshambo) – total: for all a and b: either a>b or b>a – ie the elements can be ranked Any assignment of ranks to names is feasible – in previous example we could assign any of the 8! permutations of rankings 1 to 8 to a to h Because we look for the winner, anyone who loses stops playing. Therefore there will be n-1 (n is number of inputs) games / comparisons for any tournament – notice: both sweep and tennis have 7 matches

Simpler case: n=4 Any of the 24 assignments of 1..4 to a..d is valid Build a decision tree for sweep (for n = 4), which is sometimes called "caterpillar"

Decision tree for sweep Any of the 24 assignments of 1..4 to a..d is valid Build a decision tree for sweep (for n = 4) a:b b a a:c c a b:c c b c:d d c b:d d b c:d d c a:d d a

Decision tree for sweep Any of the 24 assignments of 1..4 to a..d is valid Build a decision tree for sweep (for n = 4) a:b b a a:c c a b:c c b c:d d c b:d d b c:d d c a:d d a what do we know about the runner up for this outcome?

Decision tree for sweep Any of the 24 assignments of 1..4 to a..d is valid Build a decision tree for sweep (for n = 4) a:b b a a:c c a b:c c b c:d d c b:d d b c:d d c a:d d a what do we know about the runner up for this outcome?

tennis Build a decision tree for tennis (for n = 4)

Decision tree for tennis Build a decision tree for tennis (for n = 4) a:b b a c:d d c d c b:d d b b:c c b a:d d a a:c c a

Decision tree for tennis Build a decision tree for tennis (for n = 4) a:b b a c:d d c d c b:d d b b:c c b a:d d a a:c c a what do we know about the runner up for this outcome?

Decision tree for tennis Build a decision tree for tennis (for n = 4) a:b b a c:d d c d c b:d d b b:c c b a:d d a a:c c a what do we know about the runner up for this outcome?

Runner up In “The Art of Computer Programming, vol 3”, Donald Knuth proved that the tennis tournament algorithm provides the lower bound to determine the runner up: In the case of tennis we need lg n more comparisons to find the runner up