Tomado del libro Cormen nalysis of A lgorithms Tomado del libro Cormen

OUTLINE Quicksort Description Analysis Randomized Quicksort

Quicksort

Quicksort Quicksort(A,p,r) if p < r q  Partition(A,p,r) Quicksort(A,p,q-1) Quicksort(A,q+1,r)

Partition - two finger algorithm Partition(A,p,r) choose an element to be a pivot and pull it out of the array, say at left end maintain two fingers starting at each end of the array slide them towards each other until you get a pair of elements where right finger has a smaller element and left finger has a bigger one (when compared to pivot) swap them and repeat until fingers meet put the pivot element where they meet.

Partition - two finger algorithm Partition(A,p,r) pivot  A[p] i  p j r while i < j do while (A[i]  pivot & i  r) do i  i+1 while (A[j] > pivot & j  p) do j  j-1 if i < j then A[i]  A[j] end-while A[p]  A[j] return j

Partition ( A,1,8) pivot 2 p r 1 2 3 4 5 6 7 8 A 2 8 7 1 3 5 6 4 i j

pivot 2 1 2 3 4 5 6 7 8 2 8 7 1 3 5 6 4 i j 1 2 3 4 5 6 7 8 2 1 7 8 3 5 6 4 j i

pivot 2 1 2 3 4 5 6 7 8 2 1 7 8 3 5 6 4 j i 1 2 3 4 5 6 7 8 1 2 7 8 3 5 6 4 j i

Loop Invariant for Partition At the start of each iteration of the while loop and for any array index k, 1. If p  k  i-1, then A[k]  pivot. 2. If j+1  k  r, then A[k] > pivot.

Partition - four finger algorithm Partition(A,p,r) pivot A[r] i  p - 1 for j  p to r-1 do if A[j]  pivot then i  i + 1 A[i]  A[j] A[i + 1]  A[r] return i + 1

Partition ( A,1,8) pivot 4 p r 1 2 3 4 5 6 7 8 A 2 8 7 1 3 5 6 4 i j

pivot 4 1 2 3 4 5 6 7 8 2 8 7 1 3 5 6 4 j i 1 2 3 4 5 6 7 8 2 8 7 1 3 5 6 4 i j

pivot 4 1 2 3 4 5 6 7 8 2 8 7 1 3 5 6 4 i j 1 2 3 4 5 6 7 8 2 1 7 8 3 5 6 4 i j

pivot 4 1 2 3 4 5 6 7 8 3 2 1 8 7 5 6 4 i j 1 2 3 4 5 6 7 8 3 2 1 8 7 5 6 4 i j

pivot 4 1 2 3 4 5 6 7 8 3 2 1 8 7 5 6 4 i j 1 2 3 4 5 6 7 8 3 2 1 4 7 5 6 8 i j

Loop Invariant for Partition At the start of each iteration of the for loop and for any array index k, 1. If p  k  i, then A[k]  pivot. 2. If i +1  k  j-1, then A[k] > pivot. 2. If k = r, then A[k] = pivot.

IN PRACTICE Often choose pivot in fixed way as middle element for small arrays median of 1st, middle, and last for larger arrays median of 3 medians of 3 (9 elements in all) for largest arrays

Analysis Partition does n-1 comparisons on a list of length n pivot is compared to each other element. If pivot is ith largest then two subproblems are of size i-1 and n-i. Pivot is equally likely to be any one of 1st through nth largest

BEST CASE WORST CASE

AVERAGE CASE

Randomized Quicksort RandQS

RandPartition(A,p,r) Randomized Partition A[p]  A[random(p,r)]

RandQS RandQS(A,p,r) if p < r q  RandPartition(A,l,r) RandQS(A,p,q-1) RandQS(A,q+1,r)

Input: a set of numbers A Output: the elements of A sorted in increasing order. Steps: 1. Choose and element q uniformly at random form A. 2. Determine the set A1 of elements of A smaller that q, and the set A2 of elements of A larger that q 3. Recursively sort A1 and A2 .

Lemma 7.1 Let be X the number fo comparisons A[j]  pivot in partition. Then the running time of Quicksort and RandQS is T(n) = O(n+X)

Expected number of Comparisons Let be {z1 , z2 , …, zn } the sorted permutation of the elements of A, and Zij = {zi , zi+i, …, zj }. 1 if zi and zj are compared Xij = 0 in other case

Total number of comparisons

Expected value

Calculating Pr[ zi is compared to zi] In general, once a pivot q is chosen with zi < q <, zj, we know that zi and zj cannot be compared in subsequent time. If zi is chosen as pivot before any other item of Zij , then zi will be compared to each item of Zij except itself.

Simiarly, if zj is chosen as pivot before any other item of Zij , then zj will be compared to each item of Zij except itself.

2 | 3 | 5| 7 | 1| 4 | 8 | 6 q 2 | 3 | 5| 1| 4 | 6 7 8 q 2 | 3 | 5| 1| 4 | 6 7 8

2 | 3 | 1| 4 5 6 7 8 q 7 5 8 6 2 | 3 | 1| 4

2 | 1 3 4 5 6 7 8 q 7 5 8 3 6 4 2 | 1

7 5 8 3 6 2 4 1 1 | 2 | 3| 4 | 5| 6 | 7 | 8  = { 7 , 5, 8, 3, 6, 2, 4, 1}

 = { 7 , 5, 8, 3, 6, 2, 4, 1} root q Level 1

There is a comparison between zi and zj if and only if zi or zj occurs earlier in the permutation  than any element zk with i < k < j. Prior to the point at which an element from Zij has been chosen as pivot, the whole set Zij is in the same partition. Therefore, any element of Zij is equally likely to be chosen as pivot.

Zij has j-i+1 elements. The probability that any given element is the first one chosen as pivot is

Pr[ zi is compared to zi]

The Expected Time E[T(n)] = O(n+E[X]) = O(n+n lgn) = O(n lgn)

Observations There is more than one possible execution path for a given input By example, for n = 3 if A=[2,3,1] we have the following execution paths:

The probability of a given execution path can be different of the probability of another execution path. By example, for A=[2,3,1] n = 3 we have the following execution paths with its probabilities:

Pivot 1 1 1/3 1/3 1 3 1/2 1/3 2 2 3 1/6 1/6

Pivot 2 1/3 2 1 3 1/3

Pivot 3 3 1/3 1/3 3 1 1/2 2 1/3 2 1 1 1/6 1/6

Deviations of the expected time for RandQs (tail probabilities) PROBABILTY OF DEVIATION OF THE EXPECTED TIME We calculate the probability that a randomized algorithm follows and execution path that takes a time larger that the expected time.

Probability distribution for T(n) Pr[T(n)=k] k E[T(n)] r E[T(n)] Pr[T(n)  r E [T(n)] ]

In the case of RandQs we have that T(n) is a random variable that satisfies: n lgn  T(n)  n2 Best case time Worst case time

PROBABILTY INEQUALITIES USED Markov Inequality (bound) Chebischev Inequality (bound)

Proof of Markov bound

CHERNOFF BOUND Bernoulli trials: Let Xi , 1 i  n random variables with 1(success) with probability p Xi = 0(failure) with probability q=1-p

Binomial random variable: Let Xi be n independent Bernoulli trials then is a Binomial random variable with

Poisson trials: Let Xi , 1 i  n random variables with 1(success) with probability pi Xi = 0(failure) with probability qi=1-pi

THEOREM:Chernoff bound for sum of independent Poisson trials Let Xi be n independent Bernoulli trials with Then for

and

And for any  > 0

CHERNOFF BOUND in RandQS E[X]= O(n lgn)

THEOREM: RandQs runs in n lgn with high probability for n  16 Proof: Pr that will run in more tha 1.5 times the expected time

Proof of Chernoff bound: for any t > 0 and

and

then

using the inequality 1+x <= ex

using the inequality 1+x <= ex Minimizing the right expression