1. Chapter 7: Quick Sort Recursive sorting using Partition
Partition: algorithm that enable divide and conquer
Worst-case runtime of Quicksort = O(n2)
Best-case runtime of Quicksort = W(nlgn)
Quicksort is asymptotically optimal
Is worst case not a good characterization of Quicksort run time?
Randomized algorithms

2. Pseudocode Quicksort(A,p,r)
% to sort whole array p=1, r=length[A] if p < r then
q  Partition(A,p,r)
% q is index that defines subarrarys
Quicksort(A,p,q-1)
Quicksort(A,q+1,r)

3. Partition: Algorithm for Divide and Conquer

4. Continue until j = r-1, then exchange A[i+1] and A[r]
Note: upper and lower sub-arrays are not the same size
and not sorted
Note: pivot is sorted relative elements in upper and lower
sub-arrays
Partition runtime: T(n) =Q(n) why?

5. Quicksort’s worst case
Quicksort(A,p,r])
if p < r then
q  Partition(A,p,r)
% q is the index of pivot
Quicksort(A,p,q-1)
Quicksort(A,q+1,r)
Worst-case partitioning: (always maximum imbalance)
Partition(A,p,r) always returns q = p
What type of input would make this happen?
A worst-case Quicksort(A,p,r])
p  Worst Partition(A,p,r)
Quicksort(A,p+1,r)
What happened to the lower sub-array?
What recurrence describes this worst case?
What is its solution?

6. Worst case input is sorted in decreasing order
Worst case input is sorted in decreasing order. In very call to partition,
A(r) is the smallest element.
Worst-case runtime of Quicksort described by
T(n) = T(n-1) + Q(n)
In HW12 showed by tree analysis and informal proof that T(n) = Q(n2)

7. More rigorous worst-case analysis
We argued for T(n)=T(n-1)+Q(n) as worst case for Quicksort
based on a particular input (sorted in decreasing order).
A derive a rigorous recurrence for worst-case Quicksort:
Assume elements of A are indexed 0 to n-1
lower array contains elements 0 to q-1
upper array contains elements q+1 to n-1
On the 1st call to Partition, 0 < q < n-1
Include the pivot in the lower sub-array
Find the value of q that maximizes T(n)
T(n) = (T(q) + T(n-q-1)) + Q(n)
Show by the substitution method that solution is T(n)=O(n2).

8. T(n) = (T(q) + T(n-q-1)) + Q(n)
Replace Q(n) by dn
Assume: T(q)=O(q2), T(n-q-1)=O((n-q-1)2)
Exist c>0 such that T(q)<cq2 and T(n-q-1)<c(n-q-1)2
T(n) < (cq2 + c(n-q-1)2) + dn
< c ( (q2 + (n-q-1)2)) + dn
f(q)=q2+((n-1)-q)2 df/dq=2q-2(n-q-1)=0 qm = (n-1)/2
d2f/dq2 = 4 therefore f(q) is minimum at qm
f(0)=(n-1)2 and f(n-1)=(n-1)2 so f(q) is maximum at q= 0 and n-1
If qmax= 0 or qmax= n-1 then T(n) < c(n-1)2 + dn < cn2
which is the same inequality we encounter in proof that T(n)=T(n-1)+dn has asymptotic solution T(n)=O(n2)

9. Best case runtime of Quicksort described by
Use T(n) = (T(q) + T(n-q-1)) + Q(n)
to show by structured substitution that T(n) has asymptotic
solution T(n)=W(nlgn).
This proves that Quicksort is asymptotically optimal

12. Therefore T(n)=W(nlgn) by definition

13. Suppose partition always returned split 9n/10 and n/10
Will the performance of Quicksort be O(n2)?
What recurrence describes this performance?
What is its solution?

14. T(n)=T(9n/10)+T(n/10)+Q(n)
Guess solution by imbalanced tree analysis.

15. CptS 350 Spring 2019
Homework Assignment 14: due 3/8/19
Prove by structured substitution that T(n)=T(9n/10)+T(n/10)+Q(n) has
asymptotic solution T(n)=Q(nlgn)

16. Summary:
Worst case, imbalance 0 and n-1, Quicksort: T(n)=O(n2)
With any degree of imbalance other than 0 and n-1: T(n)=Q(nlgn)
Questions:
Is the worst case an outlier?
What is the runtime of Quicksort with a “typical” input?
Define a stochastic version of Quicksort: Randomized Quicksort
Find the expected value of the runtime
In general, “randomized” sorting means “Given array of n elements to be
sorted, choose at random one of the n! permutations of the input”
For analysis of Randomized Quicksort, a different method of randomizing
is easier to analyze.

17. Randomized by a random choice of pivot.
Randomized Quicksort
Randomized by a random choice of pivot.
Randomized-Partition(A, p, r)
i  Random(p, r)
exchange A[r]  A[i]
return Partition
Randomized-Quicksort(A, p, r)
if p < r then
q  Ramdomized-Partition(A, p, r)
Randomized-Quicksort(A, p, q-1)
Randomized-Quicksort(A, q+1, r)
Randomized-Partition just calls Partition after randomly choosing a pivot.
Runtime of Randomized Quicksort is a random variable.
Random choice of pivot varies the degree of imbalance between upper and lower subarrarys

18. Expected run time of Randomized Quicksort
What is the expected number of comparison from all calls to Randomized-Partition in a complete execution of Randomized-Quicksort(A,1,n)?
Before we answer this question
Review probability theory (Appendix C)
Learn the technique of “indicator random variables”

19. Appendix C.2: Theory discrete probabilities
Discrete probabilities are defined in terms of a sample space S.
Usually S is a collection of elementary events that are outcomes of independent experiments, such as flipping coins, throwing dice, pulling cards, etc.
|S| = size of S (also called “cardinality” of S)
Example: S = set of outcomes from flipping 2 coins = {HH, HT, TH, TT}
Events are subsets of S. (S itself is called the “certain” set.)
The event of getting 1 head and 1 trail = {HT, TH}
The empty subset, , called the “null event”

20. Brief review of set theory (Appendix B.1, text p1158)
If x  A implies x  B, then A is a subset of B (written A  B)
If, in addition, A  B then A is a proper subset of B (written A  B)
A  B = intersection of sets A and B defined by {x: x  A and x  B}
A  B = union of sets A and B defined by {x: x  A or x  B}
A - B = difference of sets A and B defined by {x: x  A and x  B}
For laws obeyed by sets see text pp

21. Definition of probability on sample space S
(1) Pr{A} > 0 for any event A
(2) Pr{S} = 1
(3) for any 2 mutually exclusive events (i.e. A  B = )
Pr{A  B} = Pr{A} + Pr{B}
By definition, elementary events are always mutually exclusive.
From the definition of discrete probabilities, it follows that
Pr{} = 0
If A  B then Pr{A} < Pr{B}
Pr{S – A} = 1- Pr{A} S-A is the complement of A
For any two events (not necessarily mutually exclusive)
Pr{A  B} = Pr{A} + Pr{B} – Pr{A  B} < Pr{A} + Pr{B}

22. Discrete probability distributions
Defined over a finite or countable infinite sample space
Let s denote elementary events of S,
then for any event A Pr{A} =  sA P{s}
If S is finite and every sS has probability 1/|S|,
then P(s) is uniformly distributed on S.
Example: S = {HH, HT, TH, TT} = elemental events from flipping 2 coins,
Pr{HH} = Pr{HT} = Pr{TH} = P{TT} = ¼ for “fair” coins
The probability of at least one head is
Pr{HH, HT, TH} = Pr{HH} + Pr{HT} + Pr{TH} = ¾
= 1 – Pr{TT} = 1 – ¼ = ¾

23. A “fair” coin flipped n times defines a uniform probability distribution on S
Elementary events are strings HHTHT…
For each of n positions in the string we have 2 choices of letter
|S| = 2n
For n=2, S = {HH, HT, TH, TT},
What is the probability of event A = {exactly k heads in n-k trails}?
Pr{A} = (nk)/2n where
(nk) = n!/(k!(n-k)!)
= number ways to choose k items out of a total of n
(nk) are binomial coefficients (see text pp )
S is normalized

24. Conditional probability: Given some knowledge about outcomes, we want
the probability of an outcome conditioned on our prior knowledge about it.
Example:
Suppose that someone flips 2 coins and tells us that at least one coin
shows heads. What is the probability that both coins are showing heads?
S = {HH, HT, TH, TT}
Our prior knowledge eliminates elementary event TT.
The remaining 3 possibilities are equally likely.
Pr{HH} conditioned on at least 1 head is showing = 1/3 > Pr{HH} in the
absence of any information about events.

25. Conditional probability: general definition
Probability of A conditioned on B, written Pr{A|B}, is meaningful only
if Pr{B}  0.
Given that B occurs, the probability that A also occurs is related
to the set of outcomes in which both A and B occur.
 Pr{A|B} is proportional to Pr{AB}
If we normalize Pr{A|B} by dividing by Pr{B} (which ≠ 0), then
Pr{B|B} = Pr{BB}/Pr{B} = Pr{B}/Pr{B} = 1

26. Apply Pr{A|B} = Pr{AB}/Pr{B} to the problem
Conditional probability: application of definition
Apply Pr{A|B} = Pr{AB}/Pr{B} to the problem
“probability of 2 heads showing given that at least one head is showing”
AB is the event with 2 heads showing and at least one head showing.
In S = {HH, HT, TH, TT}, HH is the only such event.
Pr{AB} = ¼
B is the event with at least one head showing.
3 out of 4 events in S have this property; therefore Pr{B} = ¾
Using the definition, Pr{A|B} = Pr{AB}/Pr{B}
Pr{A|B} = (1/4)/(3/4) = 1/3 > Pr{AB}
Additional knowledge increased probability

27. Two events are independent if Pr{AB} = Pr{A}Pr{B}
Conditional probability: application to independent events
Two events are independent if Pr{AB} = Pr{A}Pr{B}
If events A and B are independent and Pr{B}  0,
then Pr{A|B} = Pr{A}Pr{B}/Pr{B} = Pr{A}

28. Discrete random variables:
Mappings of finite or countable infinite sets of events onto the real numbers
For random variable X and real number x, the event X = x is the
collection of all elementary events {sS: X(s) = x}.
Pr{X = x} = {sS: X(s) = x} Pr{s}
f(x) = Pr{X = x} is the “probability density function” of random variable X
By axioms of probability f(x) > 0 and x f(x) = 1

29. Example of discrete random variable:
A pair of 6-sided dice is rolled.
Random variable X is the max of the 2 numbers showing.
What is Pr{X = 3}? Use exhaustive enumeration
Elementary events with 3 as max of the 2 numbers showing
(1,3), (2,3), (3,3), (3,2), and (3,1).
The probability of each elementary event is (1/6)(1/6) = 1/36
 Pr{X = 3} = 5/36

30. Expected value of a discrete random variable
E[X] = x xPr{X=x}
If g(X) defines a new random variable,
then E[g(X)] = x g(x)Pr{X=x}
Special case: Given random variables X and Y,
the property of sums implies
E[aX + bY] = aE[X] + bE[Y]
Called “linearity” of expectation values

31. For any two random variables X and Y
joint probability density function is Pr{X=x and Y=y}
for any fixed value of y Pr{Y=y} = x Pr{X=x and Y=y}
called “marginal” probability
Pr{X=x | Y=y} = Pr{X=x and Y=y}/Pr{Y=y}
called “conditional” probability
X and Y are independent if for all x and y
Pr{X = x and Y = y} = Pr{X = x}Pr{Y = y}
If X and Y are independent, then E[XY] = E[X]E[Y]
proof on p1198 of text

32. Indicator Random Variables
For any event A,
Indicator Random Variable I{A} = 1 if A occurs
= 0 if A does not occur
Zero and one are the only values an indicator random
variables can have
Lemma 5.1:
Let A be an event in sample space S, then the expected
value of I{A} = probability of event A.
Proof:
For any random variable X, E[X] = x x Pr{X = x}
For indicator random variables x = 0 or 1, only
The complement of event A in sample space S is S-A
E[I{A}] = (1)Pr(A) + (0)Pr(S - A) = Pr(A)

33. Structured solution to
“What is the expected number of heads for n coin flips?”
solved by indicator random variables
What is the random variable whose expectation value
we want to calculate?
2. What event defines the indicator random variable?
3. What is the probability that this occurs?
4. How is random variable whose expectation we are
calculating related to the indicator random variable?
5. What is the expectation value of this random variable?

34. Structured solution to
“What is the expected number of heads for n coin flips?”
solved by indicator random variables
1. What is the random variable whose expectation value we want
to calculate? X = number of heads showing in n coins flips
2. What event defines the indicator random variable?
Ai = ith coin flip shows heads. Let I{Ai} = Xi
3. What is the probability that Ai occurs? E(Xi) = ½
4. How is random variable whose expectation we are
calculating related to the indicator random variable?
X = i=1 to n (Xi)
5. What is the expectation value of this random variable?
E[X] = E[i=1 to n (Xi)] = i=1 to n E[Xi] = i=1 to n (1/2) = n/2

35. Expected run time of Randomized Quicksort using indicator random variables
What is the random variable whose expectation value we wish to calculate?
X=number of comparison from all calls to Randomized-Partition in a complete execution of Randomized-Quicksort(A,1,n)

36. X=number of comparison from all calls to Randomized-Partition in a complete execution of Randomized-Quicksort(A,1,n)
What is the event that defines the indicator random variable?
Let Aij be the event when element i compared to j
Xij = I{Aij}
How is X related to the indicator random variable?
X = sum over all distinct ordered pairs
This is correct iff any pair of elements can be compared at most once.

37. Last line of Partition stores the pivot between partitioned subarrays
Once used, a pivot is never involved in subsequent recursive calls
In sorting an array of length n, Partition called at most n times
Elements are compared to pivots only
Elements in separate subarrays are never compared
X= is the correct relationship random variable and
indicator random variable

38. Rename the elements <a1, a2,
Rename the elements <a1, a2, ..., an> of array A by their order statistic
Call then z1, z2, ..., zn, where zi is the ith smallest element
z1 is the minimum element, zn is maximum element
Define subsets of elements Zij = {zi, zi+1, ..., zj} to be the elements (by size)
between zi and zj inclusively
Since elements are compared to pivots only and elements in separate
subarrays are never compared, zi and zj are compared, at most, once
when one or the other is a pivot.
X = is the correct relationship between X and I{Aij}
E(X) = =

39. zi and zj are compared iff the 1st pivot chosen from the set
Zij={zi, zi+1,...zj} is either zi and zj.
If zi is the 1st pivot chosen from the set Zij then zj cannot be the
1st pivot chosen from that set
two possibilities leading to comparison of zi and zj are mutually exclusive.
 Pr{ zi is compared to zj} = Pr{ zi is the 1st pivot chosen from Zij} +
Pr{ zj is the 1st pivot chosen from Zij}
= 2/(j – i + 1)
because all elements in Zij equally likely to be first chosen

40. E(X) = =
Evaluate this sum

41. E[X] =
E[X] = (by change of variable k = j-i)
E[X] <
E[x] < O(lg(n))
E[x] = O(nlgn)
k+1 -> k increases every term
extending inner sum to n gives harmonic sum
value, ln(n) + O(1), differs from lg(n) by constant

42. Cpt S 350 Spring 2019
Homework Assignment 15: due 3/20/19
Use indicator random variables to compute the expected value of the sum of faces showing when n fair dice are tossed.
Give a structured solution.

43. Chapter 9: Selection of Order Statistics
Since this is another application of partition, delay Quiz #4 until we cover this material