1. October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson L2.1 Introduction to Algorithms 6.046J/18.401J LECTURE9 Randomly built binary search trees Expected node depth Analyzing eight Convexity lemma Jensens inequality Exponential height Post mortem Prof. Erik Demaine

2. L2.2October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Binary-search-tree sort T Create an empty BST for i= 1 to n do TREE-INSERT (T, A[i]) Perform an inorder tree walk of T. Example: A= [3 1 8 2 6 7 5] Tree-walk time = O(n), but how long does it take to build the BST?

3. L2.3October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Analysis of BST sort BST sort performs the same comparisons as quicksort, but in a different order! The expected time to build the tree is asymptot- icallythe same as the running time of quicksort.

4. L2.4October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Node depth The depth of a node=the number of comparisons made during TREE-INSERT. Assuming all input permutations are equally likely, we have Average node depth (quicksort analysis) (#comparison to insert node i )

5. L2.5October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Expected tree height But, average node depth of a randomly built BST = O(lg n)does not necessarily mean that its expected height is also O(lg n)(although it is). Example.

6. L2.6October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Height of a randomly built binary search tree Outline of the analysis: Prove Jensens inequality, which says that f(E[X]) E[f(X)] for any convex function fand random variable X. Analyze the exponential height of a randomly built BST on n nodes, which is the random variable Yn= 2 Xn, where Xn is the random variable denoting the height of the BST. Prove that 2 E[Xn]E[2Xn ] = E[Yn] = O(n3), and hence that E[Xn] = O(lg n).

7. L2.7October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Convex functions A function f : RR is convex if for all α,β0 such that α+ β= 1, we have f(αx+ βy) α f(x) + β f(y) for all x,y R.

8. L2.8October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Convexity lemma Lemma. Let f : R R be a convex function, and let α 1, α 2, …, α n be nonnegative real numbers such that Σ k α k = 1. Then, for any real numbers x 1, x 2, …, x n, we have Proof. By induction on n. For n = 1, we have α 1 = 1, and hence f(α 1 x 1 ) α 1 f(x 1 ) trivially.

9. L2.9October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Proof (continued) Inductive step: Algebra.

10. L2.10October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Proof (continued) Inductive step: Convexity.

11. L2.11October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Proof (continued) Inductive step: Induction.

12. L2.12October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Proof (continued) Inductive step: Algebra.

13. L2.13October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Convexity lemma: infinite case Lemma. Let f : R R be a convex function, and let α 1, α 2, …, be nonnegative real numbers such that Σ k α k = 1. Then, for any real numbers x 1, x 2, …, we have assuming that these summations exist.

14. L2.14October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Convexity lemma: infinite case Proof. By the convexity lemma, for any n1,

15. L2.15October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Convexity lemma: infinite case Proof. By the convexity lemma, for any n1, Taking the limit of both sides (and because the inequality is not strict):

16. L2.16October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Jensens inequality Lemma. Let f be a convex function, and let X be a random variable. Then, f(E[X]) E[f(X)]. Proof. Definition of expectation.

17. L2.17October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Jensens inequality Lemma. Let f be a convex function, and let X be a random variable. Then, f(E[X]) E[f(X)]. Proof. Convexity lemma (infinite case).

18. L2.18October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Jensens inequality Lemma. Let f be a convex function, and let X be a random variable. Then, f(E[X]) E[f(X)]. Proof. Tricky step, but truethink about it.

19. L2.19October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Analysis of BST height Let X n be the random variable denoting the height of a randomly built binary search tree on n nodes, and let Y n = 2X n be its exponential height. If the root of the tree has rank k, then X n = 1 + max {X k –1,X n –k}, since each of the left and right subtrees of the root are randomly built. Hence, we have Y n = 2·max {Y k –1,Y n –k}.

20. L2.20October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Analysis (continued) Define the indicator random variable Z nk as if the root has rank k, otherwise. Thus, Pr{Z nk = 1} = E[Z nk ] = 1/n, and

21. L2.21October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Exponential height recurrence Take expectation of both sides.

22. L2.22October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Exponential height recurrence Linearity of expectation.

23. L2.23October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Exponential height recurrence Independence of the rank of the root from the ranks of subtree roots.

24. L2.24October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Exponential height recurrence The max of two nonnegative numbers is at most their sum, and E[Z nk ] = 1/n.

25. L2.25October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Exponential height recurrence Each term appears twice, and reindex.

26. L2.26October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Solving the recurrence Use substitution to show that E[Y n ] cn 3 for some positive constant c, which we can pick sufficiently large to handle the initial conditions.

27. L2.27October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Solving the recurrence Use substitution to show that E[Y n ] cn 3 for some positive constant c, which we can pick sufficiently large to handle the initial conditions. Substitution.

28. L2.28October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Solving the recurrence Use substitution to show that E[Y n ] cn 3 for some positive constant c, which we can pick sufficiently large to handle the initial conditions. Integral method.

29. L2.29October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Solving the recurrence Use substitution to show that E[Y n ] cn 3 for some positive constant c, which we can pick sufficiently large to handle the initial conditions. Solve the integral.

30. L2.30October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Solving the recurrence Use substitution to show that E[Y n ] cn 3 for some positive constant c, which we can pick sufficiently large to handle the initial conditions. Algebra.

31. L2.31October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson The grand finale Putting it all together, we have 2 E[Xn]E[2 Xn ] Jensens inequality, since f(x) = 2 x is convex.

32. L2.32October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson The grand finale Putting it all together, we have 2 E[Xn]E[2 Xn ] = E[Y n ] Definition.

33. L2.33October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson The grand finale Putting it all together, we have 2 E[Xn]E[2 Xn ] = E[Y n ] cn 3. What we just showed.

34. L2.34October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson The grand finale Putting it all together, we have 2 E[Xn]E[2 Xn ] = E[Y n ] cn 3. Taking the lg of both sides yields E[X n ] 3lg n +O(1).

35. L2.35October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Post mortem Q. Does the analysis have to be this hard? Q. Why bother with analyzing exponential height? Q. Why not just develop the recurrence on X n = 1 + max{X k –1,X n –k} directly?

36. L2.36October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Post mortem (continued) A. The inequality max{a,b} a+ b. provides a poor upper bound, since the RHS approaches the LHS slowly as |a–b| increases. The bound max{2 a,2 b } 2 a + 2 b allows the RHS to approach the LHS far more quickly as |a–b| increases. By using the convexity of f(x) = 2 x via Jensens inequality, we can manipulate the sum of exponentials, resulting in a tight analysis.

37. L2.37October 17, 2005 Copyright©2001-5 Erik D. Demaine and Charles E. Leiserson Thought exercises See what happens when you try to do the analysis on X n directly. Try to understand better why the proof uses an exponential. Will a quadratic do? See if you can find a simpler argument. (This argument is a little simpler than the one in the bookI hope its correct!)