1. Shuchi Chawla, Carnegie Mellon University Static Optimality and Dynamic Search Optimality in Lists and Trees Avrim Blum Shuchi Chawla Adam Kalai 1/6/2002

2. Shuchi Chawla, Carnegie Mellon University List Update Problem  Unordered list  Access for x i takes i time 472912368 Query for element 9 = No = = = Yes

3. Shuchi Chawla, Carnegie Mellon University List Update Problem  Unordered list  Access for x i takes i time 947212368 Query for element 9  Replacement cost = 0  Reorder cost = 1 per operation  What should the reordering policy be?

4. Shuchi Chawla, Carnegie Mellon University Binary Search Tree  “In-order” tree  Search cost = depth of element  Cost of rotations = 1 per operation  Replacement policy = ? 7 212 914 15 54

5. Shuchi Chawla, Carnegie Mellon University Binary Search Tree  “In-order” tree  Search cost = depth of element  Cost of rotations = 1 per operation  Replacement policy = ? 7 2 12 9 14 1554 Query for element 9

6. Shuchi Chawla, Carnegie Mellon University How good is a reordering algorithm?  Compare against the best “offline” algorithm  Dynamic competitive ratio  Best offline algorithm that cannot change state of the list/tree -- “static”  Static competitive ratio

7. Shuchi Chawla, Carnegie Mellon University Optimality  Cost of ALG is at most a constant factor times the cost of OPT  Dynamic optimality  Static optimality  Strong static optimality – the ratio is (1+  )  Search optimality – ignore our rotation cost

8. Shuchi Chawla, Carnegie Mellon University Known results..  List update Dynamic ratio [Albers et al’95, Teia’93] : u.b.- 1.6; l.b.- 1.5  Trees Splay trees [Sleator Tarjan 85] : static ratio ~ 3

9. Shuchi Chawla, Carnegie Mellon University Known results..  List update Dynamic ratio [Albers et al’95, Teia’93] : u.b.- 1.6; l.b.- 1.5  Trees Splay trees [Sleator Tarjan 85] : static ratio ~ 3  Open questions that we address  Strong static opt + dynamic opt for lists  Dynamic search opt for trees - ignoring computation time & rotation costs

10. Shuchi Chawla, Carnegie Mellon University Back to list update  How can we hope to achieve strong static optimality?  Soln: Use a classic machine learning result!! Experts Algorithm

11. Shuchi Chawla, Carnegie Mellon University Experts Algorithm [Littlestone’94]  N “expert” algorithms  We want to be (1+  ) wrt the best algorithm  Weighted Majority algorithm  Assign weights to each expert by how well it performs  Pick probabilistically according to weights  Applying this to list update  Each list configuration is an expert  Too many experts – n!  Can we reduce computation?

12. Shuchi Chawla, Carnegie Mellon University Experts for two element list  List – (x,y)  Experts – (x,y) and (y,x)  weights – w x, w y  Algorithm: 1.Initialize w x, w y to r x & r y  R [1..1/  ] 2.If x accessed, w x <- w x +1 else w y <- w y +1 3.Always keep the element with higher weight in front

13. Shuchi Chawla, Carnegie Mellon University List Factoring Lemma  Under a certain condition If A performs well on a list of two elements it performs well on any arbitrary list  Condition: For an arbitrary list, given the same accesses, A should order x and y just as in a list with only x & y

14. Shuchi Chawla, Carnegie Mellon University List Factoring Lemma  e.g. Move-to-front 4729 9472 9 2497 2974 4 7 94 49 49 94 9 4 7 9 and 4 retain the same order – LFL applies

15. Shuchi Chawla, Carnegie Mellon University Extending experts to a general list  Select r i  R [1..1/  ] for element i  Initialize w i <- r i  If i th element accessed, w i <- w i +1  Order elements in decreasing order of weight  (1+  ) static competitive

16. Shuchi Chawla, Carnegie Mellon University Combining Static & Dynamic optimality  A has strong static optimality, B has dynamic optimality  Combine the two to get the best of both  Apply Experts again  Technical difficulties  Cannot estimate weights – running both simultaneously defeats our purpose  Huge cost of switching between experts Don’t switch very often

17. Shuchi Chawla, Carnegie Mellon University How to estimate weights?  The Bandits approach  Run the (so far) better expert  Assume good behavior from the other - Pessimistic approach  After a few runs, we have sampled each one sufficiently

18. Shuchi Chawla, Carnegie Mellon University Binary Search Trees Unfortunately similar “short-cuts” do not work

19. Shuchi Chawla, Carnegie Mellon University Why is BST harder?  Decide which nodes should be near the root  Decide how to move up those nodes Not straightforward 132 different ways of bringing a node at depth 7 to the root!!  We ignore the second issue try for Dynamic “search” optimality

20. Shuchi Chawla, Carnegie Mellon University Outline of our approach  Design a probability distribution p over accesses: Low offline cost => greater probability  Assume p reflects reality and predict the next access from it.  Construct tree based on conditional probability of next access  Low offline cost => node closer to root => low online cost

21. Shuchi Chawla, Carnegie Mellon University An observation about offline BSTs  An access sequence with offline cost k can be expressed in 12k bits  At most 2 12k sequences of offline cost k.

22. Shuchi Chawla, Carnegie Mellon University An observation about offline BSTs  An access sequence with offline cost k can be expressed in 12k bits  Start with a fixed tree  Express rotations from one tree to another using 6 bits per rotation  Assume algorithm first brings accessed element to root: extra factor of 2  At most 2 12k sequences of offline cost k.

23. Shuchi Chawla, Carnegie Mellon University An observation about offline BSTs  An access sequence with offline cost k can be expressed in 12k bits  At most 2 12k sequences of offline cost k.

24. Shuchi Chawla, Carnegie Mellon University 7 412 914 15 52 7 4 12 9 14 1552  Identify rotated subtree (left,up,right,up)(Right,left,up,up)  Specify its initial & final position  Uniquely specifies rotations

25. Shuchi Chawla, Carnegie Mellon University An observation about offline BSTs  An access sequence with offline cost k can be expressed in 12k bits  Start with a fixed tree  Express rotations from one tree to another using 6 bits per rotation  Assume algorithm first brings accessed element to root: extra factor of 2  At most 2 12k sequences of offline cost k.

26. Shuchi Chawla, Carnegie Mellon University Probability distribution on accesses   Distribution on accesses a : p(a) = 2 -13k where k = offline cost of a  Use this to calculate probability of next access  Run through all offline algs, obtain cost, take a weighted average  Much like experts in flavor

27. Shuchi Chawla, Carnegie Mellon University Probability distribution on accesses   Distribution on accesses a : p(a) = 2 -13k where k = offline cost of a  Use this to calculate probability of next access  Caveat: computationally infeasible  But – dynamic search optimal !

28. Shuchi Chawla, Carnegie Mellon University What next?  Can we make this algorithm computationally feasible?  True dynamic optimality, strong static optimality for BST  Lessons to take home Experts analysis is a useful tool for data structures Generic algorithm too slow

29. Shuchi Chawla, Carnegie Mellon University Outline of our approach  Design a probability distribution p over accesses: Low offline cost => greater probability  Assume p reflects reality and predict the next access from it.  Construct tree based on conditional probability of next access  Low offline cost => node closer to root => low online cost