1. Introduction to Algorithms 6.046J/18.401J L ECTURE 14 Competitive Analysis  Self-organizing lists  Move-to-front heuristic  Competitive analysis of MTF Prof. Charles E. Leiserson November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.1

2. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.2 Self-organizing lists List L of n elements The operation A CCESS (x) costs rank L (x)= distance of x from the head of L. L can be reordered by transposing adjacent elements at a cost of 1.

3. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.3 Self-organizing lists List L of n elements The operation A CCESS (x) costs rank L (x)= distance of x from the head of L. L can be reordered by transposing adjacent elements at a cost of 1. Example: L 12 3 50 14 174

4. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.4 Self-organizing lists List L of n elements The operation A CCESS (x) costs rank L (x)= distance of x from the head of L. L can be reordered by transposing adjacent elements at a cost of 1. Example: L 12 3 50 14 17 4 Accessing the element with key 14 costs 4.

5. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.5 Self-organizing lists List L of n elements The operation A CCESS (x) costs rank L (x)= distance of x from the head of L. L can be reordered by transposing adjacent elements at a cost of 1. Example: L 12 50 3 14 17 4 Transposing 3 and 50 costs 1.

6. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.6 On-line and off-line problems Definition. A sequence S of operations is provided one at a time. For each operation, an on-line algorithm A must execute the operation immediately without any knowledge of future operations (e.g., Tetris). An off-line algorithm may see the whole sequence S in advance. Goal: Minimize the total cost C A (S). The game of Tetris

7. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.7 Worst-case analysis of self- organizing lists An adversary always accesses the tail (nth) element of L. Then, for any on-line algorithm A, we have C A (S) =  (|S|  n) in the worst case.

8. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.8 Average-case analysis of self- organizing lists Suppose that element x is accessed with probability p(x). Then, we have which is minimized when L is sorted in decreasing order with respect to p. Heuristic: Keep a count of the number of times each element is accessed, and maintain L in order of decreasing count. E [C A (S)]  ∑ p( x)  rank L (x), xLxL

9. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.9 The move-to-front heuristic Practice: Implementers discovered that the move-to-front (MTF) heuristic empirically yields good results. I DEA : After accessing x, move x to the head of L using transposes: cost = 2  rank L (x). The MTF heuristic responds well to locality in the access sequence S.

10. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.10 Competitive analysis Definition. An on-line algorithm A is  -competitive if there exists a constant k such that for any sequence S of operations, C A (S)  C OPT (S) + k, where OPT is the optimal off-line algorithm (“God’s algorithm”).

11. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.11 MTF is O(1)-competitive Theorem. MTF is 4-competitive for self- organizing lists.

12. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.12 MTF is O(1)-competitive Theorem. MTF is 4-competitive for self- organizing lists. Proof. Let L i be MTF’s list after the ith access, and let L i * be OPT’s list after the ith access. Let c i = MTF’s cost for the ith operation = 2  rank L i–1 (x) if it accesses x; c i * = MTF’s cost for the ith operation = rank Li–1* (x) + t i, where t i is the number of transposes that OPT performs.

13. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.13 Potential function Define the potential function  :{L i } →  R by  (L i ) = 2  |{(x, y) : x  Li y and y  Li * x }| = 2 ⋅ # inversions.

14. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.14 Potential function Define the potential function  :{L i } →  R by  (L i ) = 2  |{(x, y) : x  Li y and y  Li * x }| = 2 ⋅ # inversions. Example. L i L i * E C A D B C A B D E

15. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.15 Potential function Define the potential function  :{L i } →  R by  (L i ) = 2  |{(x, y) : x  Li y and y  Li * x }| = 2 ⋅ # inversions.  (L i )= 2  |{…}| E CA D B C AB D E Example. L i L i *

16. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.16 Potential function Define the potential function  :{L i } →  R by  (L i ) = 2  |{(x, y) : x  Li y and y  Li * x }| = 2 ⋅ # inversions. Example. L i L i * E C A D B C A B D E  (L i )= 2  |{(E,C), …}|

17. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.17 Potential function Define the potential function  :{L i } →  R by  (L i ) = 2  |{(x, y) : x  Li y and y  Li * x }| = 2 ⋅ # inversions. Example. L i L i * E C A D B C A B D E  (L i )= 2 ⋅ |{(E,C), (E,A), … }|

18. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.18 Potential function Define the potential function  :{L i } →  R by  (L i ) = 2  |{(x, y) : x  Li y and y  Li * x }| = 2 ⋅ # inversions. Example. L i L i *  (L i )= 2 ⋅ |{(E,C), (E,A), (E,D), … }| E CA D B C AB D E

19. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.19 Potential function Define the potential function  :{L i } →  R by  (L i ) = 2  |{(x, y) : x  Li y and y  Li * x }| = 2 ⋅ # inversions. Example. L i L i *  (L i )= 2 ⋅ |{(E,C), (E,A), (E,D), (E,B), … }| E C AD B C A BD E

20. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.20 Potential function Define the potential function  :{L i } →  R by  (L i ) = 2  |{(x, y) : x  Li y and y  Li * x }| = 2 ⋅ # inversions. Example. L i L i *  (L i )= 2 ⋅ |{(E,C), (E,A), (E,D), (E,B), (D,B)}| E C A D B C A B D E

21. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.21 Potential function Define the potential function  :{L i } →  R by  (L i ) = 2  |{(x, y) : x  Li y and y  Li * x }| = 2 ⋅ # inversions. Example. L i L i *  (L i ) = 2 ⋅ |{(E,C), (E,A), (E,D), (E,B), (D,B)}| = 10. E CA D B C AB D E

22. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.22 Potential function Define the potential function  :{L i } →  R by  (L i ) = 2  |{(x, y) : x  Li y and y  Li * x }| = 2 ⋅ # inversions.

23. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.23 Potential function Define the potential function  :{L i } →  R by  (L i ) = 2  |{(x, y) : x  Li y and y  Li * x }| = 2 ⋅ # inversions. Note that  (L i )  0 for i = 0, 1, …,  (L 0 )  0 if MTF and OPT start with the same list.

24. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.24 Potential function Define the potential function  :{L i } →  R by  (L i ) = 2  |{(x, y) : x  Li y and y  Li * x }| = 2 ⋅ # inversions. Note that  (L i )  0 for i = 0, 1, …,  (L 0 )  0 if MTF and OPT start with the same list. How much does  change from 1 transpose? A transpose creates/destroys 1 inversion. ∆  = ±2.

25. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.25 What happens on an access? Suppose that operation i accesses element x, and define A ={y ∈  L i–1 : y  L B ={y ∈  L i–1 : y  L C ={y ∈  L i–1 : y  L D ={y ∈  L i–1 : y  L i–1 x and y  L x and y  L x and y  L x and y  L i–1* x}, i–1* x}. A  B x C  D A  C x B  D L i–1 L i–1 *

26. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.26 What happens on an access? L i–1 L i–1 * A  B x C  D r = rank Li–1 (x) A  C x B  D r* = rank Li–1* (x) We have r = |A| + |B| + 1 and r*  |A| + |C| + 1.

27. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.27 What happens on an access? L i–1 L i–1 * A  B x C  D r = rank Li–1 (x) A  C x B  D r* = rank Li–1* (x) We have r = |A| + |B| + 1 and r*  |A| + |C| + 1. When MTF moves x to the front, it creates |A| inversions and destroys |B| inversions. Each transpose by OPT creates ≤ 1 inversion. Thus, we have  (L i ) –  (L i–1 ) ≤ 2(|A| – |B| + t i ).

28. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.28 Amortized cost The amortized cost for the ith operation of MTF with respect to  is ĉ i = c i +  (L i ) –  (L i–1 )

29. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.29 Amortized cost The amortized cost for the ith operation of MTF with respect to  is ĉ i = c i +  (L i ) –  (L i–1 )  2r + 2(|A| – |B| + t i )

30. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.30 Amortized cost The amortized cost for the ith operation of MTF with respect to  is ĉ i = c i +  (L i ) –  (L i–1 )  2r + 2(|A| – |B| + t i ) = 2r + 2(|A| – (r – 1 – |A|) + t i ) (since r = |A| + |B| + 1)

31. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.31 Amortized cost The amortized cost for the ith operation of MTF with respect to  is ĉ i = c i +  (L i ) –  (L i–1 )  2r + 2(|A| – |B| + t i ) = 2r + 2(|A| – (r – 1 – |A|) + t i ) = 2r + 4|A| – 2r + 2 + 2t i

32. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.32 Amortized cost The amortized cost for the ith operation of MTF with respect to  is ĉ i = c i +  (L i ) –  (L i–1 )  2r + 2(|A| – |B| + t i ) = 2r + 2(|A| – (r – 1 – |A|) + t i ) = 2r + 4|A| – 2r + 2 + 2t i = 4|A| + 2 + 2t i

33. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.33 Amortized cost The amortized cost for the ith operation of MTF with respect to  is ĉ i = c i +  (L i ) –  (L i–1 )  2r + 2(|A| – |B| + t i ) = 2r + 2(|A| – (r – 1 – |A|) + t i ) = 2r + 4|A| – 2r + 2 + 2t i = 4|A| + 2 + 2t i  4(r* + t i ) (since r* = |A| + |C| + 1  |A| + 1)

34. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.34 Amortized cost The amortized cost for the ith operation of MTF with respect to  is ĉ i = c i +  (L i ) –  (L i–1 )  2r + 2(|A| – |B| + t i ) = 2r + 2(|A| – (r – 1 – |A|) + t i ) = 2r + 4|A| – 2r + 2 + 2t i = 4|A| + 2 + 2t i  4(r* + t i )  4c i *.

35. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.35 The grand finale Thus, we have C MTF (S ) = ∑ |S| i=1i=1 cici

36. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.36 The grand finale Thus, we have C MTF (S ) = ∑ |S| i=1i=1 cici = ∑ |S| i=1i=1 ( ĉi +  (L i-1 ) –  (L i- ))

37. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.37 The grand finale Thus, we have C MTF (S ) = ∑ |S| i=1i=1 cici = ∑ |S| i=1i=1 ( ĉi +  (L i-1 ) –  (L i- )) ≤ 〔 ∑ 4 c i * 〕 +  (L 0 ) –  (L |S| ) |S| i=1i=1

38. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.38 The grand finale Thus, we have C MTF (S ) = ∑ |S| i=1i=1 cici = ∑ |S| i=1i=1 ( ĉi +  (L i-1 ) –  (L i- )) ≤ 〔 ∑ 4 c i * 〕 +  (L 0 ) –  (L |S| ) |S| i=1i=1 ≤ 4 ⋅ C OPT (S ), since  (L 0 ) = 0 and  (L |S| ) ≥ 0.

39. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.39 Addendum If we count transpositions that move x toward the front as “free” (models splicing x in and out of L in constant time), then MTF is 2-competitive.

40. November 2, 2005Copyright © 2001-5 by Erik D. Demaine and Charles E. LeisersonL14.40 Addendum If we count transpositions that move x toward the front as “free” (models splicing x in and out of L in constant time), then MTF is 2-competitive. What if L 0  L 0 *? Then,  (L 0 ) might be  (n 2 ) in the worst case. Thus, C MTF (S)  4  C OPT (S) +  (n 2 ), which is still 4-competitive, since n 2 is constant as |S| →  ∞.