1. Seoul National University Archi & Network LAB LRFU (Least Recently/Frequently Used) Block Replacement Policy Sang Lyul Min Dept. of Computer Engineering Seoul National University

2. Archi & Network LAB Why file cache? Processor - Disk Speed Gap 1950s1990s Processor - IBM 701 17,000 ins/sec Disk - IBM 305 RAMAC Density - 0.002 Mbits/sq. in Average seek time - 500 ms Processor - IBM PowerPC 603e 350,000,000 ins/sec Disk - IBM Deskstar 5 Density - 1,319 Mbits/sq. in Average seek time - 10 ms x 600,000 x 50 x 20,000

3. Seoul National University Archi & Network LAB File Cache main memorydisk controller file cache or buffer cachedisk cache processor disks

4. Seoul National University Archi & Network LAB Operating System 101 LRU ReplacementLFU Replacement LRU Block MRU Block LFU Block MFU Block New reference O(1) complexityO(n) complexity heap O(log n) complexity

5. Seoul National University Archi & Network LAB Operating System 101 LRU –Advantage High Adaptability –Disadvantage Short sighted LFU –Advantage Long sighted –Disadvantage Cache pollution

6. Seoul National University Archi & Network LAB Motivation Cache size = 20 blocks

7. Seoul National University Archi & Network LAB Cache size = 60 blocks

8. Seoul National University Archi & Network LAB Cache size = 100 blocks

9. Seoul National University Archi & Network LAB Cache size = 200 blocks

10. Seoul National University Archi & Network LAB Cache size = 300 blocks

11. Seoul National University Archi & Network LAB Cache size = 500 blocks

12. Seoul National University Archi & Network LAB Observation Both recency and frequency affect the likelihood of future references The relative impact of each is largely determined by cache size

13. Seoul National University Archi & Network LAB Goal A replacement algorithm that allows a flexible trade-off between recency and frequency

14. Seoul National University Archi & Network LAB Results LRFU (Least Recently/Frequently Used) Replacement Algorithm that (1) subsumes both the LRU and LFU algorithms (2) subsumes their implementations (3) yields better performance than them

15. Seoul National University Archi & Network LAB CRF (Combined Recency and Frequency) Value Current time tc time t3t3 t2t2 t1t1 C t c (b) = F( 1 ) + F( 2 ) + F( 3 ) || || || t c - t 1 t c - t 2 t c - t 3 1 2 3

16. Seoul National University Archi & Network LAB CRF (Combined Recency and Frequency) Value Estimate of how likely a block will be referenced in the future Every reference to a block contributes to the CRF value of the block A references contribution is determined by weighing function F(x)

17. Seoul National University Archi & Network LAB Hints and Constraints on F(x) should be monotonically decreasing should subsume LRU and LFU should allow efficient implementation

18. Seoul National University Archi & Network LAB Conditions for LRU and LFU LRU Condition –If F(x) satisfies the following condition, then the LRFU algorithm becomes the LRU algorithm LFU Condition –If F(x) = c, then the LRFU algorithm becomes the LFU algorithm block a: x block b: x x x x x x x x i i+1 i+2 i+3 current time

19. Seoul National University Archi & Network LAB Weighing function F(x) F(x) = ( ) x Meaning: a references contribution to the target blocks CRF value is halved after every 1/

20. Seoul National University Archi & Network LAB Properties of F(x) = ( ) x Property 1 –When = 0, (i.e., F(x) = 1), then it becomes LFU –When = 1, (i.e., F(x) = ( ) x ), then it becomes LRU –When 0 < < 1, it is between LFU and LRU F(x) = ( ) x (LRU extreme) Spectrum (LRU/LFU) 1 0 F(x) = 1 (LFU extreme) F(x) X current time - reference time

21. Seoul National University Archi & Network LAB Results LRFU (Least Recently/Frequently Used) Replacement Algorithm that (1) subsumes both the LRU and LFU algorithms (2) subsumes their implementations (3) yields better performance than them

22. Seoul National University Archi & Network LAB Difficulties of Naive Implementation Enormous space overheads –Information about the time of every reference to each block Enormous time overheads –Computation of the CRF value of every block at each time

23. Seoul National University Archi & Network LAB Update of CRF value over time t1t1 time t2t2 = (t 2 - t 1 ) 1 2 3 C t 2 (b) = F ( 1 + ) + F ( 2 + ) + F ( 3 + ) = ( ) ( 1 + ) + ( ) ( 2 + ) + ( ) ( 3 + ) = ( ( ) 1 + ( ) 2 + ( ) 3 ) ( ) = C t 1 (b) x F ( )

24. Seoul National University Archi & Network LAB Properties of F(x) = ( ) x Property 2 –With F(x) = ( ) x, C t k (b) can be computed from C t k-1 (b) as follows C t k (b) = C t k-1 (b) F ( ) + F (0) || t k - t k-1 –Implications: Only two variables are required for each block for maintaining the CRF value One for the time of the last reference The other for the CRF value at that time

25. Seoul National University Archi & Network LAB Difficulties of Naive Implementation Enormous space overheads –Information about the time of every reference to each block Enormous time overheads –Computation of the CRF value of every block at each time

26. Seoul National University Archi & Network LAB Properties of F(x) = ( ) x Property 3 –If C t (a) > C t (b) and neither a or b is referenced after t, then C t' (a) > C t' (b) for all t' > t Why? Ct'(a) = Ct(a) F( ) > Ct(b) F( ) = Ct'(b) (since F( ) > 0 ) Implications –Reordering of blocks is needed only upon a block reference –Heap data structure can be used to maintain the ordering of blocks with O(log n) time complexity

27. Seoul National University Archi & Network LAB Optimized Implementation Blocks that can compete with a currently referenced block

28. Seoul National University Archi & Network LAB linked list heap 1. replaced 2. demoted 3. new block 4. heap restored linked list heap 1. demoted 2. promoted referenced block 3. heap restored linked list heap referenced block 1. heap restored Optimized Implementation Reference to a new block Reference to a block in the linked list Reference to a block in the heap

29. Seoul National University Archi & Network LAB Question What is the maximum number of blocks that can potentially compete with a currently referenced block?

30. Seoul National University Archi & Network LAB block a: x block b: x x x x x time + F(d threshold +2) + F(d threshold +1) + F(d threshold ) < F(0) d threshold d threshold +1 d threshold +2 current time

31. Properties of F(x) = ( ) x Property 4 : log (1- ( ) ) d threshold = log (1- ( ) ) When 0 When 1 = = 1 log (1- ( ) ) Seoul National University Archi & Network LAB

32. Optimized implementation (Contd) linked list heap (single element) LRU extreme LFU extreme heap linked list (null) hea p O(log n)O(1) Seoul National University Archi & Network LAB

33. Seoul National University Archi & Network LAB Results LRFU (Least Recently/Frequently Used) Replacement Algorithm that (1) subsumes both the LRU and LFU algorithms (2) subsumes their implementations (3) yields better performance than them

34. Seoul National University Archi & Network LAB Correlated References correlated references

35. LRFU with correlated references Masking function G c (x) C' t k (b), CRF value when correlated references are considered, can be derived from C' t k-1 (b) C' t k (b) = F(t k - t k ) + F(t k - t i )*G c (t i+1 - t i ) = F( t k - t k-1 ) * [F(0) * G c ( t k - t k-1 ) + C' t k-1 (b) - F(0)] + F(0) Seoul National University Archi & Network LAB

36. Trace-driven simulation Sprite client trace –Collection of block references from a Sprite client –contains 203,808 references to 4,822 unique blocks DB2 trace –Collection of block references from a DB2 installation –Contains 500,000 references to 75,514 unique blocks Seoul National University Archi & Network LAB

37. Effects of on the performance (b) DB2 Hit Rate X X X X (a) Sprite client X X X X Seoul National University Archi & Network LAB

38. Combined effects of and correlated period Hit Rate Correlated Period (a) Sprite client Hit Rate Correlated Period (b) DB2 Seoul National University Archi & Network LAB

39. Previous works FBR (Frequency-Based Replacement) algorithm –Introduces correlated reference concept LRU-K algorithm –Replaces blocks based on time of the Kth-to-last non-correlated references –Discriminates well the frequently and infrequently used blocks –Problems Ignores the K-1 references linear space complexity to keep the last K reference times 2Q and sLRU algorithms –Use two queues or two segments –Move only the hot blocks to the main part of the disk cache –Work very well for used-only-once blocks Seoul National University Archi & Network LAB

40. Comparison of the LRFU policy with other policies Hit Rate Cache Size (# of blocks) (a) Sprite client Hit Rate Cache Size (# of blocks) (b) DB2 Seoul National University Archi & Network LAB

41. Implementation of the LRFU algorithm Buffer cache of the FreeBSD 3.0 operating system Benchmark: SPEC SDET benchmark –Simulates a multi-programming environment –consists of concurrent shell scripts each with about 150 UNIX commands –gives results in scripts / hour Seoul National University Archi & Network LAB

42. SDET benchmark results Hit rate SDET Throughput (scripts/ hour) Hit Rate Seoul National University Archi & Network LAB

43. Seoul National University Archi & Network LAB Conclusions LRFU (Least Recently/Frequently Used) Replacement Algorithm that (1) subsumes both the LRU and LFU algorithms (2) subsumes their implementations (3) yields better performance than them

44. Seoul National University Archi & Network LAB Future Research Dynamic version of the LRFU algorithm LRFU algorithm for heterogeneous workloads –File requests vs. VM requests –Disk block requests vs. Parity block requests (RAID) –Requests to different files (index files, data files)

45. Seoul National University Archi & Network LAB People REAL PEOPLE (Graduate students) –Lee, Donghee –Choi, Jongmoo –Kim, Jong-Hun Guides (Professors) –Noh, Sam H. –Min, Sang Lyul –Cho, Yookun –Kim, Chong Sang http://archi.snu.ac.kr/symin/

46. Adaptive LRFU policy Adjust periodically depending on the evolution of workload Use the LRU policy as the reference model to quantify how good (or bad) the locality of the workload has been Algorithm of the Adaptive LRFU policy –if ( > ) value for period i+1 is updated in the same direction –else the direction is reversed Seoul National University Archi & Network LAB

47. Results of the Adaptive LRFU Client Workstation 54 DB2 Seoul National University Archi & Network LAB