1. Exploiting Gray-Box Knowledge of Buffer Cache Management Nathan C. Burnett, John Bent, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau University of Wisconsin - Madison Department of Computer Sciences

2. 2 Knowledge is Power OS usually treated as a black box –Implies no knowledge of OS internals OS can be viewed as a gray-box –We have basic knowledge of the OS File caching Process scheduling –Gray-box knowledge can be leveraged To discover more detailed knowledge To implement OS-like services from user-space

3. 3 Knowledge is Power Cache policy impacts I/O performance Programmers can use knowledge of cache state to improve overall performance Cache policy is usually hidden –Documentation can be old, vague or incorrect –Source code may not be available –Often no interface for learning cache state

4. 4 Fingerprinting Use microbenchmarking techniques to discover algorithms and policies Dust - Fingerprints buffer cache policies –Correctly identifies many different policies –Requires no kernel modification –Portable across platforms

5. 5 This Talk Dust Fingerprints of Real Systems Exploiting Gray-Box Knowledge –Cache-Aware Web Server Conclusions & Future Work

6. 6 Dust Fingerprints the buffer cache –Determines cache size –Determines cache policy –Determines cache history usage Uses only standard system calls –read/open/close/seek

7. 7 Approach Cache policies often use access order, recency, frequency Need to find an access pattern that will differentiate FIFO, LRU, LFU Explore in simulation –Keeps track of cache contents –Doesn’t actually store the data –Two possible access times (in cache or not)

8. 8 Approach Move the cache to a known state –Sets recency –Sets access order –Sets frequency Cause part of test data to be evicted Sample data to determine cache state

9. 9 Detecting FIFO Replacement Test Region slightly smaller than the cache One sequential scan sets access order Eviction Scan evicts half of the Test Region Test Region Eviction Region

10. 10 Detecting FIFO Replacement FIFO will give the later part of the file priority in the cache Newer Pages Older Pages

11. 11 Detecting FIFO Replacement FIFO evicts the first half of the test region Out of Cache In Cache

12. 12 Detecting LRU Replacement Need to differentiate from FIFO Divide Test Region into ten stripes Reads alternate between left and right pointers Left Pointer Right Pointer Test Region Eviction Region

13. 13 Detecting LRU Replacement LRU will give priority to the 2 nd and 4 th quarters of the test region

14. 14 Detecting LRU Replacement LRU evicts the first and third quarter of the test region

15. 15 Detecting LFU Replacement Read stripes near the middle more Read stripes near the ends less Read eviction region more than any stripe 23456654327 Left Pointer Right Pointer Test Region Eviction Region

16. 16 Detecting LFU Replacement LFU gives priority to the center of the test region

17. 17 Detecting LFU Replacement LFU evicts the outermost stripes. Two stripes partially evicted.

18. 18 Detecting Clock Replacement Clock often used in place of LRU Two pieces of initial state –Hand Position –Reference Bits Hand position is irrelevant – circular queue Dust must control for reference bits –Reference bits affect the order of replacement

19. 19 Detecting Clock Replacement Uniform reference bitsRandom reference bits

20. 20 Clock - Random Reference Bits Initial Sequential Scan Read pattern does not change cache state First part of left and right halves are evicted

21. 21 Clock - Uniform Reference Bits First part of test region is evicted Degenerates to FIFO

22. 22 Two fingerprints for Clock Ability to produce both will imply Clock Need a way to selectively set reference bits Clock - Controlling Reference Bits

23. 23 Clock - Controlling Reference Bits 1.To set all reference bits: a.Read in enough data to almost fill the cache b.Touch every page 2.To clear all reference bits a.Read small amount of new data to cause replacement b.Clock hand sweeps entire cache, resetting all refbits 3.To randomly set half of the bits a.Randomly touch half of the pages in the cache

24. 24 Dust Differentiates policies based on access order, recency and frequency Determines cache size Identifies more complex policies –Segmented FIFO, Random Identifies history-based policies –LRU-k, 2-Queue Robust to errors in cache size estimate

25. 25 This Talk Dust Fingerprints of Real Systems Exploiting Gray-Box Knowledge –A Cache-Aware Web Server Conclusions & Future Work

26. 26 Fingerprinting Real Systems Data is noisy Policies usually more complex Buffer Cache/VM Integration –Cache size might be changing Timing Precision

27. 27 NetBSD 1.5 Clock Fingerprint is identical to this Conclusion: LRU

28. 28 Linux 2.2.19 Very noise but looks like LRU

29. 29 Linux 2.2.19 With uniform reference bits, looks like FIFO Conclusion: Clock

30. 30 Linux 2.4.14 Shows the two “humps” of LRU Shows end “hump” of LFU Is LRU with page aging

31. 31 This Talk Dust Fingerprints of Real Systems Exploiting Gray-Box Knowledge –A Cache-Aware Web Server Conclusions & Future Work

32. 32 Algorithmic Mirroring Observe inputs presented to the kernel We have knowledge of kernel algorithms Model internal state of the kernel –Only model relevant state, not everything –Can be an approximation Use model to make application-level decisions

33. 33 A Cache-Aware Application NeST - Network Storage Technology Software based storage appliance Supports HTTP, NFS, FTP, GridFTP, Chirp Allows configurable number of requests to be serviced concurrently Any requests beyond that number are queued

34. 34 Cache-Aware NeST Takes policy & size discovered by Dust Maintains algorithmic mirror of the cache –Updates mirror on each request –No double buffering –May not be a perfect mirror Approximate shortest-job-first scheduling by scheduling in-cache-first –Reduce latency by approximating SJF –Improve throughput by reducing disk reads

35. 35 Improved Response Time Dramatic improvement in response time Robust to inaccuracies in cache estimate

36. 36 Conclusions Fingerprinting –Discovers OS algorithms and policies Dust –Discovers buffer cache policies Algorithmic Mirroring –Keep track of kernel state in user-space –Use this information to improve performance Cache-Aware NeST –Uses mirroring to improved HTTP performance

37. 37 Future Work On-line, adaptive detection of cache policy Policy manipulation Make other applications cache aware –Databases –File servers (ftp, NFS, etc.) Fingerprint other OS components