Cache Memory By JIA HUANG

"Computer Science has only three ideas: cache, hash, trash.“ - Greg Ganger, CMU

The idea of using caching Using very short time to access the recently and frequently data Using very short time to access the recently and frequently data cache: fast but expensivedisks: cheap but slow

Type of cache CPU cache CPU cache Disk cache Disk cache Other caches Other caches Proxy web cache Proxy web cache

Usage of caching Caching is used widely in: Caching is used widely in: Storage systems Storage systems Databases Databases Web servers Web servers Middleware Middleware Processors Processors Operating systems Operating systems RAID controller RAID controller Many other applications Many other applications

Cache Algorithms Famous algorithms Famous algorithms LRU (Least Recently Used) LRU (Least Recently Used) LFU (Least Frequently Used) LFU (Least Frequently Used) Not so Famous algorithms Not so Famous algorithms LRU –K LRU –K 2Q 2Q FIFO FIFO others others

LRU (Least Recently Used) LRU is implemented by a linked list. LRU is implemented by a linked list. Discards the least recently used items first. Discards the least recently used items first.

LFU (least frequently used) Counts, how often an item is needed. Those that are used least often are discarded first. Counts, how often an item is needed. Those that are used least often are discarded first.

LRU vs. LFU The fundamental locality principle claims that if a process visits a location in the memory, it will probably revisit the location and its neighborhood soon The fundamental locality principle claims that if a process visits a location in the memory, it will probably revisit the location and its neighborhood soon The advanced locality principle claims that the probability of revisiting will increased if the number of the visits is bigger The advanced locality principle claims that the probability of revisiting will increased if the number of the visits is bigger

Disadvantages LRU – problem with LRU – problem with process scans a huge database LFU – process scans a huge database, but it made the performance even worse.

can there be a better algorithm? Yes Yes

New algorithm ARC (Adaptive Replacement Cache) ARC (Adaptive Replacement Cache) - it combines the virtues of LRU and LFU, while avoiding vices of both. The basic idea behind ARC is to adaptively, dynamically and relentlessly balance between "recency" and "frequency" to achieve a high hit ratio. - - Invented by IBM in 2003 ( Almaden Research Center, San Jose)

How it works?

ARC L1: pages were seen once recently("recency") L2: pages were seen at least twice recently ("frequency") If L1 contains exactly c pages --replace the LRU page in L1 else --replace the LRU page in L2. Lemma: The c most recent pages are in the union of L1 and L2. L1 L2 LRU LRU MRU MRU

ARC Divide L1 into T1 (top) & B1 (bottom) Divide L2 into T2 (top) & B2 (bottom) T1 and T2 contain c pages in cache and in directory B1 and B2 contain c pages in directory, but not in cache If T1 contains more than p pages, --replace LRU page in T1, else --replace LRU page in T2. L2 L1 MRU MRU LRU LRU

Adapt target size of T1 to an observed workload A self-tuning algorithm: hit in T1 or T2 : do nothing hit in B1: increase target of T1 hit in B2: decrease target of T1 L2 L2"frequency" L1 L1 "recency" MidpointARC

ARC ARC has low space complexity. A realistic implementation had a total space overhead of less than 0.75%. ARC has low time complexity; virtually identical to LRU. ARC is self-tuning and adapts to different workloads and cache sizes. In particular, it gives very little cache space to sequential workloads, thus avoiding a key limitation of LRU. ARC outperforms LRU for a wide range of workloads.

Example For a huge, real-life workload generated by a large commercial search engine with a 4GB cache, ARC's hit ratio was dramatically better than that of LRU (40.44 percent vs percent). -IBM (Almaden Research Center)

ARC vs. LRU

ARC Currently, ARC is a research prototype and will be available to customers via many of IBM's existing and future products. Currently, ARC is a research prototype and will be available to customers via many of IBM's existing and future products.

References ing#Other_caches ing#Other_caches ing#Other_caches ing#Other_caches /2os/2os/os2.pdf /2os/2os/os2.pdf /2os/2os/os2.pdf /2os/2os/os2.pdf orageSystems/autonomic_stora ge/ARC/index.shtml orageSystems/autonomic_stora ge/ARC/index.shtml orageSystems/autonomic_stora ge/ARC/index.shtml orageSystems/autonomic_stora ge/ARC/index.shtml people/dmodha/arc-fast.pdf people/dmodha/arc-fast.pdf people/dmodha/arc-fast.pdf people/dmodha/arc-fast.pdf