ECE7995 Caching and Prefetching Techniques in Computer Systems Lecture 8: Buffer Cache in Main Memory (IV)

Quantifying Locality with LRU Stack Blocks are ordered by their recencies; Blocks enter from the stack top, and leave from its bottom; 1 LRU stack Recency = 1 Recency = 2

LRU Stack Blocks are ordered by recency in the LRU stack; Blocks enter from the stack top, and leave from its bottom; LRU stack Recency = 2 IRR = 2 Inter-Reference Recency (IRR) The number of other distinct blocks accessed between two consecutive references to the block. Recency = 0

Locality Strength Cache Size MULTI2 IRR (Re-use Distance in Blocks) Virtual Time (Reference Stream) LRU Good for “absolutely” strong locality Bad for relatively weak locality

LRU ’ s Inability with Weak Locality Memory scanning (one-time access)  Infinite IRR, weak locality;  should not be cached at all;  not replaced timely in LRU (be cached until their recency larger than cache size);

LRU ’ s Inability with Weak Locality Loop-like accesses (repeated accesses with a fixed interval)  IRR is the same as the interval  The interval larger than cache size, no hits  blocks to be accessed soonest can be unfortunately replaced.

LRU ’ s Inability with Weak Locality Accesses with distinct frequencies:  The recencies of frequently accessed blocks become large because of references to infrequently accessed block;  Frequently accessed blocks could be unfortunately replaced.

Looking for Blocks with Strong Locality Locality Strength Cache Size MULTI2 IRR (Re-use Distance in Blocks) Virtual Time (Reference Stream) Cover 1000 Blocks with Strongest Locality

Challenges  Address the limitations of LRU fundamentally.  Retain the low overhead and adaptability merits of LRU. Simplicity: affordable implementation Adaptability: responsive to access pattern changes

Principle of the LIRS Replacement We select the blocks with high IRRs for replacement. LIRS: Low IRR Set Replacement algorithm We keep the set of blocks with low IRRs in cache. If a block’s IRR is high, its next IRR is likely to be high again.

Requirements on Low IRR Block Set (LIRS)  The set size should be the cache size.  The set consists of the blocks with strongest locality strength (with the lowest IRRs)  Dynamically keep the set up to date

Low IRR Block Set Low IRR ( LIR ) block and High IRR ( HIR ) block LIR block set (size is L lirs ) HIR block set Cache size L = L lirs + L hirs L hirs L lirs Physical Cache Block Sets

An Example for LIRS L lirs =2, L hirs =1 LIR block set = {A, B}, HIR block set = {C, D, E}

CDECDE HIR block set ABAB ABAB E LIR block set Resident blocks Mapping to Cache Block Sets L hirs =1 L lirs =2 Physical Cache

D is referenced at time 10 The resident HIR block (E) is replaced ! Which Block is replaced ? Replace HIR Blocks

How LIR Set is Updated ? Recency of LIR Block Used

After D is Referenced at Time 10 … … E is replaced, D enters LIR set B D

If Reference is to C at Time 10 … … E is replaced, C cannot enter LIR set

The LIRS References with Weak Locality Memory scanning (one-time access)  Infinite IRR;  Not included in the LIR block set;  replaced timely.

The LIRS References with Weak Locality Loop-like accesses  The IRRs of all blocks are the same;  Once a block becomes LIR block, it can keep its status;  Any cached block can contribute a hit in one loop of accesses.

The LIRS References with Weak Locality Accesses with distinct frequencies:  The IRRs of frequently accessed blocks have smaller IRR, than infrequently accessed blocks.  Frequently accessed blocks are LIR blocks;  Always cached and get hits.

Making LIRS O(1) Efficient Rmax (Maximum Recency of LIR blocks) IRR HIR (New IRR of the HIR block) This efficiency is achieved by our LIRS stack. LRU stack + LIR block with Rmax recency in its bottom ==> LIRS stack.

Differences between LRU and LIRS Stacks resident block LIR block HIR block Cache size L = LRU stack LIRS stack L lir = 3 L hir =2  Stack size of LRU decided by cache size, and fixed; Stack size of LIRS decided by Rmax, and varied.  LRU stack holds only resident blocks; LIRS stack holds any blocks whose recencies are no more than Rmax.  LRU stack does not distinguish “ hot ” and “ cold ” blocks in it; LIRS stack distinguishes LIR and HIR blocks in it, and dynamically maintains their statues.

Rmax (Maximum Recency of LIR blocks) IRR HIR (New IRR of the HIR block) Blocks in the LIRS stack ==> IRR < Rmax Other blocks ==> IRR > Rmax LIRS Stack How does LIRS Stack Help?

LIRS Operations resident in cache LIR block HIR block Cache size L = 5 L lir = 3 L hir = LIRS stack S 5 3 Resident HIR Stack Q Initialization: All the referenced blocks are given an LIR status until LIR block set is full. We place resident HIR blocks in Stack Q

resident in cache LIR block HIR block Cache size L = 5 L lir = 3 L hir = Access an LIR Block (a Hit) LIRS stack S Resident HIR Stack Q

resident in cache LIR block HIR block Cache size L = 5 L lir = 3 L hir = Access an LIR Block (a Hit) LIRS stack S Resident HIR Stack Q

Access an LIR block (a Hit) resident in cache LIR block HIR block Cache size L = 5 L lir = 3 L hir = SQ

Access a Resident HIR Block (a Hit) resident in cache LIR block HIR block Cache size L = 5 L lir = 3 L hir = SQ

resident in cache LIR block HIR block Cache size L = 5 L lir = 3 L hir = Access a Resident HIR Block (a Hit) SQ

resident in cache LIR block HIR block Cache size L = 5 L lir = 3 L hir = Access a Resident HIR Block (a Hit) SQ

resident in cache LIR block HIR block Cache size L = 5 L lir = 3 L hir = Access a Resident HIR Block (a Hit) SQ

Access a Non-Resident HIR block (a Miss) resident in cache LIR block HIR block Cache size L = 5 L lir = 3 L hir = SQ

resident in cache LIR block HIR block Cache size L = 5 L lir = 3 L hir = Access a Non-Resident HIR block (a Miss) SQ

4 83 resident in cache LIR block HIR block Cache size L = 5 L lir = 3 L hir = Access a Non-Resident HIR block (a Miss) SQ

Workload Traces postgres is a trace of join queries among four relations in a relational database system; sprite is from the Sprite network file system; multi2 is obtained by executing three workloads, cs, cpp, and postgres, together.

Cache Partition 1% of the cache size is for HIR blocks 99% of the cache size is for LIR blocks Performance is not sensitive to a partition.

Looping Pattern: postgres (Access Map) Virtual Time (Reference Stream) Logical Block Number

Looping Pattern: Postgres (IRR Map) IRR (Re-use Distance in Blocks) Virtual Time (Reference Stream) LRU LIRS

Looping Pattern: postgres (Hit Rates)

Temporally-Clustered Pattern: sprite (Access Map) Virtual Time (Reference Stream) Logical Block Number

Temporally-Clustered Pattern: sprite (IRR Map) IRR (Re-use Distance in Blocks) Virtual Time (Reference Stream) LRU LIRS

Temporally-Clustered Pattern: sprite (Hit Ratio)

Mixed Pattern: multi2 (Access Map) Virtual Time (Reference Stream) Logical Block Number

Mixed Pattern: multi2 (IRR Map) IRR (Re-use Distance in Blocks) Virtual Time (Reference Stream) LIRS LRU

Mixed Pattern: multi2 (Hit Ratio)

Summay LIRS uses both IRR (or reuse distance) and recency for its replacement decision. 2Q uses only reuse distance. LIRS adapts to the locality changes when deciding which blocks have small IRRs. 2Q uses a fixed threshold in looking for blocks of small reuse distances. Both LIRS and 2Q are of low time overhead (as low as LRU). Their space overheads are acceptably larger.