1. Adaptive Cache Replacement Policy
By: Neslisah Torosdagli, Awrad Mohammed Ali, Stacy Gramajo
Spring 2015

2. Introduction What is Adaptive Cache Replacement? Motivations
Given more than one cache replacement algorithm, one is chosen to access memory based on recent memory accesses
Motivations
Cache performance is critical for various reasons
Changing the hardware (i.e. increasing size of cache) can only go so far
Various replacement policies work their best in different situation
Combining different replacement policies can help with performance

3. Research Paper Implementation
Author: Yannis Smaragdakis
ACM 2004
Hardware Modifications
First Structure: Two sets of parallel tag arrays
Same size of the regular tag array of adaptive cache
Contains cache contents for each replacement policy
Second Structure: Miss History Buffer
Tracks of past performance of cache misses for one replacement policy
Replacement Algorithm
For every memory reference, the parallel tag arrays and miss history buffer is updated

4. Research Paper Extension
Assume that we are running a program of 2000 instructions which’s:
first 1000 instructions perform pretty good with policy A
last 1000 instructions perform pretty good with policy B
At instruction 1000, miss rates are as follows:
policy A : 250
policy B : 750
Since miss rate of policy A is less than miss rate of policy B, policy A will definitely be prefered for instruction 1001, 1002, .., 1500 although policy B performs better due to accumulated success of policy A
Can we use an algorithm similar to tournament algorithm of branch prediction to solve this unfair selection?

5. Our Implementation - Software
Used SimpleScalar
Implementing Adaptive Cache Policy, LFR, and Branch-like Predictions
Added Features
LRU Tag Array
LFU Tag Array
Adaptive Tag Array
Local Histories
Global History
Global History keeps track of which replacement algorithm was used
Local History records the history of hits and misses of a policy.
Miss Counts

6. Research Paper Extension
Global History Buffer
32 bits
0 - Policy A is used
1 - Policy B is used
Local History Buffer
0 - Policy missed
1 - Policy Hit
Global History Buffer
bits 32 1 1
32 bits 1
Local History Buffer 1 1 1
32 bits 1

7. Issues
In the initial implementation, history buffers are initialized to 0:
Since 0 refers to policy A, voting unfairly selects policy A.
History buffers are filled randomly:
Un-consistent results are obtained directly proportional with the success of randomization function
History ready buffers are added to the implementation:
Voting algorithm uses error counts until history buffers are filled completely,
Once history buffers are ready, voting algorithm uses history buffers and error counts to make a decision

8. Our Implementation - Software
Added three additional arrays
Global History Ready (1)
Local History Ready (2)
The additional features removes the randomization problem from occurring
Miss Counts

9. Adaptive Replacement UML Diagram

10. Policies
The adaptive cache mimic either LFU or user selected cache mechanism by SimpleScalar command line arguments:
LRU: evict the blocks that least recently used
strong in conditions where access is made mainly to the most recent items, such as an application computing average temperature of last 2 hours
LFU: evict the blocks with the lowest referenced frequencies through creating a counter.
strong in conditions where large regions of blocks used only once from commonly accessed data
Fails when there is an item that is accessed frequently in the past, and not accessed anymore. LFU do not evict it
New items added to cache are more probably to be evicted

11. Configurations for Simulations
First Configuration
L1 Data and Instruction Cache
16 KB cache, 64B block, –way associative
Unified L2 Cache
512K cache size, 64B block, 8-way associative
Second Configuration
L1 Data and Instruction Cache
16 KB cache, 64B block, –way associative
Unified L2 Cache
512K cache size, 64B block 4-way associative

12. Results: Comparison
MPKI (Misses Per Thousand Instructions = (# access/ # instructions) * miss rate * 1000

13. More Results

14. Success and Difficulties
The project is implemented in an iterative manner shaped by issues faced
SimpleScalar is slow and some benchmarks are not working correctly
The results are in proportion with the paper

15. Conclusion
Adaptive cache replacement policy guaranteed a good performance with small percentage of error
We run our experiments using different configurations on different benchmarks to show that the adaptive policy is able to perform good on multiple platforms
Future Recommendations
Implement other replacement algorithms such as, Pseudo LRU (PLRU) and Segmented LRU (SLRU).
Adaptively decide for replacement algorithm among more than 2 replacement policies

16. References
[1] Smaragdakis, Yannis. "General adaptive replacement policies." Proceedings of the 4th international symposium on Memory management. ACM, 2004.
[2] Subramanian, Ranjith, Yannis Smaragdakis, and Gabriel H. Loh. "Adaptive caches: Effective shaping of cache behavior to workloads." Proceedings of the 39th Annual IEEE/ACM International Symposium on Microarchitecture. IEEE Computer Society, 2006.
[3] Smaragdakis, Yannis. "General adaptive replacement policies." Proceedings of the 4th international symposium on Memory management. ACM, 2004.
[4] E. G. Hallnor and S. K. Reinhardt. A Fully Associative Software-Managed Cache Design. In Proceedings of the 27th International Symposium on Computer Architecture, Vancouver, Canada, June

17. Questions?