Increasing Effective Cache Capacity Through the Use of Critical Words Mentor: Amirhossein Mirhosseininiri Mentees: Ben Schreiber, Pooja Welling Motivation Results Caches are an integral part of modern computing. They act as a buffer between the CPU and the main memory, storing recently used data for quick access. Caching lowers the amount of main memory accesses. The prevention of these costly accesses saves time and energy. This project seeks to increase the effective capacity of the cache. The goal of this research is to develop a scheme where only the first needed part of a block of memory is stored, with the rest of the block being fetched while the first part is processed. Background Main memory is made from Dynamic RAM (DRAM), which is slow Caching is possible because of the principle of locality Spatial Locality: Data close to an access is likely to be accessed Temporal Locality: Data is likely to be accessed multiple times Caching data by this principle helps prevent accessing Main Memory Caches are high speed storage made from the fast static RAM (SRAM) Data is pulled from memory in blocks of words, which group in sets Critical Words Cache The hit rates for the dynamic critical words cache are significantly larger in comparison to the L2 cache and the critical words cache Specifically, the dynamic critical words cache showed a 50.1% improvement in hit rate over the L2 cache, while the critical words cache only showed a 28.4% improvement This indicates that the dynamic critical words method of moving blocks into the cache is an effective solution towards improving cache performance In addition, the average memory access time for the dynamic critical words cache is less than those of the standard and critical words cache Here, the dynamic critical words cache showed a 22.7% improvement in average memory access time over the L2 cache, while the critical words cache only showed a 14.3% improvement Overall, the dynamic critical words cache is seen to be the most effective approach towards achieving the ideal L2 cache results Sometimes, the same word in a block is always accessed first The same index in an array, the same method in a class, etc. Only store the 2 words needed first, the critical words By only storing 2 words of 8, the cache effectively grows by factor of 4 When requested, the critical words are sent to the processor While the critical words are processed, the rest of the block is fetched Ideally, the rest of the block arrives before it is needed A penalty is incurred if the critical words are mispredicted Same effect as a normal cache miss Dynamic Critical Words Cache Further Plans The results indicate that this is an effective approach. The simulation will be extended to include the dynamic resizing of the critical words portion against the whole block portion. This will allow for greater flexibility against workloads that are overwhelmingly predictable or unpredictable. Furthermore, other factors must also be measured. Of particular importance is energy consumption. In the critical words scheme, memory is accessed for every read. This causes a substantial increase in power consumption that must be evaluated. To collect this data, a model of the cache will need to be simulated in Verilog. Critical words are not always predictable Split L2 into two parts: Critical word cache for predictable data Traditional, whole block cache for unpredictable data Assume new data is predictable (predictable bit is 0) If successful, keep predictable bit as “0” If not successful, set predictable bit to “1”, move to whole block side Store predictable bit with data in all higher levels of memory hierarchy Acknowledgements Thank you to our mentor, Amirhossein Mirhosseininiri and the PURE Committee for providing us with this opportunity. References Huang, Cheng-Chieh, and Vijay Nagarajan. "Increasing cache capacity via critical-words-only cache." Computer Design (ICCD), 2014 32nd IEEE International Conference on. IEEE, 2014.