Improving Multi-Core Performance Using Mixed-Cell Cache Architecture Samira Khan*†, Alaa R. Alameldeen*, Chris Wilkerson*, Jaydeep Kulkarni* and Daniel A. Jiménez§ *Intel Labs †Carnegie Mellon University §Texas A&M University

Summary Problem: Cache cells become unreliable at low voltage Mixed-cell cache: Use some larger robust cells [Ghasemi 2011] Smaller non-robust cells are turned off at low voltage Capacity loss leads to performance loss Goal: No capacity loss at low voltage to gain high performance Observation: A clean line has a duplicate copy in the memory hierarchy A modified line is the only existing copy Our Approach: Protect a modified line in larger robust cells Store a clean line in smaller non-robust cells Fetch data from the lower level on an error in a clean line Todays systems are power limited. In a given power budget, We can activate more cores if we lower the voltage. The problem is caches become unreliable in the low voltage. Prior work has proposed to use mixed cell cache To solve this problem. Mixed cell cache has some larger robust cells, these cells are reliable even in low voltage. At higher voltage both Robust and non-robust cells are operational. In the low voltage regular cells are turned off. This capacity loss leads to significant performance loss in the low voltage. The goal of this work is to gain high performance using the whole cache The intuition is, a clean line has extra copies in the memory hierarchy, So there is natural redundancy of data in the clean lines. But a modified line is the only existing copy. So a modified line is critical and needs to be protected. We propose to use the whole cache by protecting modified data in the robust cells, so that there is no error. And store the clean data in the Non-robust cells. On a error to a clean line, we propose to fetch the data from the lower level of the memory hierarchy. Our design significantly improves performance and reduces power over the prior work. Significantly improves performance and reduces power compared to prior work

Outline Summary Background and Motivation Mixed-Cell Cache Architecture Methodology and Results Conclusion

Background and Motivation Multi-core designs are power-limited Can activate more cores by lowering the voltage Todays systems are power limited. In a given power budget, we can activate more cores if we can lower the Voltage. In the figure the system can activate just two cores in its power budget. However, if it lowers the Voltage, it can turn on 4 cores. If we lower the voltage more, we can activate 8-cores. So lowering voltage Enables more active cores on the die. Voltage Scale More active cores at low voltage

Ensuring Resiliency at Lower Voltage Cache cells begin to fail at lower voltage Non-robust Robust Error Cache Mixed-Cell Cache But the problem with lowering the voltage is that cache cells begin to fail at low voltage. In the picture the black rectangle represents error. To solve this problem previous work Has proposed mixed cell cache. Mixed cell cache has some larger robust cells. These cells Can work at the lower voltage. However the have area and power overhead. In the left Picture, the bigger cells are the robust cells. At the low voltage, only robust cells are Operational and the other regular cells are disabled. Capacity loss at the lower voltage can Degrade performance, especially in the multi-core systems. Mixed-Cell Cache [Ghasemi 2011] Some ways built with robust cells + Resilient to error at low voltage - Area and power overhead Only robust cells are operational at low voltage Cache capacity loss at lower voltage can degrade performance significantly

Effect of Cache Capacity Reduction in a 4-Core System In this slide we show the effect of cache capacity reduction in a 4-core system. Here the cores are running at lower voltage, so the frequency gets scaled down Too. The cores are running at 825MHz. In the x axis we show two configurations, one is Using the whole cache and the other one id the prior work, that disables 75% of the cache. Here all the level of the caches are mixed-cell cache. The benchmarks we have used are multi-programmed SPEC 2006. This graph shows that 75% reduction in cache Capacity loss leads to 20% slow down on average. In our experiments, 75% reduction in cache capacity leads to 20% performance loss on average

Improve performance using the whole cache Goal: Improve performance using the whole cache at low voltage

Outline Summary Background and Motivation Mixed-Cell Cache Architecture Methodology and Results Conclusion Now I will describe the mixed-cell architecture that we propose.

Our Mixed-Cell Architecture Observation: A Clean line has a duplicate copy in the memory hierarchy On an error, can get the data from the duplicate copy A Modified line is the only copy in the system Critical to keep the data error free Idea: Protect a modified line using larger robust cells Store a clean line in smaller non-robust cells Use parity/ECC to detect errors in clean lines Fetch data from the lower level on an error in clean lines The intuition behind the work is a clean line has a duplicate copy in the memory hierarchy. So if there is a error, we can get the from the duplicate copy. However a modified line is the only existing copy in the system. So it is critical to keep the data error free. We propose to protect the modified lines In the robust cells. And store the clean lines in non-robust cells. We use parity/ error correction Code to detect errors in the clean lines. On an error in a clean line, we will treat it as a miss And issue a miss to get the data from the lower-level of the memory hierarchy.

Our Mixed-Cell Architecture Non-robust Robust Clean Modified Mixed-cell (Disable) [Ghasemi 2011] Our Design In this slide we show our proposed mechanism pictorially. In the left picture We show the mixed cell cache where non-robust cells are disabled in the low Voltage. We want to use both the robust and non-robust ways in the low voltage. In the right side, we show our design. We store modified data only in Robust ways and store clean lines only in non-robust ways. And we Cache only cache management policy to ensure that modified lines Are always protected in the robust ways. Use both robust and non-robust ways at low voltage A modified line is stored only in a robust way A clean line is stored only in a non-robust way Modify cache management techniques to ensure clean and modified lines are stored appropriately

Mixed-Cell Architecture: Cache Miss Write miss: Allocate line in a robust way Read miss: Allocate line in a non-robust way X Y A B On a cache miss we allocate the data depending on the type off the miss. On a read miss, we allocate the line a robust way and on a write miss We allocate the data in a non-robust way. In this figure, write misses is allocated in a robust ways and read misses Are allocated in non-robust way. LRU LRU LRU LRU Write miss X Write miss Y Read miss A Read miss B Time

Mixed-Cell Architecture: Cache Hit F G H I J K L Write Hit E Write Hit G Read Hit J Read hit: No change Write hit: Write hit in robust: No change Write hit in non-robust: We propose three mechanisms Writeback Swap Duplicate So let’s se what happens on a cache hit. If we have a read hit, We are just reading from a line, we re not changing the clean/dirty Status of that line. So we do not need to change anything. If we have a write hit, we need to consider two cases. If the write hits in a non-robust way, The modified line is already protected in the larger cells. So again we do not have to change anything. However, if the write hits in a non-robust way, that means that the line was allocated on a read miss And now we are writing to it. So now we have a modified line vulnerable in a non-robust way. We propose three simple mechanisms to solve this case. They are writeback, swap and duplicate.

Write to a Non-Robust Line: Writeback Write it back in the next level of memory hierarchy Make data clean in the non-robust cell Write Hit G E F G H I J K L In the writeback mechanism, we propose to write the data back to next level of memory hierarchy And make the data clean. For example, if we have write hit at G, now G becomes vulnerable. So we writeback G so that non-robust cell Contains clean data. The advantage of this mechanism is that this is simple. But the advantage is now we have an extra writeback at each write in a non-robust cell. Now this block contains clean data Dirty block in non-robust way is vulnerable, writeback G + Simple - An extra writeback at each write in a non-robust way

Write to a Non-Robust Line: Swap Swap modified line with the LRU robust line Writeback the robust data to next cache level Write Hit G E F G H I J K L The next mechanism we have is swap. In this mechanism, we propose to swap the modified line with the LRU Robust line. Then we writeback the robust data to the next level of hierarchy. In this example, if we have write hit to G, we swap G and E and writeback E. The advantage of this mechanism is that it increases write hits in the robust cells. This is mainly because writes have locality. The disadvantage is now we have to swap two cache lines, so there is the extra latency For the swap. Now this block contains clean data Swap E and G, E is now vulnerable, writeback E + Increases write hits in robust cells - Extra latency for swap

Write to a Non-Robust Line: Duplicate Pair two non-robust ways Static pairing: way <0,1>, <2,3>… Duplicate the data in the partner way On an error in one way, use data from the partner way Write Hit G The next mechanism that we have is duplicate. In this mechanism, we pair up two non-robust ways. We are using static pairing, so way 0 is always paired with way 1 and so on. In case of a write to a non-robust way, we duplicate the data in the partner way. If we have error in one copy, We can use the other copy to get the actual data. The probability that both of the ways will have error Is very very low. In this example, we have write hit at G. So we have duplicate G in its partner way. The advantage of this mechanism is that it is simple and does to require any extra writeback. The disadvantage is we are losing some cache capacity due to duplication and there is Also latency involved in the duplication. E F G G H I J K L , Duplicate G in the partner way + Simple, no extra writeback - Capacity loss, extra latency for duplication

Outline Summary Background and Motivation Mixed-Cell Cache Design Methodology and Results Conclusion

Evaluation Methodology Simulator: CMP$im, a Pin-based x86 simulator [Jaleel 2008] Benchmarks: 20 4-core multi-programmed mixes from SPEC 2006 Each cache has 2 robust ways L1D 32KB, 2 robust, 6 non-robust ways, 3 cycles L2 256KB, 2 robust, 6 non-robust ways, 10 cycles L3 shared 4MB, 2 robust, 14 non-robust ways, 25 cycles Memory latency 80 cycles Vmin 590 mV, 825 MHz

Comparison Points Robust: Cache uses only robust cells Smaller capacity, L1D 20KB, L2 160KB, L3 2.25MB Disable: Mixed-Cell Cache [Ghasemi 2011] Only ¼ of the cache works at low voltage, L1D 8KB, L2 64KB, L3 1MB Ideal: Cache uses only non-robust cells Larger capacity, L1 40KB, L2 320KB, L3 4.5MB Can not work at low voltage Can provide higher voltage to cache using a separate Vcc Increases complexity Adds latency to signals crossing voltage domains Now I will talk about our comparison points. We compare our mechanisms with Robust where all the cache cells are made of robust cells. But it has smaller capacity since the robust cells are bigger. We compare are mechanisms with Disable, which is the prior work where only one fourth of the cache is on At low voltage. We also compare with Idea. Here the cache has only smaller cells. So it has larger cache capacity. But note that This cells would not work at low voltage. One way to make them work is to use separate power supplies for the core And the cache, so the core can run at a lower voltage but the cache would have a higher voltage. But using separate Voltage comes with additional cost and complexity.

4-Core Performance at Low Voltage 17% 2.6% Here we show the performance of the 4 core system. In the axis we show all the mechanisms and in the Y axis we show the Weighted speedup normalized to Disable, where 3/4th of the cache is off at low voltage. The blue bars are our mechanisms. Robust performs better than disable as it has higher cache capacity. Our first mechanism writeback actually degrades performance. It shows that extra writebacks at write hits in the non-robust ways has a significant impact on the performance. The next two bars Are Swap and duplicate. They both improve the performance. On average swap improves performance by 17% compared to Disable and is within 2.6% of the Idea case. Swap provides 17% speedup over Disable Swap performs within 2.6% of Ideal

Normalized Memory Bandwidth 6.15 21% 28.5% 3% Since our mechanisms introduce extra writebacks, we show the normalized memory bandwidth in this graph. Again in the x axis we have all the mechanisms and we show the normalized memory bandwidth in the y axis Compared to prior work disable. Robust uses less bandwidth than disable as disable has smaller capacity it Issues more misses. Our mechanism writeback increases memory bandwidth by more than 6X. However, both Swap and duplicate uses lower bandwidth. Actually duplicate increases memory bandwidth by only 3% Compared to idea case. Duplicate increases memory bandwidth by only 3% compared to Ideal

Normalized LLC Static Power at Vmin (590mV) 10% 2.3X Since robust cells has lore static and dynamic power, we show the power consumption of different Mechanisms. First we show the LLC static power as that is leakage is the major power component In LLC. Again in the x axis we show the mechanisms and in the y axis we show the normalized LLC Static power compared to Disable. Robust uses more static power as it is bigger than disable. Both swap and duplicate actually increases static power as they are bigger and they uses larger Robust cells. However they reduce static power by 10% compared to Ideal case, where cache and Core uses separate power supply, without the additional overhead of the separate power supplies. Swap and Duplicate reduce LLC static power by 10% compared to Ideal

Normalized L1D Dynamic Power at Vmin (590mV) 22% 50% Here we show the dynamic power of L1 data cache. As before, in the x axis we have all the mechanisms And in the y axis we have normalized dynamic power compared to disable. Disable and robust has same Dynamic power as both of them are comprised of Only larger cells. So all the accesses are served by the Larger cells. Swap had duplicate significantly improve he dynamic power. However duplicate reduces power more than The swap mechanism. The reason is swap moves the modified lines in the robust ways. This increases write hits In the robust ways. But duplicate actually spreads the writes in the non-robust ways, so swap have more access At larger cells than the duplicate. Duplicate reduces dynamic power by 50% compared to Disable and is within 30% of the the ideal case. 30% Duplicate reduces dynamic power by 50% compared to Disable Duplicate is within 30% of the Ideal

Conclusion Problem: Cache cells become unreliable at low voltage Mixed-cell cache: Use some larger robust cells Smaller non-robust cells are turned off at low voltage Capacity loss leads to performance loss Goal: No capacity loss at low voltage to gain high performance Observation: A clean line has a duplicate copy in the memory hierarchy A modified line is the only existing copy Our Approach: Protect a modified line in larger robust cells Store a clean line in smaller non-robust cells Fetch data from the lower level on an error in a clean line The problem is caches become unreliable in the low voltage. Prior work has proposed to use mixed cell cache To solve this problem. Mixed cell cache has some larger robust cells, these cells are reliable even in low voltage. At higher voltage both Robust and non-robust cells are operational. In the low voltage regular cells are turned off. This capacity loss leads to significant performance loss in the low voltage. The goal of this work is to gain high performance using the whole cache The intuition is clean lines have extra copy in the memory hierarchy, So there is natural redundancy of data in the clean lines. But modified lines are the only existing copy. So modified lines are critical and they need to be protected. We propose to use the whole cache by protecting modified data in the robust cells and storing the clean data in the Non-robust cells. On a error to a clean line, we propose to fetch the data from the lower level of the memory hierarchy. Our design significantly improves performance and reduces power over the prior work. Improves performance by 17% and reduces L1D dynamic power by 50% compared to prior work

Thank you

Improving Multi-Core Performance Using Mixed-Cell Cache Architecture Samira Khan*†, Alaa R. Alameldeen*, Chris Wilkerson*, Jaydeep Kulkarni* and Daniel A. Jiménez§ *Intel Labs †Carnegie Mellon University §Texas A&M University