1. Reuse Distance as a Metric for Cache Behavior Kristof Beyls and Erik D’Hollander Ghent University PDCS - August 2001

2. Reuse distance as a metric for cache behavior - pdcs2001 2 Overview 1. Introduction 2. Reuse distance ↔ cache behavior 3. Effect of compiler optimization 4. Capacity miss reduction techniques 5. Conclusion

3. Reuse distance as a metric for cache behavior - pdcs2001 3 Overview 1. Introduction 2. Reuse distance ↔ cache behavior 3. Effect of compiler optimization 4. Capacity miss reduction techniques 5. Conclusion

4. Reuse distance as a metric for cache behavior - pdcs2001 4 1. Introduction  Gap between processor and memory speed widens exponentially fast –Typical: 1 memory access = 100 processor cycles  Caches can deliver data more quickly, but have limited capacity  Reuse distance is a metric for a programs cache performance

5. Reuse distance as a metric for cache behavior - pdcs2001 5 Overview 1. Introduction 2. Reuse distance ↔ cache behavior 3. Effect of compiler optimization 4. Capacity miss reduction techniques 5. Conclusion

6. Reuse distance as a metric for cache behavior - pdcs2001 6 2.a Reuse distance Definition: The reuse distance of a memory access is the number of unique addresses referenced since the last reference to the requested data. addrABCABBAC distance∞∞∞22012

7. Reuse distance as a metric for cache behavior - pdcs2001 7 2.b Reuse distance and fully assoc. cache Lemma: In a fully assoc. LRU cache with n cache lines, a reference will hit if the reuse distance d<n. Corollary: In any cache with n lines, a cache miss with reuse distance d is: d < nConflict miss n ≤ d < ∞Capacity miss d = ∞Cold miss

8. Reuse distance as a metric for cache behavior - pdcs2001 8 2.c Reuse Distance Distribution Spec95fp

9. Reuse distance as a metric for cache behavior - pdcs2001 9 2.d Classifying cache misses for SPEC95fp Cache size ConflictCapacity

10. Reuse distance as a metric for cache behavior - pdcs2001 10 2.e Reuse distance vs. hit probability

11. Reuse distance as a metric for cache behavior - pdcs2001 11 Overview 1. Introduction 2. Reuse distance ↔ cache behavior 3. Effect of compiler optimization 4. Capacity miss reduction techniques 5. Conclusion

12. Reuse distance as a metric for cache behavior - pdcs2001 12 3.a Reuse distance after optimization ConflictCapacity

13. Reuse distance as a metric for cache behavior - pdcs2001 13 3.b Effect of compiler optimization  SGIpro compiler for Itanium  30% of conflict misses are removed, 1% of capacity misses are removed.  Conclusion: much work needs to be done to remove the most important kind of cache misses: capacity misses.

14. Reuse distance as a metric for cache behavior - pdcs2001 14 Overview 1. Introduction 2. Reuse distance ↔ cache behavior 3. Effect of compiler optimization 4. Capacity miss reduction techniques 5. Conclusion

15. Reuse distance as a metric for cache behavior - pdcs2001 15 4. Capacity miss reduction 1. Hardware level –Increasing cache size CS Reuse distance must be smaller than cache size 2. Compiler level –Loop tiling –Loop fusion 3. Algorithmic level CS

16. Reuse distance as a metric for cache behavior - pdcs2001 16 4.a Hardware level  Increasing cache size  Other hardware techniques are hard to imagine. –Long distance between use and reuse of data for a capacity miss overview over a large portion of execution is needed

17. Reuse distance as a metric for cache behavior - pdcs2001 17 4.b Compiler level  Loop tiling –Reduces long reuse distances in a single loop nest.  Loop fusion –Can reduce distances between consecutive loops.  Existing compiler techniques are not powerful enough. Only 1% of capacity miss is eliminated.

18. Reuse distance as a metric for cache behavior - pdcs2001 18 4.c Algorithmic level  Programmer has a better under- standing of the global program structure.  Programmer can change algorithm, so that long distance reuses decrease.  Visualization of the long reuse distance can help the programmer to identify bad data locality in the code.

19. Reuse distance as a metric for cache behavior - pdcs2001 19 Overview 1. Introduction 2. Reuse distance ↔ cache behavior 3. Effect of compiler optimization 4. Capacity miss reduction techniques 5. Conclusion

20. Reuse distance as a metric for cache behavior - pdcs2001 20 5. Conclusion  Reuse distance predicts cache behavior accurately, even for direct mapped caches.  Compiler optimizations for eliminating capacity misses are currently not powerful enough. A large overview over the code is needed.  Programmer has large overview of code. Reuse distance visualization can help the programmer to identify regions with bad locality.