Reuse Distance as a Metric for Cache Behavior Kristof Beyls and Erik D’Hollander Ghent University PDCS - August 2001

Reuse distance as a metric for cache behavior - pdcs Overview 1. Introduction 2. Reuse distance ↔ cache behavior 3. Effect of compiler optimization 4. Capacity miss reduction techniques 5. Conclusion

Reuse distance as a metric for cache behavior - pdcs Overview 1. Introduction 2. Reuse distance ↔ cache behavior 3. Effect of compiler optimization 4. Capacity miss reduction techniques 5. Conclusion

Reuse distance as a metric for cache behavior - pdcs Introduction  Gap between processor and memory speed widens exponentially fast –Typical: 1 memory access = 100 processor cycles  Caches can deliver data more quickly, but have limited capacity  Reuse distance is a metric for a programs cache performance

Reuse distance as a metric for cache behavior - pdcs Overview 1. Introduction 2. Reuse distance ↔ cache behavior 3. Effect of compiler optimization 4. Capacity miss reduction techniques 5. Conclusion

Reuse distance as a metric for cache behavior - pdcs a Reuse distance Definition: The reuse distance of a memory access is the number of unique addresses referenced since the last reference to the requested data. addrABCABBAC distance∞∞∞22012

Reuse distance as a metric for cache behavior - pdcs b Reuse distance and fully assoc. cache Lemma: In a fully assoc. LRU cache with n cache lines, a reference will hit if the reuse distance d<n. Corollary: In any cache with n lines, a cache miss with reuse distance d is: d < nConflict miss n ≤ d < ∞Capacity miss d = ∞Cold miss

Reuse distance as a metric for cache behavior - pdcs c Reuse Distance Distribution Spec95fp

Reuse distance as a metric for cache behavior - pdcs d Classifying cache misses for SPEC95fp Cache size ConflictCapacity

Reuse distance as a metric for cache behavior - pdcs e Reuse distance vs. hit probability

Reuse distance as a metric for cache behavior - pdcs Overview 1. Introduction 2. Reuse distance ↔ cache behavior 3. Effect of compiler optimization 4. Capacity miss reduction techniques 5. Conclusion

Reuse distance as a metric for cache behavior - pdcs a Reuse distance after optimization ConflictCapacity

Reuse distance as a metric for cache behavior - pdcs b Effect of compiler optimization  SGIpro compiler for Itanium  30% of conflict misses are removed, 1% of capacity misses are removed.  Conclusion: much work needs to be done to remove the most important kind of cache misses: capacity misses.

Reuse distance as a metric for cache behavior - pdcs Overview 1. Introduction 2. Reuse distance ↔ cache behavior 3. Effect of compiler optimization 4. Capacity miss reduction techniques 5. Conclusion

Reuse distance as a metric for cache behavior - pdcs Capacity miss reduction 1. Hardware level –Increasing cache size CS Reuse distance must be smaller than cache size 2. Compiler level –Loop tiling –Loop fusion 3. Algorithmic level CS

Reuse distance as a metric for cache behavior - pdcs a Hardware level  Increasing cache size  Other hardware techniques are hard to imagine. –Long distance between use and reuse of data for a capacity miss overview over a large portion of execution is needed

Reuse distance as a metric for cache behavior - pdcs b Compiler level  Loop tiling –Reduces long reuse distances in a single loop nest.  Loop fusion –Can reduce distances between consecutive loops.  Existing compiler techniques are not powerful enough. Only 1% of capacity miss is eliminated.

Reuse distance as a metric for cache behavior - pdcs c Algorithmic level  Programmer has a better under- standing of the global program structure.  Programmer can change algorithm, so that long distance reuses decrease.  Visualization of the long reuse distance can help the programmer to identify bad data locality in the code.

Reuse distance as a metric for cache behavior - pdcs Overview 1. Introduction 2. Reuse distance ↔ cache behavior 3. Effect of compiler optimization 4. Capacity miss reduction techniques 5. Conclusion

Reuse distance as a metric for cache behavior - pdcs Conclusion  Reuse distance predicts cache behavior accurately, even for direct mapped caches.  Compiler optimizations for eliminating capacity misses are currently not powerful enough. A large overview over the code is needed.  Programmer has large overview of code. Reuse distance visualization can help the programmer to identify regions with bad locality.