1. Exploiting Spatial Locality in Data Caches using Spatial Footprints Sanjeev Kumar, Princeton University Christopher Wilkerson, MRL, Intel

2. July 1, 1998Princeton University2 Spatial Locality in Caches Current approach: Exploit spatial locality within a cache line Small cache line –Lower bandwidth – Less pollution Big cache line –Exploit more locality –Fewer tags Current caches use 32 byte lines Access Cache Line { Memory

3. July 1, 1998Princeton University3 Spatial Locality in Caches contd. Spatial locality exhibited varies –Across applications –Within an application Using fixed size line to exploit locality –Inefficient use of cache resources »Less than half the data fetched is used »Wastes bandwidth, Pollutes cache –Limited benefit of spatial locality

4. July 1, 1998Princeton University4 Outline Introduction Spatial Footprint Predictors Practical Considerations Future Work Related Work Conclusions

5. July 1, 1998Princeton University5 Spatial Footprint Predictor (SFP) Exploit more spatial locality Need to reduce pollution Fetch words selectively Requires accurately predicting spatial footprint of a block –Bit Vector –0100 1011 0010 0010 Access Memory 0100101100100010

6. July 1, 1998Princeton University6 Spatial Footprint Predictor contd. Record spatial footprints Use footprint history to make predictions –Lookup table based on »Nominating data address »Nominating instruction address »Combination L1 data caches Nominating Access (NA) Memory 0100101100100010

7. July 1, 1998Princeton University7 Simple Approach Use large cache lines –Fetch specific words –leave holes for words that were not fetched Might decrease bandwidth Increases miss ratio –missed lines –under-utilization of cache MemoryCache Line

8. July 1, 1998Princeton University8 Our Approach Regular cache with small lines –8 bytes i.e. one word Exploit spatial locality at sector granularity –16 lines i.e. 128 bytes Spatial Footprint Predictor –Fetch 1-16 lines in a sector on a miss MemoryCache Line Sector {

9. July 1, 1998Princeton University9 When to Record/Predict footprints? Sectors in memory are active or inactive –Active » Record footprints »cache miss in an inactive sector –Inactive » Use history to predict »Cache miss on a line that is marked used (footprint) in an active sector Use infinite size tables

10. July 1, 1998Princeton University10 Recording Footprints Active Sector Table Spatial Footprint History Table NA SF 60100... 1000... 1001... Done Memory Cache 0100101100100010 Access (Records FP)(Stores FP)

11. July 1, 1998Princeton University11 Predicting Footprints Spatial Footprint History Table Memory Cache Fetch Lines Predicted Footprint SF 0100... 1000... 1001... 0100101100100010 Access

12. July 1, 1998Princeton University12 The default footprint predictor When SFP has no prediction –No history –Evicted from Spatial Footprint Table Picks a single line size for the application –Based on the footprints observed

13. July 1, 1998Princeton University13 Experimental Setup Cache Parameters –16 KB L1 –4-way associative –8 bytes per line –16 lines per sector Cache simulator –Miss Ratios –Fetch bandwidth 12 Intel MRL traces –gcc and go (SPEC) –Transaction processing –Web server –PC applications »word processors »spreadsheets Normalized results –16KB conventional cache with 32 byte line

14. July 1, 1998Princeton University14 Experimental Evaluation Normalized Miss Ratios

15. July 1, 1998Princeton University15 GCC Comparison

16. July 1, 1998Princeton University16 GCC Comparison Contd. Comparing SFP cache to –Conventional caches with varying line sizes »Comparable to best miss ratio (using 64 byte lines) »Close to lowest bandwidth (using 8 byte lines) –Bigger conventional cache »Comparable to a 32 KB Cache

17. July 1, 1998Princeton University17 Outline Introduction Spatial Footprint Predictors Practical Considerations Future Work Related Work Conclusions

18. July 1, 1998Princeton University18 Decoupled Sectored Cache Seznec et. al. –Proposed to improve sectored L2 cache Decouple tag array from data array –Dynamic mapping: no longer one-to-one –Multiple lines from the same sector share tags –Flexible: Data and tag array can be of different sizes and associativities

19. July 1, 1998Princeton University19 Practical Considerations Reasonable Spatial Footprint History Table –1024 entries Reduce Tag Storage –Use Decoupled Sectored Cache –Same number of tags as a conventional cache with 32 byte lines –Both data and tag array are 4-way associative

20. July 1, 1998Princeton University20 Experimental Evaluation Normalized Miss Ratios

21. July 1, 1998Princeton University21 Cost Additional Space –9 KB –Can be reduced by »Using partial tags »Compressing footprints Time –Most predictor actions are off the critical path –Little impact on cache access latency

22. July 1, 1998Princeton University22 Future Work Improve miss ratios further –Infinite tables: 30% –Practical Implementation: 18% Reduce the additional memory required Better coarse grained predictor L2 Caches

23. July 1, 1998Princeton University23 Related Work Przybylski et. al., Smith et. al. –Statically best line size, fetch size Gonzalez et. al. –Dual data cache: temporal and spatial locality –Numeric codes Johnson et. al. –Dynamically pick line size per block (1 KB)

24. July 1, 1998Princeton University24 Conclusions Spatial Footprint Predictors –Decrease miss ratio (18% on average) –Reduce bandwidth usage –Little impact on cache access latency Can use fine-grain predictor for data caches