Exploiting Spatial Locality in Data Caches using Spatial Footprints Sanjeev Kumar, Princeton University Christopher Wilkerson, MRL, Intel

July 1, 1998Princeton University2 Spatial Locality in Caches Current approach: Exploit spatial locality within a cache line Small cache line –Lower bandwidth – Less pollution Big cache line –Exploit more locality –Fewer tags Current caches use 32 byte lines Access Cache Line { Memory

July 1, 1998Princeton University3 Spatial Locality in Caches contd. Spatial locality exhibited varies –Across applications –Within an application Using fixed size line to exploit locality –Inefficient use of cache resources »Less than half the data fetched is used »Wastes bandwidth, Pollutes cache –Limited benefit of spatial locality

July 1, 1998Princeton University4 Outline Introduction Spatial Footprint Predictors Practical Considerations Future Work Related Work Conclusions

July 1, 1998Princeton University5 Spatial Footprint Predictor (SFP) Exploit more spatial locality Need to reduce pollution Fetch words selectively Requires accurately predicting spatial footprint of a block –Bit Vector – Access Memory

July 1, 1998Princeton University6 Spatial Footprint Predictor contd. Record spatial footprints Use footprint history to make predictions –Lookup table based on »Nominating data address »Nominating instruction address »Combination L1 data caches Nominating Access (NA) Memory

July 1, 1998Princeton University7 Simple Approach Use large cache lines –Fetch specific words –leave holes for words that were not fetched Might decrease bandwidth Increases miss ratio –missed lines –under-utilization of cache MemoryCache Line

July 1, 1998Princeton University8 Our Approach Regular cache with small lines –8 bytes i.e. one word Exploit spatial locality at sector granularity –16 lines i.e. 128 bytes Spatial Footprint Predictor –Fetch 1-16 lines in a sector on a miss MemoryCache Line Sector {

July 1, 1998Princeton University9 When to Record/Predict footprints? Sectors in memory are active or inactive –Active » Record footprints »cache miss in an inactive sector –Inactive » Use history to predict »Cache miss on a line that is marked used (footprint) in an active sector Use infinite size tables

July 1, 1998Princeton University10 Recording Footprints Active Sector Table Spatial Footprint History Table NA SF Done Memory Cache Access (Records FP)(Stores FP)

July 1, 1998Princeton University11 Predicting Footprints Spatial Footprint History Table Memory Cache Fetch Lines Predicted Footprint SF Access

July 1, 1998Princeton University12 The default footprint predictor When SFP has no prediction –No history –Evicted from Spatial Footprint Table Picks a single line size for the application –Based on the footprints observed

July 1, 1998Princeton University13 Experimental Setup Cache Parameters –16 KB L1 –4-way associative –8 bytes per line –16 lines per sector Cache simulator –Miss Ratios –Fetch bandwidth 12 Intel MRL traces –gcc and go (SPEC) –Transaction processing –Web server –PC applications »word processors »spreadsheets Normalized results –16KB conventional cache with 32 byte line

July 1, 1998Princeton University14 Experimental Evaluation Normalized Miss Ratios

July 1, 1998Princeton University15 GCC Comparison

July 1, 1998Princeton University16 GCC Comparison Contd. Comparing SFP cache to –Conventional caches with varying line sizes »Comparable to best miss ratio (using 64 byte lines) »Close to lowest bandwidth (using 8 byte lines) –Bigger conventional cache »Comparable to a 32 KB Cache

July 1, 1998Princeton University17 Outline Introduction Spatial Footprint Predictors Practical Considerations Future Work Related Work Conclusions

July 1, 1998Princeton University18 Decoupled Sectored Cache Seznec et. al. –Proposed to improve sectored L2 cache Decouple tag array from data array –Dynamic mapping: no longer one-to-one –Multiple lines from the same sector share tags –Flexible: Data and tag array can be of different sizes and associativities

July 1, 1998Princeton University19 Practical Considerations Reasonable Spatial Footprint History Table –1024 entries Reduce Tag Storage –Use Decoupled Sectored Cache –Same number of tags as a conventional cache with 32 byte lines –Both data and tag array are 4-way associative

July 1, 1998Princeton University20 Experimental Evaluation Normalized Miss Ratios

July 1, 1998Princeton University21 Cost Additional Space –9 KB –Can be reduced by »Using partial tags »Compressing footprints Time –Most predictor actions are off the critical path –Little impact on cache access latency

July 1, 1998Princeton University22 Future Work Improve miss ratios further –Infinite tables: 30% –Practical Implementation: 18% Reduce the additional memory required Better coarse grained predictor L2 Caches

July 1, 1998Princeton University23 Related Work Przybylski et. al., Smith et. al. –Statically best line size, fetch size Gonzalez et. al. –Dual data cache: temporal and spatial locality –Numeric codes Johnson et. al. –Dynamically pick line size per block (1 KB)

July 1, 1998Princeton University24 Conclusions Spatial Footprint Predictors –Decrease miss ratio (18% on average) –Reduce bandwidth usage –Little impact on cache access latency Can use fine-grain predictor for data caches