1. University of Toronto Department of Electrical And Computer Engineering Jason Zebchuk zebchuk@eecg.toronto.edu RegionTracker: Optimizing On-Chip Cache Lookups

2. zebchuk@eecg.toronto.edu June 9, 2006 2 What is a Cache? CPU Memory CPU Memory Cache Either Big & Slow or Small & Fast Big & Slow

3. zebchuk@eecg.toronto.edu June 9, 2006 3 How Does a Cache Work? Tag 1Tag 2Tag 3Tag 4 Data 1Data 2Data 3Data 4 TagIndexOffset Address: ==== Select Tag ArrayData Array

4. zebchuk@eecg.toronto.edu June 9, 2006 4 Towards Large On-Chip Caches CPU CPU: Memory: CPUCache 1971 1989

5. zebchuk@eecg.toronto.edu June 9, 2006 5 Towards Large On-Chip Caches Cache CPU 2006 1995 CPU Larger, Slower caches  How can we make caches Fast again?

6. zebchuk@eecg.toronto.edu June 9, 2006 6 Tag Lookup Plays Critical Role Larger Caches are Slower, and use More Power Serial Tag-Data lookup to reduce power CPU 1.Search for Tag in Tag Array 2.Then read desired block from data array Tag Array Data Array ð Lots of fine-grain information ð Can We Improve This?

7. zebchuk@eecg.toronto.edu June 9, 2006 7 Key Insight Only a few regions in cache at a time. Can we use something small and fast to track these regions? Memory Cache

8. zebchuk@eecg.toronto.edu June 9, 2006 8 RegionTracker Design Cached Block Vector (CBV) Cached Region Hash (CRH) (not shown) Track which Regions are currently Cached Allocated new CBV entry on access to new region Combine Coarse-Grain and Fine-Grain Information block 0 block n-1 region location 4 28

9. zebchuk@eecg.toronto.edu June 9, 2006 9 Results and Insights  Takes advantage of common program behavior  Access a few large, continuous areas of memory at a time  Simple, Small Structures  Compact Information Encoding & Fast Access  Still use tag array when RegionTracker misses  Reduces Cache Latency  Can Reduce Cache Lookup Power  Software does NOT change