University of Toronto Department of Electrical And Computer Engineering Jason Zebchuk RegionTracker: Optimizing On-Chip Cache Lookups

June 9, What is a Cache? CPU Memory CPU Memory Cache Either Big & Slow or Small & Fast Big & Slow

June 9, How Does a Cache Work? Tag 1Tag 2Tag 3Tag 4 Data 1Data 2Data 3Data 4 TagIndexOffset Address: ==== Select Tag ArrayData Array

June 9, Towards Large On-Chip Caches CPU CPU: Memory: CPUCache

June 9, Towards Large On-Chip Caches Cache CPU CPU Larger, Slower caches  How can we make caches Fast again?

June 9, Tag Lookup Plays Critical Role Larger Caches are Slower, and use More Power Serial Tag-Data lookup to reduce power CPU 1.Search for Tag in Tag Array 2.Then read desired block from data array Tag Array Data Array ð Lots of fine-grain information ð Can We Improve This?

June 9, Key Insight Only a few regions in cache at a time. Can we use something small and fast to track these regions? Memory Cache

June 9, RegionTracker Design Cached Block Vector (CBV) Cached Region Hash (CRH) (not shown) Track which Regions are currently Cached Allocated new CBV entry on access to new region Combine Coarse-Grain and Fine-Grain Information block 0 block n-1 region location 4 28

June 9, Results and Insights  Takes advantage of common program behavior  Access a few large, continuous areas of memory at a time  Simple, Small Structures  Compact Information Encoding & Fast Access  Still use tag array when RegionTracker misses  Reduces Cache Latency  Can Reduce Cache Lookup Power  Software does NOT change