ISLPED’99 International Symposium on Low Power Electronics and Design Way-Predicting Set-Associative Cache for High Performance and Low Energy Consumption Koji Inoue, Tohru Ishihara, and Kazuaki Murakami Department of Computer Science and Communication Engineering Kyushu University ppram@c.csce.kyushu-u.ac.jp

Conventional 4-Way Set-Associative Cache Tag subarray Cache-line subarray Way 0 Way 1 Way 2 Way 3 Ｓｔｅｐ１． Address Decode Decode circuit Ｓｔｅｐ２．Read out of a tag and a line from each way Activate of word line Activate senseamps pre(dis)charge bit lines Total energy for an access for decode for I/O pin drive Ecache = Edecode + Ememory + Eio Ｓｔｅｐ３． Tag comparison for SRAM access Hit Miss Ｓｔｅｐ４．Provide the required data Ｓｔｅｐ４．Cache replacement Activate of I/O pins

Phased 4-Way Set-Associative Cache for Low Energy Consumption Energy consumption improvement by sacrificing the performance Ｓｔｅｐ１． Address Decode Ｓｔｅｐ２．Read out of only tags Cycle 1 Ｓｔｅｐ３． Tag comparison Miss Hit Ｓｔｅｐ４． Cache replacement Cycle 2 Ｓｔｅｐ４．Read out of only the desired line Ｓｔｅｐ５．Provide the required data

Way-Predicting Set-Associative Cache - Concept - How can we achieve high-performance and low energy consumption at the same time? Fast access by reading out both of tag and line simultaneously Conventional : Good! Phased : Bad! Low energy by avoiding unnecessary line read access Conventional : Bad! Phased : Good! Predict which way has the data desired by the processor before the cache access is started

4Way-Predicting Set-Associative Cache - Operation - Way Prediction (Cache-line Base MRU Algorithm) Ｓｔｅｐ０．Way prediction Ｓｔｅｐ１． Address decode Ｓｔｅｐ２．Read out the predicted tag and line Cycle 1 Ｓｔｅｐ３． Tag comparison Miss Prediction Hit Ｓｔｅｐ４．Read out the remaining tags and lines Ｓｔｅｐ４．End Cycle 2 Ｓｔｅｐ５． Tag comparison Prediction Miss Cache Miss Ｓｔｅｐ６．End Ｓｔｅｐ６．Cache replacement

4Way-Predicting Set-Associative Cache - Organization - MRU Algorithm

Evaluation Environment Cache Models Conventional 4-way Set-Associative Cache (4SACache) Phased 4-way Set-Associative Cache (P4SACache) Way-Predicting 4-way Set-Associative Cache (WP4SACache) Cache Size : 16 K Byte, Cache-line Size : 32 Byte, Replacement Algorithm : LRU Evaluation Items Performance (Tcache)： average number of clock cycles for an access Energy (Ecache) ： average energy consumption for an access Energy consumed for accessing a tag-subarray Energy consumed for accessing a line-subarray Ecache ~ Ememory = Ntag ｘ Etag ＋ Ndata ｘ Edata Ave. number of tag-subarray accessed for an access Ave. number of line-subarray accessed for an access

Static Analysis - Energy and Performance Expression - 4SACache P4SACache E4SACache EP4SACache 4 Etag + 4 Edata 4 Etag + Edata x CHR T4SACache TP4SACache 1 1 + 1 x CHR EWP4SACache WP4SACache (Etag + Edata) + (3 Etag + 3 Edata) x (1 - PHR) TWP4SACache CHR:Cache Hit Rate PHR:Prediction Hit Rate 1 + 1 x (1 - PHR)

Static Analysis - Best and Worst Case - 4SACache (Conventional) P4SACache (Phased) WP4SACache (Ours) Energy Consumption (Etag = 0.078Edata) Performance Compare with Conventional (4SACache) Best Case (PHR = 100%) ： 75% energy improvement without any performance degradation Worst Case (PHR = 0%) ： 100% performance overhead without any energy improvement

Experimental Analysis - Prediction Hit Rate -

Experimental Analysis - Result of Instruction Cache - 4SACache = 1.0 P4SACache Normalized Tcache WP4SACache (Our approach) Normalized Ecache

Experimental Analysis - Result of Data Cache - 4SACache = 1.0 P4SACache Normalized Tcache WP4SACache (Our approach) Normalized Ecache

Experimental Analysis - Energy and Performance - Average of all benchmarks Conventional (4SACache) Phased (P4SACache) Way-Predicting (WP4SACache) 199.4% 195.8% 200 200 I-Cache D-Cache 113.0% 104.1% Normalized Results (%) 100 Normalized Results (%) 100 30.3% 29.4% 28.1% 35.2% Ecache Tcache Ecache Tcache

Cache Power Consumption Cache Size trend Effect of on-chip caches to total chip power consumption DEC 21164 CPU* StrongARM SA-110 CPU* Bipolar ECL CPU** 25% 43% 50% * Kamble, et. Al., “Analytical energy Dissipatiion Models for Low Power Caches”, ILPED’97 ** Joouppi, et. Al., “A 300-MHz 115-W 32-b Bipolar ECL Microprocessor” ,IEEE Journal of Solid-State Circuits’93

Energy Consumption Model Components of the power dissipation Bit line Word line Sense Amp Output driver Addr input Comparator Latche 32KB Direct-mapped I-Cache 32KB 4-way D-Cache Ememory=95.6% Ememory=97.7% Ghose, et. Al. : Energy Efficient Cache Organizations for Superscalar Processors, Power-Driven microarchitecture Workshop in Conjunction with ISCA’98 Average Energy Consumption for an access Energy consumed for accessing a tag-subarray Energy consumed for accessing a line-subarray Ecache ~ Ememory = Ntag ｘ Etag ＋ Ndata ｘ Edata Ave. number of tag-subarray accessed for an access Ave. number of line-subarray accessed for an access

Experimental Analysis - Environment - Benchmarks SPECint95 099.go, 124.m88ksim, 126.gcc, 129.compress, 130.li, 132.ijpeg, 134.perl, 147.vortex SPECfp95 101.tomcatv, 102.swim, 103.su2cor, 104.hydro2d