CS252/Kubiatowicz Lec /22/03 CS252 Graduate Computer Architecture Lecture 15 Prediction (Finished) Caches I: 3 Cs and 7 ways to reduce misses October 22 nd, 2003 Prof. John Kubiatowicz

CS252/Kubiatowicz Lec /22/03 Review: Yeh and Patt classification GBHR GPHT PABHR PAPHT PABHR GAg PAgPAp GAg: Global History Register, Global History Table PAg: Per-Address History Register, Global History Table PAp: Per-Address History Register, Per-Address History Table

CS252/Kubiatowicz Lec /22/03 Review: Other Global Variants: Try to Avoid Aliasing GAsGShare GAs: Global History Register, Per-Address (Set Associative) History Table Gshare: Global History Register, Global History Table with Simple attempt at anti-aliasing GBHR PAPHT GPHT GBHR (  n) Address (n) 

CS252/Kubiatowicz Lec /22/03 An anti-aliasing predictor: Bi-Mode [Chih-Chieh Lee, I-Cheng K. Chen, and Trevor N. Mudge] Two separate Gshare predictors+Choser –One for each bias –Only one used/updated! Sort branches by bias –Meta predictor chooses –Contructive aliasing helps rather than hinders Direction Predictors History (  n) Address (n)  Bias: True Bias: False Result Choice Predictor s bits n bits

CS252/Kubiatowicz Lec /22/03 Review: What are Important Metrics? Clearly, Hit Rate matters –Even 1% can be important when above 90% hit rate Speed: Does this affect cycle time? Space: Clearly Total Space matters! –Papers which do not try to normalize across different options are playing fast and lose with data –Try to get best performance for the cost How many different predictors are there? –MANY, MANY, MANY!

CS252/Kubiatowicz Lec /22/03 An alternative: Genetic Programming for Design Genetic programming has two key aspects: –An Encoding of the design space. »This is a symbolic representation of the result space (genome). »Much of the domain-specific knowledge and “art” involved here. –A Reproduction strategy »Includes a method for generating offspring from parents Mutation: Changing random portions of an individual Crossover: Merging aspects of two individuals »Includes a method for evaluating the effectiveness (“fitness”) of individual solutions. Generation of new branch predictors via genetic programming: –Everything derived from a “basic” predictor (table) + simple operators. –Expressions arranged in a tree –Mutation: random modification of node/replacement of subtree –Crossover: swapping the subtrees of two parents. Paper by Joel Emer and Nikolas Gloy, "A Language for Describing Predictors and its Application to Automatic Synthesis"

CS252/Kubiatowicz Lec /22/03 Review: Memory Disambiguation Memory disambiguation buffer contains set of active stores and loads in program order. –Loads and stores are entered at issue time –May not have addresses yet Optimistic dependence speculation: assume that loads and stores don’t depend on each other Need disambiguation buffer to catch errors. All checks occur at address resolution time: –When store address is ready, check for loads that are (1) later in time and (2) have same address. »These have been incorrectly speculated: flush and restart –When load address is ready, check for stores that are (1) earlier in time and (2) have same address if (match) then if (store value ready) then return value else return pointer to reservation station else optimistically start load access

CS252/Kubiatowicz Lec /22/03 Review: STORE sets Naïve speculation can cause problems for certain load-store pairs. “Counter-Speculation”: For each load, keep track of set of stores that have forwarded information in past. –If (prior store in store-set has unresolved address) then wait for store address to be completed else if (match) then if (store value ready) then return value else return pointer to reservation station else optimistically start load access Index SSID Store Inum Load/Store PC Store Set ID Table (SSIT) Last Fetched Store Table (LFST)

CS252/Kubiatowicz Lec /22/03 Processor Only Thus Far in Course: –CPU cost/performance, ISA, Pipelined Execution CPU-DRAM Gap 1980: no cache in µproc; level cache on chip (1989 first Intel µproc with a cache on chip) Review: Who Cares About the Memory Hierarchy? µProc 60%/yr. DRAM 7%/yr DRAM CPU 1982 Processor-Memory Performance Gap: (grows 50% / year) Performance “Moore’s Law” “Less’ Law?”

CS252/Kubiatowicz Lec /22/03 Generations of Microprocessors Time of a full cache miss in instructions executed: 1st Alpha: 340 ns/5.0 ns = 68 clks x 2 or136 2nd Alpha:266 ns/3.3 ns = 80 clks x 4 or320 3rd Alpha:180 ns/1.7 ns =108 clks x 6 or648 Why not recompute the value rather than taking the time to fetch it from memory?

CS252/Kubiatowicz Lec /22/03 Processor-Memory Performance Gap “Tax” Processor % Area %Transistors (­cost)(­power) Alpha %77% StrongArm SA11061%94% Pentium Pro64%88% –2 dies per package: Proc/I$/D$ + L2$ Caches have no inherent value, only try to close performance gap

CS252/Kubiatowicz Lec /22/03 What is a cache? Small, fast storage used to improve average access time to slow memory. Exploits spacial and temporal locality In computer architecture, almost everything is a cache! –Registers a cache on variables –First-level cache a cache on second-level cache –Second-level cache a cache on memory –Memory a cache on disk (virtual memory) –TLB a cache on page table –Branch-prediction a cache on prediction information? Proc/Regs L1-Cache L2-Cache Memory Disk, Tape, etc. BiggerFaster

CS252/Kubiatowicz Lec /22/03 Example: 1 KB Direct Mapped Cache For a 2 ** N byte cache: –The uppermost (32 - N) bits are always the Cache Tag –The lowest M bits are the Byte Select (Block Size = 2 ** M) Cache Index : Cache Data Byte : Cache TagExample: 0x50 Ex: 0x01 0x50 Stored as part of the cache “state” Valid Bit : 31 Byte 1Byte 31 : Byte 32Byte 33Byte 63 : Byte 992Byte 1023 : Cache Tag Byte Select Ex: 0x00 9 Block address

CS252/Kubiatowicz Lec /22/03 Set Associative Cache N-way set associative: N entries for each Cache Index –N direct mapped caches operates in parallel Example: Two-way set associative cache –Cache Index selects a “set” from the cache –The two tags in the set are compared to the input in parallel –Data is selected based on the tag result Cache Data Cache Block 0 Cache TagValid ::: Cache Data Cache Block 0 Cache TagValid ::: Cache Index Mux 01 Sel1Sel0 Cache Block Compare Adr Tag Compare OR Hit

CS252/Kubiatowicz Lec /22/03 Disadvantage of Set Associative Cache N-way Set Associative Cache versus Direct Mapped Cache: –N comparators vs. 1 –Extra MUX delay for the data –Data comes AFTER Hit/Miss decision and set selection In a direct mapped cache, Cache Block is available BEFORE Hit/Miss: –Possible to assume a hit and continue. Recover later if miss. Cache Data Cache Block 0 Cache TagValid ::: Cache Data Cache Block 0 Cache TagValid ::: Cache Index Mux 01 Sel1Sel0 Cache Block Compare Adr Tag Compare OR Hit

CS252/Kubiatowicz Lec /22/03 What happens on a Cache miss? For in-order pipeline, 2 options: –Freeze pipeline in Mem stage (popular early on: Sparc, R4000) IF ID EX Mem stall stall stall … stall Mem Wr IF ID EX stall stall stall … stall stall Ex Wr –Use Full/Empty bits in registers + MSHR queue »MSHR = “Miss Status/Handler Registers” (Kroft) Each entry in this queue keeps track of status of outstanding memory requests to one complete memory line. Per cache-line: keep info about memory address. For each word: register (if any) that is waiting for result. Used to “merge” multiple requests to one memory line »New load creates MSHR entry and sets destination register to “Empty”. Load is “released” from pipeline. »Attempt to use register before result returns causes instruction to block in decode stage. »Limited “out-of-order” execution with respect to loads. Popular with in-order superscalar architectures. Out-of-order pipelines already have this functionality built in… (load queues, etc).

CS252/Kubiatowicz Lec /22/03 Review: Cache Performance CPU time = (CPU execution clock cycles + Memory stall clock cycles) x clock cycle time Memory stall clock cycles = (Reads x Read miss rate x Read miss penalty + Writes x Write miss rate x Write miss penalty) Memory stall clock cycles = Memory accesses x Miss rate x Miss penalty Average Memory Access time (AMAT) = Hit Time + (Miss Rate x Miss Penalty) Note: memory hit time is included in execution cycles. Memory Time from view of Memory Stage

CS252/Kubiatowicz Lec /22/03 Impact on Performance Suppose a processor executes at –Clock Rate = 200 MHz (5 ns per cycle), Ideal (no misses) CPI = 1.1 –50% arith/logic, 30% ld/st, 20% control Suppose that 10% of memory operations get 50 cycle miss penalty Suppose that 1% of instructions get same miss penalty CPI = ideal CPI + average stalls per instruction 1.1(cycles/ins) + [ 0.30 (DataMops/ins) x 0.10 (miss/DataMop) x 50 (cycle/miss)] + [ 1 (InstMop/ins) x 0.01 (miss/InstMop) x 50 (cycle/miss)] = ( ) cycle/ins = % of the time the proc is stalled waiting for memory! AMAT=(1/1.3)x[1+0.01x50]+(0.3/1.3)x[1+0.1x50]=2.54

CS252/Kubiatowicz Lec /22/03 Review: Four Questions for Memory Hierarchy Designers Q1: Where can a block be placed in the upper level? (Block placement) –Fully Associative, Set Associative, Direct Mapped Q2: How is a block found if it is in the upper level? (Block identification) –Tag/Block Q3: Which block should be replaced on a miss? (Block replacement) –Random, LRU Q4: What happens on a write? (Write strategy) –Write Back or Write Through (with Write Buffer)

CS252/Kubiatowicz Lec /22/03 CS 252 Administrivia Don’t have test graded (Really sorry!) –May be a bit before this happens. Homework: send it to me –By (say) midnight tonight Proposals: –Send me your tentative proposals via today »Use my normal address –I will try to comment on them very soon –Final version will be due next Wednesday