Case Study - SRAM & Caches By Nikhil Suryanarayanan

Outline Introduction SRAM Cell Peripheral Circuitry Practical Cache Designs Technology scaling & Power Reduction Summary

Cache Sizes in recent Processors Core 2 Duo 6M Atom 512K Core i7 Extreme 8M Core i7 Extreme Mobile Core 2 Quad 12M Itanium 2 24M Xeon 16M

SRAM Cell Basic building block of on-chip cache Various design 12T, 10T, 8T, 6T, 4T 6T is the most widely used by industry giants in contemporary microprocessors Data bit and its complement and stored in a cross-coupled inverter Simple in design

A 6T SRAM Cell Cross-coupled inverter Access Transistors Access transistors good at passing zeros

Column Circuitry Bitline Conditioning Sense Amplifiers Multiplexer (column decoder) Three most major column circuitry components

Bitline Conditioning Precharge high before reads Equalize bitline delays to minimize voltage difference…. Why? The second circuits provides the added functionality of equalizing the bitlines during precharge

Sense Amplifier Bitlines have many cells attached; enormous capacitive loads Bitlines swing slowly Voltages equalized during precharge SA detects small swing as bitlines are driven apart and bring it up to normal logic Reduced delay by not waiting for full-swing SAs are commonly used to magnify small differential input voltages into larger output voltages They are commonly used in memories in which differential bitlines have enormous load capacitances, Because of the large load the, the bitlines swing slowly To reduce this delay, bitline voltages are equalized first and a small swing can bring it upto normal logic

Types of SA Differential Pair requires no clock but always dissipates static power Clocked sense amp saves power and also isolates the large bitline capacitances Improved gain offered by Cross Coupled Amplifier Different versions of the second sense amplifier are used currently, the basic idea of operation is the same

Read Circuit General Diagram representing a read circuit

Write Circuit Diagram representing a write circuit

Drive strengths Read Stability Writeability NMOS pull down – Strongest Access NMOS – intermediate PMOS pull up - weakest When BL bar is zero, World line is raised, BL_bar should be pulled down through M5 and M1, At same time Q_bar tends to rise, because of the current flowing in from M5, but it should be pulled/kept low by M1 Hence M1 must be stronger than M5 Assume Q_bar is low, and we wish to write 1. Because of read stability constraints, 1 cannot be written through M1, hence write 0 through M6, M4 opposes this operation, hence M4 must be weaker than M6

Industry Designs Novel Design Techniques Embedded Dual 512 KB L2$ in Ultrasparc (2004) L1$, L2$ & L3$ - Designs in two Itanium Generations (2002-2003) Effects of Technology scaling Overview of current Design in 45nm

Design Considerations Larger on-chip cache running at processor frequency improves performance Latency also increases as size grows Memory cell performance does not improve rapidly with new generation Change of design focus with the cache level Memory cell performance does not improve as logic transistor performance Higher wire delays and additional logic for soft error protection are introduced Hence it is important to optimize designs for minimum hit latency, since a large fraction of data and program can reside within the larger L2$, Extra design effort must be taken to meet aggressive clock cycle times and also to convert off-chip memory to on-chip memory Develop optimal solutions for architectural features as well as design methodology that can balance complexity with performance and efficient use of silicon area 1st level optimized for latency 2nd level optimized for b/w 3rd level optimized for size

Itanium 2 L1 cache 16KB 4-ways I & D caches Feature – Eliminates read stall during cache updates Focus – Circuit Technique

L1$ Circuit technique L1$ circuit technique to eliminate stalls during cache update without incurring area penalty of an extra read port. The two single ended ports are shared between read and write. Write needs dual-ended access to the array on the same cycle, since only a discharge of bitline can effectively flip the memory cell state. Here, writes take two cycles with o being written in the first cycle and 1 being written in the subsequent cycle (control logic allows writes to start with either 0 or 1)

UltraSparc(2004) Dual 512 KB L2$ 4-way set-associative design 64 byte line size, 128 bit data bus, 4 cycle to fill a line 8 bit ECC for a 64 bit datum Conversion from off-chip direct mapped cache to on-chip 4-way set-associative cache /******************************************************************** Pseudo random replacement algorithm Almost similar to pseudo LRU approach */ 4 tags

UtraSparc L2$ Since data occupies most of silicon area, is goal was process tolerance & area efficiency Tag array was designed for tight timing considerations Conversion from off-chip direct-mapped to on-chip cache Feature: Read tags for next process during current process Focus : Use of differential access times to tag and data array & Layout Design optimization requires which of the four ways to write during stores

Read Access Pipeline L2$ is pipelined and can handle multiple outstanding stores A special case arises from the need to support the simultaneous ability of store buffer to read the tags and to store other data Tags are re-read dynamically to determine the way to write This requires tag lookup and store support in the same cycle, ideally needing a dual port array This can be achieved by making single cycle access and dual o/p latches 1 cycle for tag and data to reach the farthest end of the L2$ if needed They are latched in the +ve edge of the subsequent cycle Data takes 2 cycles Tag lookup is completed in one cycle and next tag lookup begins in the next cycle Scheme helps meet tight timing requirement by eliminating clock and flop overheads like clock skew, setup time, and clk2q delays Smaller number of flop elements and clock buffers needed by the half-frequency reduced overall power consumption and area compared to single cycle design

Tag array I/O circuit diagram This technique eliminates the need for duplicating word line decoder and I/O peripherals in the 32KB size of tag arrays Clocked self-resetting dynamic circuit with a pseudo-current mode sense amplifier is used to achieve faster access time and higher tag efficiency. The current sense amplifier demonstrates a 15% speed improvement compared to a conventional cross-coupled latch type sense amplifier under same bitline loading conditions P3 and P4 are both tied to bitlines. Column select signals are generated at about 40% of Vdd level. This puts transistors P1 and P2 in saturation mode to act as low impedance devices for current-mode sensing At the beginning eq_l goes high and equalizes node sa and sa_l. Then eq_l turns off when enough differential current is established from N1 and N2 As one of the sense amplifier’s internal nodes goes above the trip point of nand gate, the corresponding PMOS turns on and speeds up the transition to VDD

Data organization 256 rows* 256 columns(8 KB unit) to compose a 128KB unit by tiling 16 8KB banks This 128KB unit is used as a building block for the 512KB L2$ Integrating the SRAM arrays and logic units as one single subsystem achieves both a short latency and an efficient use of silicon area and power

Itanium 3MB L3$ 180nm process 12 way set-associative Fits in an irregular structure Feature : Size of cache Focus : Regular & efficient partition in an irregular space This is for each subarray 96 words with 256 bitlines*8 groups -> 96*256*8=24KB The memory cells in each sub-array are further divided into eight groups with a local sense amplifiers and driver circuitry 8 global column selects, 8 group clocks that activate one of the eight groups

Itanium 3 MB L3$

Sensing Unit During write, the write data pulls down the selected bitline by enabling a pull down chain, while holders hold the bitline complement high

Itanium 2 6MB L3$ 24-way set-associative 130nm process 64 bit EPIC architecture processor Double the number of transistors compared to the previous generation Same power dissipation as previous generation......hmmm 1.5x freq, 2x L3 cache, 3.5x leakage This architecture is inherited from its 180nm predecessor The focus was shrinking the complex circuit topologies to 130nm process, pushing the frequency envelope while maintaining strict control over the power

Technology Scaling Current Model Previous Model The current model has a 50% higher frequency , a 2x L3$ cache and 3.5X leakage current In order to stay within the same power envelope, the active power had to be reduced from 90% to 74%. This was accomplished through an aggressive management of the dynamic power Reduced clock loading Lower power contention Better L3$ power management Current Model Previous Model

L3 Power Reduction Scheme Reduce the number of active arrays to access a cache line

Itanium

Core 2 Duo

Power Reduction techniques used in Silverthorne Control registers disable a pair of ways(out of 8) during low performance operation A way is entirely flushed before disablement During Deep Power Down, the entire voltage plane to L2 is cut off General Cache Architecture has remained the same

Questions?

References CMOS VLSI Design, Weste & Harris 3e On-Chip 3MB Subarray based 3rd level Cache on Itanium Microprocessor - Don Weiss, John J Wuu, Victor Chin Design and Implementation of an Embedded 512KB Level-2 Cache Subsystem, Shin, Petrick, Singh, Leon A 1.5 GHz 130nm Itanium 2 Processor with 6-MB On-die L3 Cache

Need for Caches Large Access Delays of off-chip memory Fast on-chip Memory is expensive Hierarchical Memory System is a solution Bring limited amount of data on-chip and reduce latencies Increase processor performance Large Access Delays to read and write stuff Best solution, make everything fast – not possible Fast memory is expensive and everything cannot be brought on-chip Follow a hierarchy which will provide cost/byte almost as low as cheapest level of memory and speed as fast as the fastest level Memory is a known bottleneck, so improving the speed ad capacity of caches can directly improve the processor performance

Data Array read & write critical signals The high density memory cell selected allows an area-efficient design at the expense of higher delay to develop a bit-line differential signal, due to lower memory cell current

Write Circuit