COMPUTER ARCHITECTURE (P175B125) Assoc.Prof. Stasys Maciulevičius Computer Dept.

©S.Maciulevičius DRAM cell Row (word) line Column (bit) line Storing of one bit in dynamic memory cell needs one transistor only (static memory cell has 6-8 transistors). In order to reduce the number of chip contacts, traditionally address has been transferred in two steps: first are transferred higher bits – row address, later – column address This results in a greater number of cycles in access. Information is stored in the form of load capacitor within an integrated circuit. Since real capacitors leak charge, the information eventually fades unless the capacitor charge is refreshed periodically DRAM works approximately 10 times slower than SRAM

©S.Maciulevičius Structure of 1 M DRAM chip Data D0-D3 OE# WE# Column addr. buffer Refresh counter Row addr. buffer N.1 clock oscillator Column decoder Amplifiers and write control DRAM array (matrix) 512  512  4 Row decoder I/O control and data buffers A0-A8 CAS# Refresh controller RAS# N.2 clock oscillator

©S.Maciulevičius RDRAM, DDR, DDR 2, … DRAM roadmap BEDO 4M 256M 64M 16M 1M Ordinary FPM EDO SDRAM

©S.Maciulevičius Ordinary DRAM Col.2 Row 2Col.1Row 1 RAS# CAS# Addr Data Every access - individual Data1Data 2

©S.Maciulevičius Fast Page Mode (FPM) DRAM RAS# CAS# Addr Data For successive reads or writes within the row CAS# should be repeated When CAS#  H, data output lines  Z state Col. 3 Col. 2Col. 1 Row 1 Data1Data2Data3

©S.Maciulevičius Extended Data Output RAM (EDO RAM) For transferring of burst CAS# should be repeated It differs from FPM with the additional feature that a new access cycle can be started while keeping the data output of the previous cycle active Therefore, it can achieve a smaller period (higher frequency) RAS# CAS# Addr Data Col.3Col.2Col.1Row1 Data1Data2Data3

©S.Maciulevičius Burst Extended Data Output RAM (BEDO) RAS# CAS# Addr Data Col.2 Col.1 Row1 D10D11D12D13D20

©S.Maciulevičius Burst Extended Data Output RAM (BEDO) A pipelined stage was added allowing page- access cycle to be divided into two components An address counter on the chip was added to keep track of the next address Quicker access time is achieved (up to 50% for large blocks of data) than with traditional EDO Could process four memory addresses in one burst, for a maximum of , when EDO RAM

©S.Maciulevičius Synchronous Dynamic RAM (SDRAM) Traditionally DRAM has an asynchronous interface which means that it responds as quickly as possible to changes in control inputs SDRAM has a synchronous interface, meaning that it waits for a clock signal before responding to control inputs and is therefore synchronized with the computer's system bus All of them are designed to work in burst mode, transfering one portion of data each clock. Programmable burst length - 1, 2, 4, 8 or 256 Could process four portions in one burst for a maximum of

©S.Maciulevičius Synchronous Dynamic RAM (SDRAM) DRAM Regist e r Address Control signals Data Clock This is realized by adding registers (latches) to fix the address, data and control signals:

©S.Maciulevičius SDRAM Clock is used to drive an internal finite state machine that pipelines incoming instructions Pipelining means that the chip can accept a new instruction before it has finished processing the previous one. E.g. in a pipelined read, the requested data appears after a fixed number of clock pulses after the read instruction, and additional instructions can be sent during this time For indicating DRAM speed two principles are used: Minimal interval between adjacent portions of the bundle (8 ns, 7 ns, and 6 ns, etc.) Bus frequency (100 MHz corresponds to 8-ns, 133 MHz -to 6-ns, etc.). Don’t forget that the first portion can have significant latency!

©S.Maciulevičius Timing of PC100 SDRAM Row Col RowCol WWWWWWWWWWRRWWRR 2 cycle addressing Bubbles increase latency, decrease bandwidth Data Instr Bubbles

©S.Maciulevičius Timing of PC100 SDRAM This diagram should be drawn with attention to two SDRAM technology-driven issues: In PC platform unbuffered SDRAM DIMMs require the so-called '2-cycles addressing‘ - the row and column addresses on the bus are retained two cycles. This is necessary when several DIMM slots are on board. In the case only 1 DIMM, just 1 cycle is sufficient Changing of address (the selection of other column, by reading, as well as by writting) needs for a small pause ('bubbles')

©S.Maciulevičius Compare classical DRAMs TypeTypeStandard bus speeds, MHz Access rateDRAM access time Ordinary ns FPM ns EDO ns BEDO ns SDRAM ns

©S.Maciulevičius New DRAM types If the above DRAM types may be considered as relatively classic, in past years new types of DRAMs were developed, which were and are used into computers: DDR SDRAM - Double Data Rate SDRAM DDR2 SDRAM – twice faster than DDR DDR3 SDRAM – four times faster than DDR

©S.Maciulevičius DDR SDRAM DDR - Double Data Rate SDRAM - It achieves nearly twice the bandwidth of the preceding single data rate (SDR) SDRAM by transferring data on the rising and falling edges of the clock signal Bandwidth: 1 generation - with a bus frequency of 100 MHz, DDR SDRAM gives a maximum transfer rate of 1600 MB/s later GB/s (= 200  2  8 B; frequency of 200 MHz)

©S.Maciulevičius DDR SDRAM DDR read operations can be explained using this simplified scheme: Data register (n-bit) Data register (n-bit) D0 MUX Q D1 2n bits n bits From memory array

©S.Maciulevičius DDR modules Standard name Mem. clock (MHz) Cycle time (ns) I/O bus clock (MHz) Data transf. rate (MHz) Module name Peak transfer rate (MB/s) DDR PC DDR PC DDR PC DDR PC Some DDR modules are specified here:

©S.Maciulevičius DDR2 DDR2 core performs read and write operations in same frequency, as DDR or SDRAM However : I/O buffers operating frequency is double Twice expanded bus that connects the core and the buffers Therefore the data are multiplexed and transmitted at a double frequency using the normal width bus Thus, DDR2 533 work in the same frequency as DDR266 or PC133 SDRAM

©S.Maciulevičius DDR2 SDRAM read operation Data register (n-bit) Data register (n-bit) D0 D1 D2 MUX Q D1 4n bits n bits From memory array Data register (n-bit) Data register (n-bit) n bits

©S.Maciulevičius DDR2 modules Standard name Mem. clock (MHz) Cycle time (ns) I/O bus clock (MHz) Data transf. per sec (Mln) Module name Peak transfer rate (MB/s) DDR PC DDR PC DDR PC DDR PC DDR PC Some DDR2 modules are specified here:

©S.Maciulevičius DDR and DDR 2 Increased delay in clock periods, but data are transferred faster

©S.Maciulevičius SDRAM, DDR, and DDR 2 As you can see, all the SDRAM parts operate at the basic (core) frequency, while the data is transmitted once a clock DDR parts operate at the basic (core) frequency, while the data is transmitted twice per clock DDR 2 output buffers operate at the double frequency, while the data is transmitted twice per buffers clock (four times per core clock)

©S.Maciulevičius DDR3 Core Data buffer Data output frequency 100 MHz frequency 400 MHz rate 800 MHz Memory core (cell array) Data output buffers

©S.Maciulevičius Benefits of DDR3 First of all – less energy consumption (by 40%) compared to the popular DDR2 (this is due to reduction of supply voltage: 1,5 V - DDR3, 1,8 V - DDR2, or 2,5 V – DDR) The higher working speed - DDR3 frequency range 800 МHz – 1600 МHz (clock frequency 400 МHz – 800 МHz); while the DDR2 frequency range 400 МHz МHz (clock frequency 200 МHz МHz), and DDR – 200 МHz МHz only DDR3 drawback – increased latency (in clock periods)

©S.Maciulevičius DDR3 modules Standard name Mem. clock (MHz) Cycle time (ns) I/O bus clock (MHz) Data transf. per sec (Mln) Module namePeak transfer rate (MB/s) DDR PC DDR PC DDR PC DDR PC Some DDR3 modules are specified here:

©S.Maciulevičius DDR  DDR2  DDR3 (market)

©S.Maciulevičius DDR4 DDR4 is the next evolution in DRAM, bringing even higher performance and more robust control features while improving energy economy Feature/OptionDDR3DDR4 Voltage (core and I/O)1.5V1.2V Data rate (Mb/s) 800, 1066, 1333, 1600, 1866, , 1866, 2133, 2400, 2667, 3200 Densities512Mb–8Gb2Gb–16Gb Internal banks816

©S.Maciulevičius Increasing DRAM speed

©S.Maciulevičius DDR timing Main DDR DRAM timing parameters are: t RCD - RAS to CAS delay – the number of clock cycles needed between a row address strobe and a column address strobe t CL - CAS delay (latency) – the number of clock cycles required to access a specific column of data t RP - RAS precharge – the number of clock cycles needed to close one row of memory and open another t RAS - active to precharge delay – The number of clock cycles needed to access a specific row of data in RAM E.g., “DDR ” shows the values of these four parameters

©S.Maciulevičius DDR timing Typical values of these parameters for DDR chips: RAS to CAS Delay: 2, 3, 4; CAS Latency: 2.0, 2.5, 3.0; RAS Precharge: 2, 3, 4

©S.Maciulevičius SPD In accordance with JEDEC standards in each module must be small special ROM chip called the SPD (Serial Presence Detect) with access information about a computer memory module: configuration and type timing producer (his code) serial number production date other information Total ROM size is 128 bytes

©S.Maciulevičius SPD E.g., CPU-Z test extracts such information from SPD:

©S.Maciulevičius DRAM refresh Memory refresh is the process of periodically reading information from an area of computer memory, and immediately rewriting the read information to the same area with no modifications Each memory refresh cycle refreshes a succeeding area of memory Classic asynchronous DRAM is refreshed by opening each row in turn For convenience, the refresh counter is incorporated into RAM chips

©S.Maciulevičius DRAM refresh In CAS-before-RAS (CBR) refresh the CAS# line is driven low before RAS#, then the DRAM ignores the address inputs and uses an internal counter to select the row to open (refresh) Hidden refresh allows PC RAM refresh memory cycles to take place in memory banks not used by the CPU at the time, instead or together with the normal refresh cycles Refresh period – T ref in first DRAMs was 2 ms, now – 64 ms or even 128 ms

©S.Maciulevičius Memory controller CPU D A Rd Wr D A RAS# CAS# WE# OE# DRAM control- ler DRAM The memory controller is a digital circuit which manages the flow of data going to and from the main memory:

©S.Maciulevičius Memory controller  It can be a separate chip or integrated into another chip  Computers using Intel microprocessors traditionally had a memory controller implemented on their motherboard's northbridge (“northern” part of chipset)  AMD's Athlon 64 and Opteron processors, Intel Core i7 have a memory controller on the microprocessor die to reduce the memory latency. This also adds some restrictions for using some DRAM types

©S.Maciulevičius Memory controller in chipset  Computers using Intel Core 2 (Duo and Quad) microprocessors had a memory controller implemented on their motherboard's northbridge ( e.g., on P45 MCH - Memory Controllel Hub ):

©S.Maciulevičius Memory controller in Core i7 Integrated Memory Controller

©S.Maciulevičius DRAM modules SIPP – Single In-Line Pin Package 30 pins used in some 286-based computers often bent or broke during installation SIMM – Single In-Line Memory Module “short” (90 mm) – 30 pins, 8 bits of data “long” (108 mm) – 72 pins, 4 bytes of data 32, 36 (with parity), ECC-36 and ECC-40 – with an error- correcting code some - with PD (Presence Detect, indicates size 4, 8, 16, 32 MB) DIMM – Dual In-Line Memory Module 133,35 mm – pins, 8 bytes 64 (ordinary) bit word, 72 or 80 bits (with parity or error- correcting code)

©S.Maciulevičius SIMM modules

©S.Maciulevičius SDRAM module

DDR modules ©S.Maciulevičius Comparison of memory modules for desktop PCs (DIMM)

Registered memory modules  Registered (also called buffered) memory modules have a register between the DRAM modules and the system's memory controller  They place less electrical load on the memory controller and allow single systems to remain stable with more memory modules than they would have otherwise  There is a performance penalty for using registered memory. Each read or write is buffered for one cycle between the memory bus and the DRAM, so the registered RAM can be thought of as running one clock cycle behind the equivalent unregistered DRAM ©S.Maciulevičius

Registered memory modules ©S.Maciulevičius

FB-DIMM  Fully Buffered DIMM (or FB-DIMM) is a memory technology which can be used to increase reliability and density of memory systems  Conventionally, data lines from the memory controller have to be connected to data lines in every DRAM module  Fully buffered DIMM architecture introduces an advanced memory buffer (AMB) between the memory controller and the memory module ©S.Maciulevičius

FB-DIMM ©S.Maciulevičius

FB-DIMM  FB-DIMM uses 10 pairs of lines carrying commands and data from the processor to memory and 14 bit lanes carrying data from memory to the processor  Each bit is carried over a differential pair (signal and inversion), clocked at 12 times the basic memory clock rate, 6 times the double-pumped data rate ©S.Maciulevičius

FB-DIMM  While Fully-Buffered DIMM was originally a good idea, the industry soon found that it has implementation problems  First, the serial input frequency has to be 4 times higher than the memory clock frequency. This puts it into the microwave frequency range and is a whole new page of technical difficulties  The higher serial input frequency also increases the heat generation to an unacceptable point.  Smart engineers soon announced the alternative approach, the LRDIMM ©S.Maciulevičius

LRDIMM  LRDIMM (Load Reduced Dual-inline Memory Module) is designed with a buffer chip to replace the register to help minimize loading, it can increase overall server system memory capacity and speed  It is pin-compatible with existing DDR3 DIMM sockets and LRDIMM is JEDEC standard  LRDIMM can contain 72 modern 40nm 4 gigabit DDR3 SDRAM  Dual server can have at most 16 ordinary DIMMs, but using LRDIMM – even 24 DIMMs ©S.Maciulevičius

LRDIMM and FBDIMM ©S.Maciulevičius