The Main Memory system: DRAM organization Nayan Deshmukh

The Main Memory system DRAM organization DRAM cell operation Outline The Main Memory system DRAM organization DRAM cell operation Address Mapping

The Main Memory system Main memory is a critical component of all computing systems: server, mobile, embedded, desktop, sensor Processors, caches and Memory controller Main Memory Storage (SSD/HDD)

The Main Memory system DRAM organization DRAM cell operation Outline The Main Memory system DRAM organization DRAM cell operation Address Mapping

DRAM organisation Channel DIMM Rank Chip Bank Row/Column Cell

DRAM organisation Channel DIMM Rank Chip Bank Share the same physical link (addr, cmd, data bus) from the memory controller Row/Column Cell Source: Google Images

DRAM organisation DIMM (Dual inline memory module) Channel DIMM corresponds to our familiar notion of RAM Source: Google Images

Rank is a collection of DRAM chips/devices that work in unison DRAM organisation Side View DIMM Back Front Rank 0 Rank 1 Rank is a collection of DRAM chips/devices that work in unison Source: Google Images

Rank is a collection of DRAM chips/devices that work in unison DRAM organisation Side View DIMM Back Front Rank 0 Rank 1 Rank is a collection of DRAM chips/devices that work in unison Source: Google Images

DRAM organisation 8b 64b All the chips share the cmd and addr bus, but have different data buses Source: Google Images

DRAM organisation Chip: collection of banks Each bank functions independently Bank can run in parallel Each bank takes row/column addr and outputs 8b Bank 0 Bank 1 Bank 3 Bank 2 64b Source: Google Images

DRAM organisation Chip: collection of banks Each bank functions independently Bank can run in parallel Each bank takes row/column addr and outputs 8b Bank 0 Bank 1 Bank 2 Bank 3 Source: Google Images

DRAM organisation ... Bank 0 8 DRAM arrays DRAM Array <0:7> Source: Computer Architecture Lectures, ETH Zurich, Fall 2017.

DRAM organisation … 2D Array of DRAM Cells Sense amplifiers The horizontal wires called wordline The vertical wires are called bitline Each cell stores single bit of data All the sense amplifiers collectively called Row Buffer Source: https://people.inf.ethz.ch/omutlu/pub/ambit-bulk-bitwise-dram_micro17-talk.pdf.

Transferring a cache block Physical memory space 0xFFFF…F Channel 0 ... DIMM 0 Mapped to 0x40 Rank 0 64B cache block 0x00 Source: Computer Architecture Lectures, ETH Zurich, Fall 2017.

Transferring a cache block Physical memory space Chip 0 Chip 1 Chip 7 Rank 0 0xFFFF…F . . . ... <0:7> <8:15> <56:63> 0x40 64B cache block Data <0:63> 0x00 Source: Computer Architecture Lectures, ETH Zurich, Fall 2017.

Transferring a cache block Physical memory space Chip 0 Chip 1 Chip 7 Rank 0 0xFFFF…F . . . Row 0 Col 0 ... <0:7> <8:15> <56:63> 0x40 64B cache block Data <0:63> 0x00 Source: Computer Architecture Lectures, ETH Zurich, Fall 2017.

Transferring a cache block Physical memory space Chip 0 Chip 1 Chip 7 Rank 0 0xFFFF…F . . . Row 0 Col 0 ... <0:7> <8:15> <56:63> 0x40 64B cache block Data <0:63> 8B 0x00 8B Source: Computer Architecture Lectures, ETH Zurich, Fall 2017.

Transferring a cache block Physical memory space Chip 0 Chip 1 Chip 7 Rank 0 0xFFFF…F . . . Row 0 Col 1 ... <0:7> <8:15> <56:63> 0x40 64B cache block Data <0:63> 8B 0x00 Source: Computer Architecture Lectures, ETH Zurich, Fall 2017.

Transferring a cache block Physical memory space Chip 0 Chip 1 Chip 7 Rank 0 0xFFFF…F . . . Row 0 Col 1 ... <0:7> <8:15> <56:63> 0x40 64B cache block 8B Data <0:63> 8B 0x00 8B Source: Computer Architecture Lectures, ETH Zurich, Fall 2017.

Transferring a cache block Physical memory space Chip 0 Chip 1 Chip 7 Rank 0 0xFFFF…F . . . Row 0 Col 1 ... <0:7> <8:15> <56:63> 0x40 64B cache block 8B Data <0:63> 8B 0x00 A 64B cache block takes 8 I/O cycles to transfer. During the process, 8 columns are read sequentially. Source: Computer Architecture Lectures, ETH Zurich, Fall 2017.

The Main Memory system DRAM organization DRAM cell operation Outline The Main Memory system DRAM organization DRAM cell operation Address Mapping Now I’ll introduce the motivation and the key idea of our work.

DRAM cell operation Bitline wordline DRAM Cell Bitline Sense Amplifier DRAM Cell Bitline Sense Amplifier (Row Buffer)

DRAM cell operation VDD VDD/2 DRAM Cell Sense Amplifier (Row Buffer) DRAM Cell Sense Amplifier (Row Buffer) VDD/2 Source: https://people.inf.ethz.ch/omutlu/pub/rowclone_seshadri_micro13-talk.pdf.

Amplify the difference DRAM cell operation VDD/2 + δ VDD VDD/2 VDD VDD/2 + δ Amplify the difference DRAM Cell Cell loses charge Restore Cell Data READ/WRITE Sense Amplifier (Row Buffer) VDD/2 ACTIVATE PRECHARGE Source: https://people.inf.ethz.ch/omutlu/pub/rowclone_seshadri_micro13-talk.pdf.

DRAM cell operation Chip I/O Memory Channel Chip I/O ACTIVATE: Copy data from row to row buffer READ: Transfer data to channel using the shared bus ROW HIT: If column in the same row PRECHARGE: Ready the row buffer for next activate Source: https://people.inf.ethz.ch/omutlu/pub/rowclone_seshadri_micro13-talk.pdf.

The Main Memory system DRAM organization DRAM cell operation Outline The Main Memory system DRAM organization DRAM cell operation Address Mapping Now I’ll introduce the motivation and the key idea of our work.

Cache block offset (6 bits) Address Mapping Single-channel system with 8-byte memory bus 2GB memory, 8 banks, 16K rows & 2K columns per bank Byte in bus (3 bits) Cache block offset (6 bits)

Cache block offset (6 bits) Address Mapping Row Interleaving Single-channel system with 8-byte memory bus 2GB memory, 8 banks, 16K rows & 2K columns per bank Row (14 bits) Bank (3 bits) Column (11 bits) Byte in bus (3 bits) Cache block offset (6 bits)

Cacheline Interleaving Row Interleaving Cacheline Interleaving Single-channel system with 8-byte memory bus 2GB memory, 8 banks, 16K rows & 2K columns per bank Row (14 bits) High Column Bank (3 bits) Column (11 bits) Bank (3 bits) Low Col. Byte in bus (3 bits) 8 bits 3 bits

Acknowledgements https://people.inf.ethz.ch/omutlu/pub/rowclone_micro13.pdf https://people.inf.ethz.ch/omutlu/pub/rowclone_seshadri_micro13-talk.pdf https://safari.ethz.ch/architecture/fall2017/doku.php?id=schedule https://people.inf.ethz.ch/omutlu/pub/kim_isca12_talk.pptx. https://people.inf.ethz.ch/omutlu/pub/ambit-bulk-bitwise-dram_micro17-talk.pdf.