Serial Network SDRAM ENEE 759H Spring 2003

Introduction SDRAM system drawbacks  No parallelism for memory accesses  Multitude of pins for address/command/data Overall Goals  Increase parallelism, reduce latency  Reduce pin count  Attempt to increase bandwidth

Motivation Poulton’s idea  Bi-directional serial links.  Theoretically high bandwidth!  Less pins required for same functionality!  Looks perfect! *Graphic from Poulton’s Signaling Tutorial

Evolution I Initial design  Split topology.  Effectively halve latency.  Complicated protocol and connection details.

Evolution II Initial design  Individual DRAM chips directly connected.  High overall bandwidth.  Inflexible, lower capacity for system. We need a better design!

The Next Step Want simple system interconnects Keep basic SDRAM chip structure intact Utilize the strengths of both parallel and serial connections Create a system that facilitates parallelism

System Overview Take a “step back”… Consider memory module interface. Consider inter-chip interface on module.

System Overview 1 logical channel, 4 physical channels 3.2 GHz point-to-point connections Each channel called “module” 5 pins/module on memory controller Intra-module connections: parallel External connections: high speed serial

Module Topology

System Details I

System Details II

System Details – Protocol I The Command Set CMDUSEOPADDR? NOPNo operation.000N ACTActivate a row; uses bank and row address.001Y READSelects bank/column, initiates read burst.010Y WRITESelect bank and column, initiate write burst.011Y PRECPrecharge; deactivate row in bank.100* AUTORAuto-refresh; enter refresh mode.101N XXXReserved110 XXXReserved111

System Details – Protocol II Packets  18 bit command/address  32 bit data packets COMMANDActivate this row and bank… COMMANDStart a READ burst at this column… *Operating at 3.2GHz, command packets take 5.62ns; data packets take 10ns (the same as SDRAM operating at 100 MHz).

Cubing I “Chip stacking” Developed by Irvine- Sensors Corp. Currently can stack two 256 Mbit chips. Smaller footprint/area! Much shorter connection wires! *Graphics from Irvine-Sensors Data Sheet

Cubing II – Serial Network Point-to-point star topology. Dedicated circuits - high speed serial lines. Departure from “traditional” bus concept.

System Access Protocol Consecutive access to same module  Similar timing as SDRAM. Bandwidth matched between parallel and serial. DIN/DOUT buffers - no additional timing constraints. *Graphic from Dr. Jacob and Dave Wang

System Access Protocol Independent, simultaneous access to separate modules.  No inter-module timing issues. *Graphic from Dr. Jacob and Dave Wang Conventional SDRAM:

Serial Network Advantages I Path length matching  No more heroic routing!  Star topology is symmetric. No clock mismatch issues…  Everyone is on time! *Graphic from Dr. Jacob and Dave Wang

Serial Network Advantages IIa No need for bus termination.  Point-to-point communication, terminated in module. *Graphic from Dr. Jacob and Dave Wang

Serial Network Advantages IIb Serial/P2P vs. RAMBUS multi-drop.  Faster signaling!  No ringing!  Clean timing.  Serial wins… RAMBUSted! *Graphic from Dr. Jacob and Dave Wang

System Simulation SimpleScalar  Single CPU, Single Thread  SNSDRAM(32 Meg x 8)  1 rank in every memory module  Channel width : 32 bits  One extra cycle of Transaction Queue Delay to model the parallel to serial conversion

Simulation Run I - Parallel Bus Channel Rank Per Channel Sim_Cycles

Simulation Run I - Serial Network Channel Rank Per Channel Sim_Cycles

Simulation I Cycles Chart

Simulation Run II – Parallel Bus Channel Rank Per Channel Sim_Cycles

Simulation Run II – Serial Network Channel Rank Per Channel Sim_Cycles

Simulation II Cycles Chart

Memory Mapping Basic SDRAM High Performance SDRAM Row IDRankBankHi Col IDChannel IDLo Col IDCol Size RankRow IDBankHi Col IDChannel IDLo Col IDCol Size

Analysis Cache line = 64 byte channel width Read after Read Multi-CPU Single CPU Multi-Thread

Summary I Recall…  SDRAM has complex interface, simple chips.  RDRAM has a simple interface, but very complex chips. SNSDRAM…  Blends these seemingly split philosophies!

Summary II Advantages  Smaller pin count on memory controller.  Independent memory modules facilitate parallelism.  Simulated performance improvement over similar SDRAM configurations.  Smaller system footprint with cubing technology.  Theoretically scalable.