1. Serial Network SDRAM ENEE 759H Spring 2003

2. Introduction SDRAM system drawbacks  No parallelism for memory accesses  Multitude of pins for address/command/data Overall Goals  Increase parallelism, reduce latency  Reduce pin count  Attempt to increase bandwidth

3. Motivation Poulton’s idea  Bi-directional serial links.  Theoretically high bandwidth!  Less pins required for same functionality!  Looks perfect! *Graphic from Poulton’s Signaling Tutorial

4. Evolution I Initial design  Split topology.  Effectively halve latency.  Complicated protocol and connection details.

5. Evolution II Initial design  Individual DRAM chips directly connected.  High overall bandwidth.  Inflexible, lower capacity for system. We need a better design!

6. The Next Step Want simple system interconnects Keep basic SDRAM chip structure intact Utilize the strengths of both parallel and serial connections Create a system that facilitates parallelism

7. System Overview Take a “step back”… Consider memory module interface. Consider inter-chip interface on module.

8. System Overview 1 logical channel, 4 physical channels 3.2 GHz point-to-point connections Each channel called “module” 5 pins/module on memory controller Intra-module connections: parallel External connections: high speed serial

9. Module Topology

10. System Details I

11. System Details II

12. System Details – Protocol I The Command Set CMDUSEOPADDR? NOPNo operation.000N ACTActivate a row; uses bank and row address.001Y READSelects bank/column, initiates read burst.010Y WRITESelect bank and column, initiate write burst.011Y PRECPrecharge; deactivate row in bank.100* AUTORAuto-refresh; enter refresh mode.101N XXXReserved110 XXXReserved111

13. System Details – Protocol II Packets  18 bit command/address  32 bit data packets COMMANDActivate this row and bank… 001011111001001111 COMMANDStart a READ burst at this column… 010011011001001100 *Operating at 3.2GHz, command packets take 5.62ns; data packets take 10ns (the same as SDRAM operating at 100 MHz).

14. Cubing I “Chip stacking” Developed by Irvine- Sensors Corp. Currently can stack two 256 Mbit chips. Smaller footprint/area! Much shorter connection wires! *Graphics from Irvine-Sensors Data Sheet

15. Cubing II – Serial Network Point-to-point star topology. Dedicated circuits - high speed serial lines. Departure from “traditional” bus concept.

16. System Access Protocol Consecutive access to same module  Similar timing as SDRAM. Bandwidth matched between parallel and serial. DIN/DOUT buffers - no additional timing constraints. *Graphic from Dr. Jacob and Dave Wang

17. System Access Protocol Independent, simultaneous access to separate modules.  No inter-module timing issues. *Graphic from Dr. Jacob and Dave Wang Conventional SDRAM:

18. Serial Network Advantages I Path length matching  No more heroic routing!  Star topology is symmetric. No clock mismatch issues…  Everyone is on time! *Graphic from Dr. Jacob and Dave Wang

19. Serial Network Advantages IIa No need for bus termination.  Point-to-point communication, terminated in module. *Graphic from Dr. Jacob and Dave Wang

20. Serial Network Advantages IIb Serial/P2P vs. RAMBUS multi-drop.  Faster signaling!  No ringing!  Clean timing.  Serial wins… RAMBUSted! *Graphic from Dr. Jacob and Dave Wang

21. System Simulation SimpleScalar  Single CPU, Single Thread  SNSDRAM(32 Meg x 8)  1 rank in every memory module  Channel width : 32 bits  One extra cycle of Transaction Queue Delay to model the parallel to serial conversion

22. Simulation Run I - Parallel Bus Channel Rank Per Channel Sim_Cycles 1 1 884521 1 2 881421 1 4 880361 1 8 880361

23. Simulation Run I - Serial Network Channel Rank Per Channel Sim_Cycles 1 1 885291 2 1 805721 4 1 766711 8 1 766711

24. Simulation I Cycles Chart

25. Simulation Run II – Parallel Bus Channel Rank Per Channel Sim_Cycles 1 1 13206613 2 1 13169500 4 1 13144737 8 1 13144737

26. Simulation Run II – Serial Network Channel Rank Per Channel Sim_Cycles 1 1 13264603 2 1 12633349 4 1 12510912 8 1 12510912

27. Simulation II Cycles Chart

28. Memory Mapping Basic SDRAM High Performance SDRAM Row IDRankBankHi Col IDChannel IDLo Col IDCol Size RankRow IDBankHi Col IDChannel IDLo Col IDCol Size

29. Analysis Cache line = 64 byte channel width Read after Read Multi-CPU Single CPU Multi-Thread

30. Summary I Recall…  SDRAM has complex interface, simple chips.  RDRAM has a simple interface, but very complex chips. SNSDRAM…  Blends these seemingly split philosophies!

31. Summary II Advantages  Smaller pin count on memory controller.  Independent memory modules facilitate parallelism.  Simulated performance improvement over similar SDRAM configurations.  Smaller system footprint with cubing technology.  Theoretically scalable.