Bandwidth Rocks (1) Latency Lags Bandwidth (last ~20 years) Performance Milestones Disk: 3600, 5400, 7200, 10000, RPM

Bandwidth Rocks (2) Latency Lags Bandwidth (last ~20 years) Performance Milestones Memory Module: 16bit plain DRAM, 16b Page Mode DRAM, 32b, 64b, 64b SDRAM, 64b DDR SDRAM Disk: 3600, 5400, 7200, 10000, RPM

Bandwidth Rocks (3) Latency Lags Bandwidth (last ~20 years) Performance Milestones Ethernet: 10Mb, 100Mb, 1000Mb, Mbit/sec Memory Module: 16bit plain DRAM, 16b Page Mode DRAM, 32b, 64b, 64b SDRAM, 64b DDR SDRAM Disk: 3600, 5400, 7200, 10000, RPM

Bandwidth Rocks (4) Latency Lags Bandwidth (last ~20 years) Performance Milestones Processor: ‘286, ‘386, ‘486, Pentium, Pentium Pro, Pentium 4 Ethernet: 10Mb, 100Mb, 1000Mb, Mbit/sec Memory Module: 16bit plain DRAM, 16b Page Mode DRAM, 32b, 64b, 64b SDRAM, 64b DDR SDRAM Disk: 3600, 5400, 7200, 10000, RPM

Bandwidth Rocks (5) Rule of Thumb for Latency Lagging BW In the time that bandwidth doubles, latency improves only by factors of 1.2 to 1.4 (and capacity approximately triples in the time that bandwidth doubles)

Bandwidth Rocks (6) Improvement in Time for BW to Double CPUMemoryLANDisk Latency / Year Capacity / Year Bandwidth / Year Capacity Improvement Years for 2X Bandwidth Latency Improvement

Bandwidth Rocks (7) Recent Improvement in Time for BW to Double CPUMemoryLANDisk Latency / Year Capacity / Year Bandwidth / Year Capacity Improvement Years for 2X Bandwidth Latency Improvement (Last 3 Performance Milestones)

Bandwidth Rocks (8) 1.Latency helps BW, but not vice versa Spinning disk faster improves both bandwidth and rotational latency Higher linear density helps BW, not Latency 2.Bandwidth hurts latency Longer queues help BW, hurt Latency 3.Distance limits latency Speed of light and computers on network Size of DRAM block => long bit and word lines => most of DRAM access time 6 Reasons Latency Lags Bandwidth

Bandwidth Rocks (9) 6 Reasons Latency Lags Bandwidth 4.Moore’s Law helps BW more than latency Smaller, faster transistors communicate over relatively longer lines: limits latency Faster transistors, more transistors, more pins all help BW 5.Software overhead hurts latency Long messages amortize overhead; overhead more for short messages 6.Bandwidth is easier to sell E.g., 10 Gbit/s vs. 10  sec latency Ethernet

Bandwidth Rocks (10) 4 Ways to Cope with Latency Lags Bandwidth 1.Caching (Leveraging Capacity) Processor caches, file cache, disk cache 2.Replication (Leveraging Capacity) Read from nearest head in RAID, Distributed caches in SMP, Distributed ISPs 3.Parallelism and Out-of-Order Processing (Leveraging Bandwidth) Processors, Disk controllers, Messages 4.Prediction (Leveraging Bandwidth) Branches, Prefetching: disk, caches

Bandwidth Rocks (11) Conclusion If everything improves at the same rate, then nothing really changes –When rates vary, we see real dislocation that in turn requires real innovation Hardware and software developers should plan that performance advances will emphasize BW over latency