1. Bandwidth Rocks (1) Latency Lags Bandwidth (last ~20 years) Performance Milestones Disk: 3600, 5400, 7200, 10000, 15000 RPM

2. Bandwidth Rocks (2) Latency Lags Bandwidth (last ~20 years) Performance Milestones Memory Module: 16bit plain DRAM, 16b Page Mode DRAM, 32b, 64b, 64b SDRAM, 64b DDR SDRAM Disk: 3600, 5400, 7200, 10000, 15000 RPM

3. Bandwidth Rocks (3) Latency Lags Bandwidth (last ~20 years) Performance Milestones Ethernet: 10Mb, 100Mb, 1000Mb, 10000 Mbit/sec Memory Module: 16bit plain DRAM, 16b Page Mode DRAM, 32b, 64b, 64b SDRAM, 64b DDR SDRAM Disk: 3600, 5400, 7200, 10000, 15000 RPM

4. Bandwidth Rocks (4) Latency Lags Bandwidth (last ~20 years) Performance Milestones Processor: ‘286, ‘386, ‘486, Pentium, Pentium Pro, Pentium 4 Ethernet: 10Mb, 100Mb, 1000Mb, 10000 Mbit/sec Memory Module: 16bit plain DRAM, 16b Page Mode DRAM, 32b, 64b, 64b SDRAM, 64b DDR SDRAM Disk: 3600, 5400, 7200, 10000, 15000 RPM

5. Bandwidth Rocks (5) Rule of Thumb for Latency Lagging BW In the time that bandwidth doubles, latency improves only by factors of 1.2 to 1.4 (and capacity approximately triples in the time that bandwidth doubles)

6. Bandwidth Rocks (6) Improvement in Time for BW to Double CPUMemoryLANDisk Latency / Year 1.171.071.101.09 Capacity / Year --1.52--1.28 Bandwidth / Year 1.501.271.391.18 Capacity Improvement -- 3.4 -- 2.8 Years for 2X Bandwidth1.72.92.14.2 Latency Improvement1.31.2 1.4

7. Bandwidth Rocks (7) Recent Improvement in Time for BW to Double CPUMemoryLANDisk Latency / Year 1.221.061.131.09 Capacity / Year --1.49--1.37 Bandwidth / Year 1.551.301.781.24 Capacity Improvement -- 2.9 -- 2.8 Years for 2X Bandwidth1.62.71.23.2 Latency Improvement1.41.2 1.3 (Last 3 Performance Milestones)

8. Bandwidth Rocks (8) 1.Latency helps BW, but not vice versa Spinning disk faster improves both bandwidth and rotational latency Higher linear density helps BW, not Latency 2.Bandwidth hurts latency Longer queues help BW, hurt Latency 3.Distance limits latency Speed of light and computers on network Size of DRAM block => long bit and word lines => most of DRAM access time 6 Reasons Latency Lags Bandwidth

9. Bandwidth Rocks (9) 6 Reasons Latency Lags Bandwidth 4.Moore’s Law helps BW more than latency Smaller, faster transistors communicate over relatively longer lines: limits latency Faster transistors, more transistors, more pins all help BW 5.Software overhead hurts latency Long messages amortize overhead; overhead more for short messages 6.Bandwidth is easier to sell E.g., 10 Gbit/s vs. 10  sec latency Ethernet

10. Bandwidth Rocks (10) 4 Ways to Cope with Latency Lags Bandwidth 1.Caching (Leveraging Capacity) Processor caches, file cache, disk cache 2.Replication (Leveraging Capacity) Read from nearest head in RAID, Distributed caches in SMP, Distributed ISPs 3.Parallelism and Out-of-Order Processing (Leveraging Bandwidth) Processors, Disk controllers, Messages 4.Prediction (Leveraging Bandwidth) Branches, Prefetching: disk, caches

11. Bandwidth Rocks (11) Conclusion If everything improves at the same rate, then nothing really changes –When rates vary, we see real dislocation that in turn requires real innovation Hardware and software developers should plan that performance advances will emphasize BW over latency