1. Effects of Storage Trends on Databases
May 27, 2016

2. The Humble Disk Drive
IBM introduced the disk drive in 1956.

3. The Humble Disk Drive
A relatively simple landscape for 50+ years.

4. Media Landscape 1956-2006 DRAM Capacity DISK $$$ TAPE PerformanceFC
SCSI
SATA
TAPE
Performance
Access Latency

5. Media Landscape
Today
FLASH
Capacity

6. The Memory Hierarchy Application Oracle SAP HANA
Updates (read/write/lseek)
Kernel (POSIX)
Buffer
Cache
VFS
NFS
blocks

7. The Memory Hierarchy We’re slaves to the Application Persistence layer
Oracle
SAP HANA
Updates (read/write/lseek)
Kernel (POSIX)
Buffer
Cache
VFS
NFS
blocks

8. The Memory Hierarchy Application std::map (log2N, log2N)
std:list (constant, N)
boost::XXX
bytes
Control path (mmap, truncate)
Kernel (POSIX)
Buffer
Cache
VFS
NFS

9. (Naïve) History of Databases
Horizontal Decomposition (tables)
Record oriented
SEQUEL
(SQL)
Oracle
MonetDB
1956
1974
1979
1999
2006

10. Asymptotic Opportunity
End of the artificial dichotomy of OLAP/OLTP
Many “in-memory” databases have missed the opportunity
Performance improvement restricted to inherent medium speedup (DRAM)
New data structures with random access possible!
Tables (B+ tree derivative) or columns are artifacts of history
O(N) is not always the best way.
Frequently leads to O(N2)

11. Doug Santry