Data Management Conference Tackling the Data Management Challenge Mark Whitehorn Mark@Penguinsoft.co.uk London September 29th

Agenda State Speed Simplification

Agenda State Speed Simplification Finish on time

State of Data Management In the beginning......

State of Data Management In the beginning...... there was Bob Cratchit He (and his ilk) certainly managed data but there was no real science involved

State of Data Management And then there were computer based databases For the first time we systematically studied data

State of Data Management Most of the early work was on transactional Systems (OLTP)

State of Data Management Most of the early work was on transactional Systems (OLTP) Jim Gray

State of Data Management We learnt to separate data processing from applications And so the job roles of DBD and DBA appeared

State of Data Management So, what’s new?

State of Data Management So, what’s new? BI – Business Intelligence

State of Data Management Actually, not that new One of the earliest British computers was used both for transactions and reporting

State of Data Management

State of Data Management 1951

State of Data Management LEO 1 Control Desk Image used with the kind permission of the Copyright holder LEO Computers society - www.leo-computers.org.uk

State of Data Management LEO 1 Decimal/sterling binary converter Image used with the kind permission of the Copyright holder LEO Computers society - www.leo-computers.org.uk

State of Data Management BI – Business Intelligence - Reporting and...... OLAP – (On Line Analytical Processing) Data Warehousing Data Mining Dimensional Modelling ETL – (Extract Transform Load)

State of Data Management So where does that leave current DBAs and DBDs?

State of Data Management So where does that leave current DBAs and DBDs? BI brings analytical data management We think in terms of measures and dimensions Analysis moves from the application to the engine We structure the data for analysis

State of Data Management So where does that leave current DBAs and DBDs? Greater job satisfaction More money (try job searching on ‘MDX’...)

Speed New sentence that you have never heard before

Speed New sentence that you have never heard before “Please slow the database down, it’s running too fast.”

Speed - CPU CPU performance has tended to follow Moore’s Law and double every 18-24 months

Speed - Disks There is a hidden agenda here......disks

Speed - Disks There is a hidden agenda here......disks SSD – Solid State Disks Shock resistant Cool

Speed - Disks So they are great for laptops: extending battery life cooler laps But they are woefully underused in databases

SSD MLC vs. SLC Cost Speed Energy efficiency

MLC vs. SLC SSDs use flash memory technology Non-volatile NAND

MLC vs. SLC NAND (Not AND) - Boolean logic operator that describes how the device stores data, deep in the underworld of bits and electrical impulses

MLC vs. SLC NAND has 2 flavours: single-level cell (SLC) – 1bpc multi-level cell (MLC) – 2 (or more) bpc (bit per cell)

MLC vs. SLC Both display the same rapid read rate, which is so rapid that the disks can easily saturate the 3 Gigabit per second SATA 3 bus (250 MB/sec)

MLC vs. SLC SLC has twice the write rate of MLC but is, inevitably, more expensive

Cost In 1985 a terabyte of HDD storage cost £100M….. Today it’s about £100

Cost In 1985 a terabyte of HDD storage cost £100M….. Today it’s about £100 (back in 1956 a 5 Mbyte IBM drive was $50K so 1 Tbyte was about $10 thousand million)

Cost SSD Price - Expensive but prices dropping

Cost Price 80 GB SSD (MLC): April 2009 - £500 (£6.2/GB). June 2009 - £350 (£4.4/GB) Last Friday 5:45 PM - £290 (£3.6/GB) So I make that £3,600 per Tbyte For bulk buying, call it £3K - £3/GB HDD - £0.1/GB

Cost So I make that £3,600 per Tbyte For bulk buying, call it £3K - £3/GB But, just to put that into perspective, that was the same price as we were paying for 1 Tbyte of HDD back in 2001….

Speed HDDs may have become massive and cheap but the I/O performance has not kept pace IOPS – Input Output Per Second

speed Look to the latest batch of SSDs: For example, Intel’s X18-M and X25-M 160 Gbyte drive: Sequential Read (MB/sec) – 250 Sequential Write (MB/sec) – 70 Random Read (IOPS) – 35,000 Random Write (IOPS) – 8,600 http://download.intel.com/design/flash/nand/mainstream/322296.pdf

Speed Calling for an end to Unrealistic SSD vs HDD IOPS comparisons “How much hay has your car eaten recently?” http://www.storagesearch.com/ssd-hdd-comparisons.html

Speed Random Write IOPS (4k blocks) SSD – 8,600 HHD – 200

Energy efficiency Energy consumption SSD – 0.15 Watts HHD – 16.00 Watts

Energy efficiency SSDs in Servers: Waste heat can be significantly reduced Cooling system can be simplified Disks packed closer together

Energy efficiency

SSD benefits We have several dimensions here: MLC vs. SLC Cost Speed Energy efficiency So we can start to cross join...

SSD benefits SLC twice the write rate of MLC but is more expensive So think: SLC for transactional (OLTP) MLC for BI (OLAP)

SSD benefits IOPS per watt SSD – 8,600/0.15 = 57,000 HDD – 200/16 = 12.5

SSD benefits Large databases: hot data on SSDs cold data on HDD Small databases: Put it all on SSD

SSD broader benefits Indexing strategy profoundly affected Michael Stonebraker’s arguments about Column vs. Row oriented databases

SSD benefits The take-home message is that SSDs are no longer exotic

SSD Faster SQL Server Access with Solid State Disks Woody Hutsell www.texmemsys.com/files/f000174.pdf A Comparison of Solid State Drives to Serial ATA (SATA) Drive Arrays for Use with Oracle Mike Ault www.dba-oracle.com/art_dbazine_ault_ssd.htm

Simplicity – BI 101 OLTP systems Data Warehouse Data Marts Image used with the kind permission of the Copyright holder LEO Computers society - www.leo-computers.org.uk

Simplicity Why?

Simplicity Why? Why so complex? Why so many copies of the data? Why so many aggregations?

Simplicity Because the relational model is too slow

Simplicity Because the relational model is too slow ARRRGGHHHH!!!!!!!

Simplicity Because relational databases are too slow

Simplicity OLTP systems Data Warehouse Data Marts Image used with the kind permission of the Copyright holder LEO Computers society - www.leo-computers.org.uk

Simplicity OLTP systems Data Warehouse Logical Analytical Layer Image used with the kind permission of the Copyright holder LEO Computers society - www.leo-computers.org.uk

Simplicity Is this where Madison is heading?

Agenda Impartiality

Agenda Oracle IBM Teradata Microsoft Loves You Lots

7/27/2019 8:17 PM © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.