very large Databases in your future Eric Peterson

VLDB how large? impact? why? easy fixes other fixes maintenance operations

how large? terabyte + depends on hardware examples 2.2 TB (2,222 GB) db EMC VNX 5500 SAN 2 - 4 gb HBA 10 gb nic

impact? Everything takes longer ! Example db raw disk I/O SQLIO raw network 2.2 TB = 2,222 GB sqliosim | iometer https://flic.kr/p/e1Mzeu I/O 397 MB/sec = 23 GB/min 10gb/sec = 1.25 GB/sec = 75 GB/min

impact?

impact? backup / restore copies & initial sync re-index 19 GB/min or 2,222 GB in 108 min 75 GB/min or 2,222 GB in 30 min 11 GB/min or 2,222 GB in 300 min

I/O data transfers backup 19 GB/min hh:mm GB 00:05 100 00:49 1000 01:48 2222 08:06 10,000

I/O data transfers copy 75 GB/min hh:mm GB 00:01 100 00:13 1,000 00:30 2,222 02:13 10,000

I/O data transfers reindex 11 GB/min hh:mm GB 00:14 100 02:15 1,000 05:00 2,222 2230 10,000

impact? maintenance window query optimizer; 200 step histogram test environment large logs TempDB temp tables, sorts, joins, online indexing, snapshot isolation

why? queries & rollbacks network copies backups synchronization one table 266m rows 222GB data, 285GB index 20 sec table scan graphic of disk, memory & network

Easy fixes compress data Enterprise Edition less I/O, more CPU per table / index sp_estimate_data_compression_savings no LOB https://www.flickr.com/photos/marcovdz/4520986339/ compression estimator sql slavasql.blogspot.co.uk/2014/08/sql-server-data-compression-hidden-gem.html

“easy” fixes archive / purge data old / unnecessary data multiple TempDB files upgrade system / hardware add RAM faster disks

Other fixes partitioning Enterprise Edition per table needs useful primary key sliding window automation reference to codeplex.com partitioning

other fixes

other fixes multiple data files concurrent reads/writes requires file groups maintenance window to move data columnstore (in-memory) indexes be aware of restrictions

other fixes multiple data files

Maintenance online index rebuild Enterprise Edition duplicate space no XML, spatial, columnstore single partition reindex Paul Randall DBCC with VLDB article

Maintenance DBCC run on copy CheckAlloc, CheckCatalog, CheckTable Paul Randall DBCC VLDB article filegroups manually grow equal size files

Operations monitoring sp_whoisactive sys.dm_exec_procecure_stats history output from whoisactive and sys.dm…

Operations monitoring (with history) sp_whoisactive sys.dm_exec_procedure_stats history output from whoisactive and sys.dm…

operations look for anomalies can change query plan

operations query tuning scans of large tables joins to large tables excessive reads

operations query tuning large table scans large joins excessive reads

Operations work in batches

Operations snapshot isolation Change Data Capture new data to warehouse NOLOCK ? parallelism cost threshold

thanks! contact info Eric Peterson eric@rdc-corp.com

Thank you to all of our Sponsors! Diamond Sponsors Platinum Sponsors Gold Sponsors Marquee Sponsors 9/19/2015 SQLSaturday #441 – Denver 2015

Thank you to all of our Sponsors! Silver Sponsors SWAG Sponsors 9/19/2514 SQLSaturday #441 – Denver 2015