Working with Very Large Tables Like a Pro in SQL Server 2014 Guy Glantser Image courtesy of marin / FreeDigitalPhotos.net

Copyright © 2014 Madeira Ltd All Rights Reserved Full rights, including copyrights, belong exclusively to Madeira Ltd No use of the materials, in any form, is allowed, unless receiving a prior written permission from Madeira Ltd

A Few Words about Me… Name: Guy Glantser Email Address: guy@madeira.co.il Twitter: @guy_glantser Blog: www.madeirasql.com/author/guyglantser Podcast: www.sqlserverradio.com Image courtesy of Mister GC / FreeDigitalPhotos.net

Working with Very Large Tables Like a Pro in SQL Server 2014 Agenda Working with Very Large Tables Like a Pro in SQL Server 2014 Image courtesy of marin / FreeDigitalPhotos.net

Working with Very Large Tables Like a Pro in SQL Server 2014 Agenda Working with Very Large Tables Like a Pro in SQL Server 2014 Image courtesy of marin / FreeDigitalPhotos.net

Working with Very Large Tables Like a Pro in SQL Server 2014 Agenda Working with Very Large Tables Like a Pro in SQL Server 2014 Image courtesy of marin / FreeDigitalPhotos.net

Working with Very Large Tables Like a Pro in SQL Server 2014 Agenda Working with Very Large Tables Like a Pro in SQL Server 2014 Image courtesy of marin / FreeDigitalPhotos.net

Working with Very Large Tables Like a Pro in SQL Server 2014 Agenda Working with Very Large Tables Like a Pro in SQL Server 2014 Image courtesy of marin / FreeDigitalPhotos.net

So What is a Very Large Table? 5GB 5TB Images courtesy of satit_srihin & John Kasawa / FreeDigitalPhotos.net

So What is a Very Large Table? It depends… Image courtesy of David Castillo Dominici / FreeDigitalPhotos.net

So What is a Very Large Table? is One that Requires Special Care Image courtesy of stockimages / FreeDigitalPhotos.net

So What is a Very Large Table? File.Events Web.PageViews Audit.Messages Monitor.Measures Billing.Transactions

So What is a Very Large Table? Web.PageViews

Working with Very Large Tables

Working with Very Large Tables Query the Table Image courtesy of Master isolated images / FreeDigitalPhotos.net

Working with Very Large Tables Query the Table Update the Table Image courtesy of Stuart Miles / FreeDigitalPhotos.net

Working with Very Large Tables Query the Table Update the Table Manage the Table Image courtesy of Stuart Miles / FreeDigitalPhotos.net

Working with Very Large Tables Query the Table Update the Table Manage the Table Maintain the Table Image courtesy of xedos4 / FreeDigitalPhotos.net

What We are NOT Going to Cover… Partition the Table Archive/Delete Old Partitions Spread Partitions Between Storage Tiers Image courtesy of farconville / FreeDigitalPhotos.net

What We are NOT Going to Cover… Compress the Data Bulk Insert Rows Optimize Parallelism Image courtesy of farconville / FreeDigitalPhotos.net

What We are NOT Going to Cover… Use a Lower Isolation Level In-Memory OLTP Columnstore Indexes Image courtesy of farconville / FreeDigitalPhotos.net

Let’s start with querying the table… Image courtesy of Master isolated images / FreeDigitalPhotos.net

The Ascending Key Problem Image courtesy of suphakit73 / FreeDigitalPhotos.net

Problem #1 – Solutions Update Statistics Manually Trace Flag 2371 New Cardinality Estimator Filtered Statistics Incremental Statistics Image courtesy of suphakit73 / FreeDigitalPhotos.net

What about the data load? Image courtesy of Stuart Miles / FreeDigitalPhotos.net

Problem #2 The WRITELOG Wait Type Image courtesy of stockimages / FreeDigitalPhotos.net

Problem #2 – Solution Delayed Durability Image courtesy of stockimages / FreeDigitalPhotos.net

Delayed Durability Transaction Properties (ACID): Atomicity Consistency Isolation Durability

Fully-Durable Transactions Delayed Durability Data File Buffer Pool Commit Checkpoint Request Transaction Log Buffer Log File WRITELOG

Delayed Durability Transactions Data File Buffer Pool Commit Checkpoint Request Transaction Log Buffer Log File

Delayed Durability Can be controlled by: Database (Disabled | Allowed | Forced) Transaction (Commit) Atomic Block (Natively Compiled Procedure) Image courtesy of digitalart / FreeDigitalPhotos.net

Delayed Durability The log buffer is flushed to disk when: A fully durable transaction in the same database makes a change in the database and successfully commits Image courtesy of Keerati / FreeDigitalPhotos.net

Delayed Durability The log buffer is flushed to disk when: The user executes the system stored procedure “sys.sp_flush_log” successfully Image courtesy of Keerati / FreeDigitalPhotos.net

Delayed Durability The log buffer is flushed to disk when: The in-memory transaction log buffer fills up and automatically flushes to disk Image courtesy of Keerati / FreeDigitalPhotos.net

Delayed Durability The log buffer is flushed to disk when: Every few milliseconds Image courtesy of Keerati / FreeDigitalPhotos.net

Delayed Durability The following transactions are always fully durable: Transactions that perform write operations against tables that are enabled for either Change Tracking or Change Data Capture Image courtesy of Iamnee / FreeDigitalPhotos.net

Delayed Durability The following transactions are always fully durable: Cross-database transactions or distributed transactions Image courtesy of Iamnee / FreeDigitalPhotos.net

Delayed Durability The following features apply only to transactions that have been made durable: Transaction Replication – Transactions are only replicated after they have been made durable Image courtesy of Stuart Miles / FreeDigitalPhotos.net

Delayed Durability The following features apply only to transactions that have been made durable: Log Backup – Only transactions that have been made durable are included in the backup Image courtesy of Stuart Miles / FreeDigitalPhotos.net

Delayed Durability The following features apply only to transactions that have been made durable: Log Shipping – Only transactions that have been made durable are shipped to the secondaries Image courtesy of Stuart Miles / FreeDigitalPhotos.net

Delayed Durability What are the benefits? Less roundtrips to the log file on disk Less WRITELOG waits Shorter transactions Locks are released earlier Better throughput Image courtesy of Ambro / FreeDigitalPhotos.net

Delayed Durability But remember that you might lose data!

Finally, let’s talk about maintenance… Image courtesy of xedos4 / FreeDigitalPhotos.net

Problem #3 Index Rebuild Image courtesy of suphakit73 / FreeDigitalPhotos.net

Index Rebuild Before SQL Server 2014 we could: Rebuild all partitions offline Rebuild all partitions online Rebuild a specific partition offline Image courtesy of David Castillo Dominici / FreeDigitalPhotos.net

Index Rebuild In SQL Server 2014 we can: Rebuild a specific partition online Wait at low priority when: Rebuilding an index online Switching partitions Image courtesy of David Castillo Dominici / FreeDigitalPhotos.net

Wait at Low Priority Wait in a separate low priority queue Wait for a specified number of minutes Image courtesy of ddpavumba / FreeDigitalPhotos.net

Wait at Low Priority After that, choose one of 3 options: Continue to wait at normal priority Abort the operation Kill all blockers and start the operation Image courtesy of ddpavumba / FreeDigitalPhotos.net

Wait at Normal Priority Queue Insert Insert Online Index Rebuild Running Image courtesy of artur84 / FreeDigitalPhotos.net

Wait at Low Priority “Normal” Queue Insert Insert Online Index Rebuild Running Low Priority Queue Image courtesy of artur84 / FreeDigitalPhotos.net

Working with Very Large Tables Like a Pro in SQL Server 2014 Summary Working with Very Large Tables Like a Pro in SQL Server 2014 Image courtesy of marin / FreeDigitalPhotos.net

Working with Very Large Tables Like a Pro in SQL Server 2014 Summary Working with Very Large Tables Like a Pro in SQL Server 2014 Image courtesy of marin / FreeDigitalPhotos.net

Summary A Very Large Table is One that Requires Special Care Image courtesy of stockimages / FreeDigitalPhotos.net

Working with Very Large Tables Like a Pro in SQL Server 2014 Summary Working with Very Large Tables Like a Pro in SQL Server 2014 Image courtesy of marin / FreeDigitalPhotos.net

Summary Query the Table Update the Table Manage the Table Maintain the Table Image courtesy of Master isolated images, Stuart Miles & xedos4 / FreeDigitalPhotos.net

Working with Very Large Tables Like a Pro in SQL Server 2014 Summary Working with Very Large Tables Like a Pro in SQL Server 2014 Image courtesy of marin / FreeDigitalPhotos.net

Summary Solving the Ascending Key problem Update Statistics Manually Trace Flag 2371 New Cardinality Estimator Filtered Statistics Incremental Statistics Image courtesy of suphakit73 / FreeDigitalPhotos.net

Summary Solving the WRITELOG problem with Delayed Durability Less roundtrips to the log file on disk Less WRITELOG waits Shorter transactions & Locks Better throughput Image courtesy of Ambro / FreeDigitalPhotos.net

Summary Solving the Index Rebuild problem Rebuild a specific partition online Wait at low priority Image courtesy of suphakit73 / FreeDigitalPhotos.net

Working with Very Large Tables Like a Pro in SQL Server 2014 Summary Working with Very Large Tables Like a Pro in SQL Server 2014 Image courtesy of marin / FreeDigitalPhotos.net

Summary New Features in SQL Server 2014 New Cardinality Estimator Incremental Statistics Delayed Durability Online Indexing Enhancements

Stay In Touch… Name: Guy Glantser Email Address: guy@madeira.co.il Twitter: @guy_glantser Blog: www.madeirasql.com/author/guyglantser Podcast: www.sqlserverradio.com Image courtesy of Nuttapong / FreeDigitalPhotos.net

Image courtesy of David Castillo Dominici / FreeDigitalPhotos.net