1. Working with Very Large Tables Like a Pro in SQL Server 2014
Guy Glantser
Image courtesy of marin / FreeDigitalPhotos.net

2. Copyright © 2014 Madeira Ltd All Rights Reserved
Full rights, including copyrights, belong exclusively to Madeira Ltd
No use of the materials, in any form, is allowed,
unless receiving a prior written permission from Madeira Ltd

3. A Few Words about Me… Name: Guy Glantser
Address:
Twitter: @guy_glantser
Blog:
Podcast:
Image courtesy of Mister GC / FreeDigitalPhotos.net

5. Working with Very Large Tables Like a Pro in SQL Server 2014
Agenda
Working with Very Large Tables Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net

6. Working with Very Large Tables Like a Pro in SQL Server 2014
Agenda
Working with Very Large Tables Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net

7. Working with Very Large Tables Like a Pro in SQL Server 2014
Agenda
Working with Very Large Tables Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net

8. Working with Very Large Tables Like a Pro in SQL Server 2014
Agenda
Working with Very Large Tables Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net

9. Working with Very Large Tables Like a Pro in SQL Server 2014
Agenda
Working with Very Large Tables Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net

10. So What is a Very Large Table?
5GB
5TB
Images courtesy of satit_srihin & John Kasawa / FreeDigitalPhotos.net

11. So What is a Very Large Table?
It depends…
Image courtesy of David Castillo Dominici / FreeDigitalPhotos.net

12. So What is a Very Large Table?
is One that Requires
Special Care
Image courtesy of stockimages / FreeDigitalPhotos.net

13. So What is a Very Large Table?
File.Events
Web.PageViews
Audit.Messages
Monitor.Measures
Billing.Transactions

14. So What is a Very Large Table?
Web.PageViews

15. Working with Very Large Tables

16. Working with Very Large Tables
Query the Table
Image courtesy of Master isolated images / FreeDigitalPhotos.net

17. Working with Very Large Tables
Query the Table
Update the Table
Image courtesy of Stuart Miles / FreeDigitalPhotos.net

18. Working with Very Large Tables
Query the Table
Update the Table
Manage the Table
Image courtesy of Stuart Miles / FreeDigitalPhotos.net

19. Working with Very Large Tables
Query the Table
Update the Table
Manage the Table
Maintain the Table
Image courtesy of xedos4 / FreeDigitalPhotos.net

20. What We are NOT Going to Cover…
Partition the Table
Archive/Delete Old Partitions
Spread Partitions Between Storage Tiers
Image courtesy of farconville / FreeDigitalPhotos.net

21. What We are NOT Going to Cover…
Compress the Data
Bulk Insert Rows
Optimize Parallelism
Image courtesy of farconville / FreeDigitalPhotos.net

22. What We are NOT Going to Cover…
Use a Lower Isolation Level
In-Memory OLTP
Columnstore Indexes
Image courtesy of farconville / FreeDigitalPhotos.net

23. Let’s start with querying the table…
Image courtesy of Master isolated images / FreeDigitalPhotos.net

24. The Ascending Key Problem
Image courtesy of suphakit73 / FreeDigitalPhotos.net

25. Problem #1 – Solutions Update Statistics Manually Trace Flag 2371
New Cardinality Estimator
Filtered Statistics
Incremental Statistics
Image courtesy of suphakit73 / FreeDigitalPhotos.net

26. What about the data load?
Image courtesy of Stuart Miles / FreeDigitalPhotos.net

27. Problem #2 The WRITELOG Wait Type
Image courtesy of stockimages / FreeDigitalPhotos.net

28. Problem #2 – Solution Delayed Durability
Image courtesy of stockimages / FreeDigitalPhotos.net

29. Delayed Durability
Transaction Properties (ACID):
Atomicity
Consistency
Isolation
Durability

30. Fully-Durable Transactions
Delayed Durability
Data File
Buffer Pool
Commit
Checkpoint
Request
Transaction Log Buffer
Log File
WRITELOG

31. Delayed Durability Transactions
Data File
Buffer Pool
Commit
Checkpoint
Request
Transaction Log Buffer
Log File

32. Delayed Durability Can be controlled by:
Database (Disabled | Allowed | Forced)
Transaction (Commit)
Atomic Block (Natively Compiled Procedure)
Image courtesy of digitalart / FreeDigitalPhotos.net

33. Delayed Durability The log buffer is flushed to disk when:
A fully durable transaction in the same database makes a change in the database and successfully commits
Image courtesy of Keerati / FreeDigitalPhotos.net

34. Delayed Durability The log buffer is flushed to disk when:
The user executes the system stored procedure “sys.sp_flush_log” successfully
Image courtesy of Keerati / FreeDigitalPhotos.net

35. Delayed Durability The log buffer is flushed to disk when:
The in-memory transaction log buffer fills up and automatically flushes to disk
Image courtesy of Keerati / FreeDigitalPhotos.net

36. Delayed Durability The log buffer is flushed to disk when:
Every few milliseconds
Image courtesy of Keerati / FreeDigitalPhotos.net

37. Delayed Durability
The following transactions are always fully durable:
Transactions that perform write operations against tables that are enabled for either Change Tracking or Change Data Capture
Image courtesy of Iamnee / FreeDigitalPhotos.net

38. Delayed Durability
The following transactions are always fully durable:
Cross-database transactions or distributed transactions
Image courtesy of Iamnee / FreeDigitalPhotos.net

39. Delayed Durability
The following features apply only to transactions that have been made durable:
Transaction Replication – Transactions are only replicated after they have been made durable
Image courtesy of Stuart Miles / FreeDigitalPhotos.net

40. Delayed Durability
The following features apply only to transactions that have been made durable:
Log Backup – Only transactions that have been made durable are included in the backup
Image courtesy of Stuart Miles / FreeDigitalPhotos.net

41. Delayed Durability
The following features apply only to transactions that have been made durable:
Log Shipping – Only transactions that have been made durable are shipped to the secondaries
Image courtesy of Stuart Miles / FreeDigitalPhotos.net

42. Delayed Durability What are the benefits?
Less roundtrips to the log file on disk
Less WRITELOG waits
Shorter transactions
Locks are released earlier
Better throughput
Image courtesy of Ambro / FreeDigitalPhotos.net

43. Delayed Durability
But remember that
you might lose data!

44. Finally, let’s talk about maintenance…
Image courtesy of xedos4 / FreeDigitalPhotos.net

45. Problem #3 Index Rebuild
Image courtesy of suphakit73 / FreeDigitalPhotos.net

46. Index Rebuild Before SQL Server 2014 we could:
Rebuild all partitions offline
Rebuild all partitions online
Rebuild a specific partition offline
Image courtesy of David Castillo Dominici / FreeDigitalPhotos.net

47. Index Rebuild In SQL Server 2014 we can:
Rebuild a specific partition online
Wait at low priority when:
Rebuilding an index online
Switching partitions
Image courtesy of David Castillo Dominici / FreeDigitalPhotos.net

48. Wait at Low Priority Wait in a separate low priority queue
Wait for a specified number of minutes
Image courtesy of ddpavumba / FreeDigitalPhotos.net

49. Wait at Low Priority After that, choose one of 3 options:
Continue to wait at normal priority
Abort the operation
Kill all blockers and start the operation
Image courtesy of ddpavumba / FreeDigitalPhotos.net

50. Wait at Normal Priority
Queue
Insert
Insert
Online
Index Rebuild
Running
Image courtesy of artur84 / FreeDigitalPhotos.net

51. Wait at Low Priority “Normal” Queue Insert Insert Online Index Rebuild
Running
Low Priority
Queue
Image courtesy of artur84 / FreeDigitalPhotos.net

52. Working with Very Large Tables Like a Pro in SQL Server 2014
Summary
Working with Very Large Tables Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net

53. Working with Very Large Tables Like a Pro in SQL Server 2014
Summary
Working with Very Large Tables Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net

54. Summary A Very Large Table is One that Requires Special Care
Image courtesy of stockimages / FreeDigitalPhotos.net

55. Working with Very Large Tables Like a Pro in SQL Server 2014
Summary
Working with Very Large Tables Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net

56. Summary Query the Table Update the Table Manage the Table
Maintain the Table
Image courtesy of Master isolated images, Stuart Miles & xedos4 / FreeDigitalPhotos.net

57. Working with Very Large Tables Like a Pro in SQL Server 2014
Summary
Working with Very Large Tables Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net

58. Summary Solving the Ascending Key problem Update Statistics Manually
Trace Flag 2371
New Cardinality Estimator
Filtered Statistics
Incremental Statistics
Image courtesy of suphakit73 / FreeDigitalPhotos.net

59. Summary Solving the WRITELOG problem with Delayed Durability
Less roundtrips to the log file on disk
Less WRITELOG waits
Shorter transactions & Locks
Better throughput
Image courtesy of Ambro / FreeDigitalPhotos.net

60. Summary Solving the Index Rebuild problem
Rebuild a specific partition online
Wait at low priority
Image courtesy of suphakit73 / FreeDigitalPhotos.net

61. Working with Very Large Tables Like a Pro in SQL Server 2014
Summary
Working with Very Large Tables Like a Pro in SQL Server 2014
Image courtesy of marin / FreeDigitalPhotos.net

62. Summary New Features in SQL Server 2014 New Cardinality Estimator
Incremental Statistics
Delayed Durability
Online Indexing Enhancements

63. Stay In Touch… Name: Guy Glantser Email Address: guy@madeira.co.il
Twitter: @guy_glantser
Blog:
Podcast:
Image courtesy of Nuttapong / FreeDigitalPhotos.net

64. Image courtesy of David Castillo Dominici / FreeDigitalPhotos.net