1. Strategies for Working with Texas-sized Databases Robert L Davis Database Engineer www.sqlsoldier.com @SQLSoldier

2. Robert L Davis Microsoft Certified Master Data Platform MVP @SQLSoldier www.sqlsoldier.com Database Engineer BlueMountain Capital Management 15+ years working with SQL Server PASS Security Virtual Chapter http://security.sqlpass.org Volunteers needed

3. What is a Texas-sized Database? What is the smallest database that qualifies as a VLDB? 5 TB (35%, 7 Votes) 1 TB (25%, 5 Votes) 500 GB (20%, 4 Votes) 100 GB (15%, 3 Votes) 750 GB (5%, 1 Votes) 250 GB (0%, 0 Votes) 50 GB (0%, 0 Votes) Total Voters: 20

4. What is a Texas-sized Database? What is the largest db you have worked with? 1 TB to 5 TB (25%, 3 Votes) Less than 100 GB (17%, 2 Votes) 100 GB to 500 GB (17%, 2 Votes) 500 GB to 1 TB (17%, 2 Votes) 5 TB to 10 TB (8%, 1 Votes) 20 TB to 50 TB (8%, 1 Votes) More than 50 TB (8%, 1 Votes) 10 TB to 20 TB (0%, 0 Votes) Total Voters: 12

5. What is a Texas-sized Database? What are your biggest challenges with working with VLDBs? (choose up to 3) Index maintenance (79%, 11 Votes) Backups (57%, 8 Votes) Integrity checks (50%, 7 Votes) Archiving/purging data (36%, 5 Votes) Disk/disk space management (29%, 4 Votes) Partitioning management (21%, 3 Votes) Inefficient queries (14%, 2 Votes) Disk performance (7%, 1 Votes) Total Voters: 14

6.  Very large tables have very large indexes  Be selective  Reorganize vs. Rebuild  Index maintenance at partition level  Yay for online partition index maintenance!  Don’t check fragmentation levels  Check on different day  Design with index maintenance in mind  No LOB, XML, or spatial columns in tables that will get large  Combine partitioned views with partitioned tables  Minion Reindex: http://minionware.net/reindex/http://minionware.net/reindex/ Index Maintenance

7.  Partitioned views with partitioned tables Index Maintenance

8.  Everything takes longer  Back up time  Copy time  Restore time  Uses too much disk space  Uses too many resources  Network throughput  SAN throughput Backups: What’s Difficult

9.  Multiple dedicated drives  1 backup thread per drive/mount point  1 file per drive/mount point  Differential/Partial backup  Read-only filegroups/Read-write filegroups  Per file or filegroup  Tune backup options  MaxTransferSize  BufferCount  http://sirsql.net/content/2012/12/13/20121212automated-backup-tuning/ http://sirsql.net/content/2012/12/13/20121212automated-backup-tuning/  Fast drives  Crank up the SAN throughput  Maximize Queue Depth  Increase the SAN paths  BONUS: works for restores too Backups: Speeding up

10.  Takes a very long time  Restore backup to another server and run there  Run with PHYSICAL_ONLY option  Break it up into smaller parts over multiple nights  Per filegroup  Per table  See Paul Randal’s post: http://www.sqlskills.com/BLOGS/PAUL/post/CHECKDB-From- Every-Angle-Consistency-Checking-Options-for-a-VLDB.aspx Integrity Checks

11.  Takes a long time  Causes excessive blocking (lock escalation)  Deleted space difficult to recover  Partitioning addresses all of this  Switch partitions out  Drop table/remove filegroup Archiving / Purging Data

12.  No partitioning?  Delete in small batches  Ensure that log backups or Checkpoints are running between batches  Use targeted deletes  Identify the PK columns to be deleted  No good solution for reclaiming space  Try running DBCC SHRINKFILE with TRUNCATEONLY  If you must shrink, plan to rebuild indexes Archiving / Purging Data

13.  Log shipping and database mirroring require initializing with a full backup  Replication requires that the data get to the subscriber  Initialize from backup  Log shipping is the gateway  Set up log shipping  Convert to replication or mirroring Replication/High Availability

14.  Disable all backup jobs that may hit the database  Disable the log shipping jobs  Run each step of the log shipping jobs in order  Drop log shipping  Manually backup the primary database log using WITH NORECOVERY  Puts the primary database into a recovering state  Restore log backup on secondary using WITH RECOVERY  Brings the secondary database online  Take log backup of new primary database  Restore log backup on new secondary database using WITH NORECOVERY  Reconfigure log shipping Reversing Log Shipping

15. Strategies for Working with Texas-sized Databases

16.  Configure replication publication  Disable all backup jobs that may hit the database  Disable the log shipping jobs  Run each step of the log shipping jobs in order  Drop log shipping secondary  Change publication to allow initialization from backup  Back up the log of the publisher  Restore the log backup using WITH RECOVERY to recover the secondary  Create subscription on publisher  @backupdevicename = path/name of log backup created above  @sync_type = initialize with backup  Create subscription on subscriber  Re-enable any backup jobs  Drop log shipping primary if desired Convert Log Shipping to Replication

17. Before: 2 TB – Log shipped to 2 servers

18. After: Reverse log shipping and convert 1 server to replication

19. RESOURCES VLDB Series for SQL University: http://www.sqlsoldier.com/wp/sqlserver/sqluvldbwe ekintrotovldbs http://www.sqlsoldier.com/wp/sqlserver/sqluvldbwe ekintrotovldbs MinionIndex: http://minionware.net/reindex/http://minionware.net/reindex/ Automated Backup Tuning: http://sirsql.net/content/2012/12/13/20121212auto mated-backup-tuning/ http://sirsql.net/content/2012/12/13/20121212auto mated-backup-tuning/

20. Strategies for Working with Texas-sized Databases

21. Thanks! Thank you for coming!  My blog: www.sqlsoldier.comwww.sqlsoldier.com  Twitter: twitter.com/SQLSoldiertwitter.com/SQLSoldier