The Ins and Outs of Partitioned Tables SQL Saturday Houston Brian Dudley Senior Solution Architect Perficient, Inc.

Large Databases What is large? 2 | 9/18/2018 | Footer Goes Here

Large Databases What is large? Issues with large databases Backups Contention Maintenance Loading Archiving Index rebuilds Performance 3 | 9/18/2018 | Footer Goes Here

https://en.wikipedia.org/wiki/Partition_(database) What is Partitioning? “A partition is a division of a logical database or its constituent elements into distinct independent parts. Database partitioning is normally done for manageability, performance or availability reasons, as for load balancing.“ https://en.wikipedia.org/wiki/Partition_(database) Criteria Range – only one supported by SQL Server List Hash Partitioning Methods Vertical – splitting out columns Horizontal – dividing by row 4 | 9/18/2018 | Footer Goes Here

Benefits of Partitioning Partitioning is largely handled transparently Ability to move large amounts of data in or out of a table with metadata operation. Ease of maintenance Rebuild indexes by partition Piecemeal backup and restore at the table level Tuning of tables Location of data Compression by partition Performance gains Partition elimination Better joins across partitioned tables Increased options for parallel operation 5 | 9/18/2018 | Footer Goes Here

Caveats Partitioning is an Enterprise Version feature Use Developer Edition for testing and learning Partitioning is an involved topic. Test your assumptions 6 | 9/18/2018 | Footer Goes Here

Thank You Sponsors! Visit the Sponsor tables to enter their end of day raffles. Turn in your completed Event Evaluation form at the end of the day in the Registration area to be entered in additional drawings. Want more free training? Check out the Houston Area SQL Server User Group which meets on the 2nd Tuesday of each month. Details at http://houston.sqlpass.org 5/14/2016

Partitioning Mechanics Determine tables to partition What pain point does it address? What is the strategy? Decide on Partition Key Data Type Range Should be commonly used in queries Define Partition Function Develop Partition Scheme Create Tables and Indexes on Partition 8 | 9/18/2018 | Footer Goes Here

Partition Function Create Partition Function <name> (<input parameter data type>) As Range [Left|Right] For Values (<list>) Input Parameter Data Type Most types can be used Date, Time, Integer Date Key, Int are Common Enter sorted or it will be done Consider collation Watch formatting of Date/Time values Value List Can use variables and functions Cannot use SQL Expressions 0 to 14,999 entries Lends itself well to being automated No duplicate values Null can be specified, but should be avoided. Always sorts first. $Partition.<function name>(<value>) tells on which partition a row will be placed 9 | 9/18/2018 | Footer Goes Here

Left or Right Illustrated Create Partition Function PF_Left(int) As Range Left For Values(10, 20, 30, 40) Create Partition Function PF_LeftEquiv(int) As Range Right For Values(11, 21, 31, 41) Create Partition Function PF_Right(int) As Range Right For Values(10, 20, 30, 40) Create Partition Function PF_RightEquiv(int) As Range Left For Values(9, 19, 29, 39) Left <-…10 11…20 21…30 31…40 41…-> Right <-…9 10…19 20…29 31…39 40…-> Date Example – Right or Left? <- 2015 Dec 31 2016 Jan 1…31 2016 Feb 1…29 2016 Mar 1…31 2016 Apr 1 -> 10 | 9/18/2018 | Footer Goes Here

Partition Scheme File Group List Design Decision Create Partition Scheme <name> As Partition <partition function> To (<file group list>) Create Partition Scheme <name> As Partition <partition function> All To (<file group>) File Group List Number of Boundary Values + 1 Can specify “Next Used” Design Decision Single File group File group Per Partition Older data on slower media Mix of Read only and Writeable Rotating List to “Stripe” the data 11 | 9/18/2018 | Footer Goes Here

Partition a Table Table must have a column that serves as the Partition Key Must be identical to the function parameter Can be a Persisted Computed Column e.g. Composite key Include On <Partition Scheme>(<key>) Many Tables can use the same Scheme Partition existing table with Clustered Index SSMS has Wizards 12 | 9/18/2018 | Footer Goes Here

Indexes on Partitioned Tables Aligned – uses the same partition scheme as the table. Default behavior for an index is to be aligned. Partition key is added to the index if it is not already specified. Has an impact on unique indexes. All active indexes must be aligned when swapping partitions. Logically, each partition has its own index. Non-aligned – explicitly created on its own File Group or Partition Can give better performance for searches that cross partitions. Have to be disabled in order to swap data. Factor in the cost of rebuilding.

Inspecting with System Views sys.Partition_Functions sys.Partition_Range_Values Left Join because of the extra partition Add 1 to boundary_id for Range Right sys.Partition_Schemes sys.Indexes sys.Data_Spaces Holds both Partition Schemes and File Groups Type and Type Description fields sys.Destination_Data_Spaces sys.FileGroups sys.Partitions Every table and index has an entry 14 | 9/18/2018 | Footer Goes Here

Switching Partitions Major benefit of Partitioned Tables Requirements Able to move a large amount of data as a metadata operation Load new data Remove old data for archive or truncation Requirements Two tables are required, at least one is partitioned Partition schemes must be compatible Tables must match in column name, data type, and order Indexes must all be aligned and must match Destination partition must exist and be empty Must acquire a schema lock Can cause time outs SSMS has Wizards 15 | 9/18/2018 | Footer Goes Here

Switch Illustrated Alter Table Right Switch Partition 3 To StagingTable Partition 3 Right <-…9 10…19 20…29 31…39 40…-> Data is moved with a metadata operation StagingTable <-…9 10…19 20…29 31…39 40…->

Split and Merge Used to maintain partition functions and schemes All Partition Schemes using a Partition Function are changed. All Tables on a Partition Scheme are changed. Can cause data movement Best case is to only effect empty partitions Sliding Window use case uses both splitting and merging 17 | 9/18/2018 | Footer Goes Here

Merge Illustrated Alter Partition Function PF_Right() Merge Range(10) <-…9 10…19 20…29 31…39 40…-> Alter Partition Function PF_Right() Merge Range(10) This file group is removed. Any data is moved. 10…19 Right <-…19 20…29 31…39 40…->

Split Illustrated Right <-…9 10…19 20…29 31…39 40…-> Alter Partition Scheme PS_Right Next Used FG1 Alter Partition Function PF_Right() Split Range(50) FG1 Scheme knows what File Group to use. Data will be moved if necessary. Right <-…9 10…19 20…29 31…39 40…49 50…->

ETL – Incremental Load Scenario: Data mart is refreshed monthly with new transactional data. Use split to create a new partition on a designated Next File Group. Partitions can also be created in advance. Create a non-partitioned staging table matching the target structure. File group matches the new partition Populate the staging table with a bulk copy Add all necessary indexes. Add a Check constraint. Swap data into target table. Update statistics on the target table.

ETL – Sliding Window Scenario: Maintain a fixed number of months of data. Create a non-partitioned archive table with necessary indexes. File group matches the last partition. Swap out the last partition. Data can be truncated or archived. Merge the now empty partition. Use split to create a new partition on a designated Next File Group. Create a non-partitioned staging table matching the target table. File group matches the new partition Populate the staging table with a bulk copy Add all necessary indexes. Add a Check constraint. Swap data into target table. Update statistics on the target table.

Odds and Ends Partition Elimination Identity Columns Transactions Shows in the actual plan detail Identity Columns Enforcing Uniqueness Ident_Current() Transactions

References Microsoft Whitepapers Brent Ozar Books Online “Partitioned Table and Index Strategies Using SQL Server 2008” https://technet.microsoft.com/en-us/library/dd578580(v=sql.100).aspx Brent Ozar SQL Server Table Partitioning: Resources https://www.brentozar.com/sql/table-partitioning-resources/ Books Online

Questions?

Thank You Sponsors! Visit the Sponsor tables to enter their end of day raffles. Turn in your completed Event Evaluation form at the end of the day in the Registration area to be entered in additional drawings. Want more free training? Check out the Houston Area SQL Server User Group which meets on the 2nd Tuesday of each month. Details at http://houston.sqlpass.org 5/14/2016