1. Managing Table Partitions at the Extreme
Ron Talmage
Managing Table Partitions at the Extreme

2. Please Thank our Sponsors:

3. Agenda Overview of Table Partitioning The Partition Function
The Partition Scheme
Upper Extreme: Adding new partitions
Lower Extreme: NULL and archiving

4. Table Partitioning Overview
Some uses for table partitioning:
Eliminate partitions from queries
Manage indexes by partition
Help with archiving data:
SQL can TRUNCATE by partition
By SWITCH

5. Basics Partition Function Partitioned Partition Table Scheme
Fillegroups
Files

6. Table Partitioning: There
Partitioned Table
Ptn Function
Ptn Scheme
CREATE TABLE <tbl> AS …
XXDate DateTime
ON PtnScheme(xxDate);
Refers to Ptn Function
Maps partitions to FGs
Specifies data type
Enumerates Boundary Points
Ptn Key
Data type
Ptn Scheme
Filegroups
File(s) per filegroup

7. And back again Partitioned Table Ptn Scheme P1 P2 P3 P4 Ptn Function
CREATE TABLE <tbl> AS …
XXDate DateTime
ON PtnScheme(xxDate);
FG1
FG2
Refers to Ptn Function
Maps partitions to FGs
Specifies data type
Enumerates boundary values
Ptns Per FG
Archive unit
RANGE RIGHT/LEFT
Sorted boundary values
NULL values in partition 1
“Past” partition
“Future” partition
$PARTITION()
SPLIT/MERGE
Ptn Key
Data type
Ptn Scheme
Filegroups
ALTER TABLE SWITCH
TRUNCATE TABLE Ptn
File(s) per filegroup

8. Zeroing in on the Extremes
Upper Extreme: Adding new partitions Lower Extreme: NULL and Past data, archiving
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5

9. Partition Function

10. The Partition Function: Boundary Values
Lowest
Value
Highest
Value
Range is a series of discrete possible values
Range values are based on data type, sorted from lowest value to highest value
To form partitions, choose boundary values
Boundary values are taken from the available range values
Decide RANGE RIGHT or RANGE LEFT

11. Why choose RANGE RIGHT or LEFT?
Consider a single boundary value (two partitions)
Lowest
value
Highest
Value
We must choose the type of partition that the boundary value will go into
RANGE RIGHT
RANGE LEFT

12. The Partition Function: RANGE RIGHT/LEFT
Lowest
value
Highest
Value
RANGE RIGHT: boundary_value_on_right = 1 P2 P1 P3 P4
RANGE LEFT: boundary_value_on_right = 0 P1 P2 P3 P4

13. Why RANGE RIGHT is preferred
Lowest
value
Highest
Value
RANGE RIGHT: boundary_value_on_right = 1
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
<
OR NULL
>=
AND
<
>=
AND
<
>=
AND
<
>=
AND
<
>=
Suppose the DATE data type and partition by month, and partition boundaries of , , , etc.
RANGE RIGHT produces partitions of
to ,
to , etc.
RANGE LEFT results in
to , to
Each partition ends up with data from two months.
Would need boundaries of , , etc. to have monthly ptns
RANGE LEFT: boundary_value_on_right = 0
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
<=
OR NULL
>
AND
<=
>
AND
<=
>
AND
<=
>
AND
<=
>

14. Creating a NULL partition
NULL can be a partition boundary value
Only for partition 1
Some data data types in the partition function definition must use explicit conversions
Date
DateTime2()

15. Demo 1: Partition functions
PtnExtremeDemo1- PartitionFunctions.sql

16. Partition Scheme

17. Partition Schemes
Partition schemes map resulting partitions to filegroups
To illustrate, let’s assume:
Datetime data type
Partition by quarter
1 year per filegroup
Year 2019 in one filegroup

18. Adding Filegroups Lowest value Highest Value Ptn 1 Ptn 2 Ptn 3 Ptn 4
Lowest
value
Highest
Value
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
<
OR NULL
>=
AND
<
>=
AND
<
>=
AND
<
>=
AND
<
>=
“Past” FG
FG2019
“Future” FG

19. How many partitions in a filegroup?
Possible criteria: simplicity and archiving
Assume a date-oriented partition by quarter, 10 years and 40 Ptns
10 Years
Ptn by Qtr:
= 40 Ptns
Archive
Unit
Ptns/FG
Total
FGs
Files/FG
TotalFiles
/DB
Balanced
I/O Y/N
Mtc/Complexity
Level 1
Qtrly 40 4
160 Y
High 2
Yearly 10
Medium 3
2 yrs 8 5 20
5 yrs
Low

20. Demo 2
PtnExtremeDemo2 – PartitionSchemes.sql

21. Upper Extreme

22. Upper Extreme: Adding new partitions
Current:
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
FG2019
FG2020
>=
FGPast
<
OR NULL
Desired:
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
Ptn 7
Ptn 8
Ptn 9
Ptn 10
FGPast
<
OR NULL
FG2019
FG2020
FG2021
>=

23. Extending rowstore partitions – Unstacking Boxes
Start with the farthest out
Start with the nearest 4 4 3 3 4 2 2 3 4 1 2 3 4 1 2 3 4
Moves 3 boxes
Moves = 6 boxes

24. Rowstore first try: using SPLIT in ascending order
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
Ptn 7
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
Ptn 7
Ptn 8
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
Ptn 7
Ptn 8
Ptn 9
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
Ptn 7
Ptn 8
Ptn 9
Ptn 10
FGPast
<
OR NULL
FG2019
FG2020
FG2021
>=

25. Rowstore second try: SPLIT in descending order
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
Ptn 7
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
Ptn 7
Ptn 8
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
Ptn 7
Ptn 8
Ptn 9
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
Ptn 7
Ptn 8
Ptn 9
Ptn 10
FGPast
<
OR NULL
FG2019
FG2020
FG2021
>=

26. Undoing the expansion: Re-Stacking boxes
Hmm…
Do the closest first 4 4 3 3 4 2 2 3 4 1 2 3 4 1 2 3 4
Moves 3 boxes
Moves = 6 boxes

27. Reversing: MERGE in descending order
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
Ptn 7
Ptn 8
Ptn 9
Ptn 10
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
Ptn 7
Ptn 8
Ptn 9
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
Ptn 7
Ptn 8
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
Ptn 7
FGPast
<
OR NULL
FG2019
FG2020
FG2021
>=

28. CCI Expanding Partitions:
Problem: Only empty partitions can be split or merged on CCI tables
Strategy 1: SWITCH out the last partition, add the new partitions, and load them from the switched table
Strategy 2: Convert the table to rowstore using a clustered index and expand.

29. Demo 3
PtnExtremeDemo3 - Expanding Partitions.sql

30. Lower Extreme

31. Lower Extreme: NULL and Archiving
SQL Server allows NULL as a boundary value
Works for datetime, integers, char/varchar
Does not work for date or datetime2
The NULL partition will always be Ptn 1
With no NULL partition, partition key NULL values are stored in Ptn 1 with Past data

32. Adding a NULL Partition
Lowest
value
Highest
Value
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
Ptn 7
>= Lowest AND
<
>=
AND
<
>=
AND
<
>=
AND
<
>=
AND
<
>=
NULL
NULL FG
“Past” FG
FG2019
“Future” FG

33. Archiving: No NULL partition
Current:
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
Ptn 7
Ptn 8
Ptn 9
Ptn 10
FGPast
<
OR NULL
FG2019
FG2020
FG2021
>=
Desired:
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
FG2021
>=
FGPast
<
OR NULL
FG2020
FGPre2019
FG2019

34. Archiving: With NULL partition
Current:
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
Ptn 7
Ptn 8
Ptn 9
Ptn 10
Ptn 10
FGNULL
FGPast
<
FG2019
FG2020
FG2021
>=
Desired:
Ptn 1
Ptn 2
Ptn 3
Ptn 4
Ptn 5
Ptn 6
Ptn 7
FG2021
>=
FGNULL
FGPast
<
FG2020
FGPre2019
FG2019

35. Thanks! And references Dan Guzman, Table Partitioning Best Practices
Catherine Wilhelmsen, Table Partitioning in SQL Server – The Basics
Niko Neugebauer, Columnstore Indexes – part 116 (“Partitioning Specifics”)