An Introduction to Partitioning Kevin Krenz An Introduction to Partitioning

Thank you Sponsors! Platinum Sponsor: Gold Sponsors: Visit the Sponsor Booths Lots of Great Raffle Prizes! Get your parking paid via Sponsor Bingo Thank you Sponsors! Platinum Sponsor: Gold Sponsors: Global Alliance Partners:

PASSMN – News/Info Thanks to all our sponsors of 2019! We need Speakers & Sponsors for 2020 PASSMN Meetings! Sign up to present at one of the monthly meetings! Monthly Meetup: 3rd Tuesday of Each Month (except Oct) at Microsoft MTC in Edina (food usually provided) Signup on Meetup: https://www.meetup.com/MN-SQL-Server-User-Group-PASSMN/ Board Member Elections in November/December: Your chance to help out the MN SQL community!

November 3th Through November 8th Join the brightest data professionals focused on the Microsoft Data Platform! November 3th Through November 8th Pre-Conference Sessions – Monday/Tuesday Conference – Wednesday through Friday No Kevin this year. :-(

SQLSaturday #913 – After Party Location: 4th Floor of Mall of America Time: 6:30PM – 10:00PM There will be drinks and appetizers, as well as free game cards and bowling! Hang out with some new friends you’ve made.

A bit about me Technical solutions engineer at Epic Previously: math & stats teacher Working with SQL Server since 2015 And sometimes Oracle Interested in how databases manage and use resources Virtualization & operating systems, too! Live in Madison with my wife and three-year-old +Two cats :-/

Why am I interested in partitioning? I’m interested in how databases use resources. Our R&D team has been doing work with partitioning recently, so I wanted to learn. I think what I learned might be useful for others.

An Introduction to Partitioning

By the end of this session, you will be able to... Configure partitioned tables. Explain the basics of how partitioning can affect queries. Switch partitions in and out of tables.

What’s the problem? Large tables can be difficult to work with. Examples: Loading/modifying data Maintenance Query performance

Partitioning on its own does not solve these problems. If used purposefully, it can help.

The Big Idea Partition the table horizontally. Store the partitions separately. Work at the level of a partition instead of a table.

So, how will this work? Choose a column to partition on. Map rows to partition. Map partitions to storage.

Partitioning column The column that determines which partition a row belongs in. Choose a column that makes sense based on the work that is done with this table.

Partition function Defines the range of values for each partition. Choose whether boundaries are included in partition to left or right.

Partition scheme Maps the partitions to filegroups. Example reason for mapping to different filegroups: backups!

Summary Partitioning column Use values from this column to partition. Partition function Maps rows to partitions. Partition scheme Maps partitions to filegroups.

[Demo] Query Execution

Goal Develop an understanding of how partitioning affects query execution.

Approach Query tables that are and are not partitioned. Look at I/O stats to infer how query execution changes. Do this for heaps, clustered rowstores, and clustered columnstores.

Review of table structures Heap Pages contain rows. No particular order to storage. Clustered rowstore Pages contain rows. Data sorted by key(s) with a B-tree. Clustered columnstore Pages contain columns. No particular order to storage. Subsets of rows are stored in compressed rowgroups.

Demo!

Partition elimination The process by which the query optimizer accesses only the relevant partitions to satisfy the filter criteria of the query. This is an opportunity to choose the partitioning column wisely.

Summary Partitioning column Use values from this column to partition. Partition function Maps rows to partitions. Partition scheme Maps partitions to filegroups. Partition elimination The query optimizer can use partition ranges in planning.

Partition Switching

Partition Switching Idea Partitioning breaks a table into horizontal chunks. The table that a partition is associated with is metadata. So, we can “move” partitons in and out of tables as metadata-only operations. That means “fast”. 

Partition Switching Demo Pull out data from the 1980’s from dbo.OrdersColumnstorePartitioned by partition switching. Modify the order IDs in the switch out table. Switch the partition back in. We’re going to bump into a few issues.

[Demo] Partition Switching

Partition Switching Requirement Ideas Tables have same structure. Tables are in the same filegroup. SQL Server “knows” the data in the source partition matches the target partition. There’s nuance here that I’m not getting into.

Summary Large tables can be a challenge. Partitioning can help you work with them. Partitioning structures and stores the data differently. You can use this to improve query performance. You can use this to switch partitions in and out. Partitioning isn’t magic.

References Cathrine Wilhemsen Andrew Pruski Jake Manske Series: Table Partitioning in SQL Server Andrew Pruski PASS Summit presentation (2018) Jake Manske

Thank you! Kevin Krenz kevin.krenz@gmail.com