1. SQLintersection Putting the "Squeeze" on Large Tables Improve Performance and Save Space with Data Compression Justin Randall jrandall@sqlsentry.com Tuesday, 2:15-3:30

2. © SQLintersection. All rights reserved. http://www.SQLintersection.com Overview  Why compress data in SQL Server?  Data compression basics  Deciding what to compress  Planning your compression efforts  Resource requirements, options, and side effects  Summary

3. © SQLintersection. All rights reserved. http://www.SQLintersection.com Why Compress Data? Save Space on Disk & in Memory Improve Performance

4. © SQLintersection. All rights reserved. http://www.SQLintersection.com Enterprise Edition & Azure SQL DB Only! Reference: http://msdn.microsoft.com/en-us/library/cc645993(v=sql.120).aspx#Scalabilityhttp://msdn.microsoft.com/en-us/library/cc645993(v=sql.120).aspx#Scalability Scalability and Performance

5. © SQLintersection. All rights reserved. http://www.SQLintersection.com Data Compression Basics  Compression does not change syntax or semantics of data  Application changes are not required  Compression is applied to a table, index, or partition  A whole table stored as a heap  A whole table stored as a clustered index  A whole non-clustered index  A whole indexed view  Partition-specific compression on partitioned tables

6. © SQLintersection. All rights reserved. http://www.SQLintersection.com Data Compression Basics  In-row data only  Compression is applied at the data page level  Compressed pages remain compressed in memory  Fill factor is not affected  Special cases: Unicode and Columnstore  More rules and conditions in Books Online http://msdn.microsoft.com/en-us/library/cc280449.aspx http://msdn.microsoft.com/en-us/library/cc280449.aspx

7. © SQLintersection. All rights reserved. http://www.SQLintersection.com Data Compression Types: Row  A storage optimization technique  Reduces metadata overhead associated with the row  Stores fixed-length data types in variable-length storage format  Applies to leaf-level & non-leaf-level pages of indexes  Removes blank characters  NULL and 0 values require no additional space  Effect on Storage by data type  https://msdn.microsoft.com/en- US/library/cc280576%28v=sql.120%29.aspx https://msdn.microsoft.com/en- US/library/cc280576%28v=sql.120%29.aspx

8. © SQLintersection. All rights reserved. http://www.SQLintersection.com Data Compression Types: Row  Highest gains: data pages containing rows with a lot of unused space  NULL values  fixed-length character and numeric values not needing max defined space  Examples  char(100) column with most values < 50 – 75 characters  integer values substantially smaller than max size allowed

9. © SQLintersection. All rights reserved. http://www.SQLintersection.com Data Compression Types: Page  A superset of row compression  Seeks to minimize data redundancy  Applies equally to all data types  Applies to leaf-level pages of tables and indexes only  Pages must be full or nearly full before page level compression is attempted  Two techniques  Prefix compression  Dictionary compression

10. © SQLintersection. All rights reserved. http://www.SQLintersection.com Data Compression Types: Page  Prefix Compression  Per column, identify a "common" prefix  Row of all prefix values for a column is stored in CI structure of page  Repeated values in column replaced by reference to prefix in CI Example from SQL Server 2014 Books Online: https://msdn.microsoft.com/en-us/library/cc280464%28v=sql.120%29.aspx Prefix compression applied

11. © SQLintersection. All rights reserved. http://www.SQLintersection.com Data Compression Types: Page  Dictionary Compression  Follows prefix compression  Finds repeated values anywhere on the page  Repeated values replaced by reference to value in CI Example from SQL Server 2014 Books Online: https://msdn.microsoft.com/en-us/library/cc280464%28v=sql.120%29.aspx Dictionary compression applied

12. Data Compression Syntax Compress a Table, Partition, or Index ALTER TABLE T1 REBUILD with (Data_Compression = Row | Page | None) ALTER TABLE PT1 REBUILD PARTITION = 1 with (Data_Compression = Row | Page | None) ALTER INDEX IDX1 ON T1 REBUILD with (Data_Compression = Row | Page | None)

13. Demo Examining data compression's effect on data pages and data types

14. © SQLintersection. All rights reserved. http://www.SQLintersection.com Deciding What to Compress Questions to Ask

15. © SQLintersection. All rights reserved. http://www.SQLintersection.com Evaluate Potential Targets Compute Resources, SQL Server Instance, and Databases  SQL Server Edition  Database portability  Need for space savings  Database Files on Disk  Backup Files  Available CPU headroom  Maintenance window and available resources for initial compression and index rebuilds

16. © SQLintersection. All rights reserved. http://www.SQLintersection.com Evaluate Potential Targets Data Suitability  Is data likely to achieve high compression ratios?  Compressible data types  Empty space in columns  NULLS  Repeated values EXEC sp_estimate_data_compression_savings @Schema_Name = 'dbo', @Object_Name = 'Contact', @Index_ID = 1, @Partition_Number = NULL, @Data_Compression = 'ROW';

17. Demo Estimating compression ratios

18. © SQLintersection. All rights reserved. http://www.SQLintersection.com Evaluate Row vs. Page Compression Workload Suitability  Use row compression at a minimum when:  Compression results in (significant) space savings, and  System can accommodate < 10% increase in CPU utilization  Consider page compression when  Low % of update operations vs. total operations  High % of scans vs. total operations  Total operations = scans + DMLs + lookups

19. © SQLintersection. All rights reserved. http://www.SQLintersection.com Evaluate Workload SELECT ObjectName = object_schema_name(idx.object_id) + '.' + object_name(idx.object_id),IndexName = idx.name,IndexType = CASE WHEN is_unique = 1 THEN 'Unique ' ELSE '' END + idx.type_desc,User_Scans = us.user_scans,User_Lookups = us.user_lookups,User_Updates = us.user_updates FROM sys.indexes idx LEFT JOIN sys.dm_db_index_usage_stats us ON idx.object_id = us.object_id AND idx.index_id = us.index_id AND us.database_id = db_id() WHERE OBJECT_NAME (us.[object_id]) = 'customer' ORDER BY us.user_scans + us.user_updates DESC

20. © SQLintersection. All rights reserved. http://www.SQLintersection.com Decision Matrix Table.Index Savings ROW % Savings PAGE % SUDecisionNotes customer. customer_pk 12%26%77.7%22.2%PAGE Large, High S, Low U Big Savings customer. ix_customer_nationkey 29%34%34.1%65.9%ROW Large, Low S, High U Similar Savings customer. ix_customer_mktsegment_name 16%66%23.3%76.7%ROW Medium, Low S, High U Similar Savings customer. ix_customer_name_phone_acctbal 16%42%85%15%PAGE Medium, High S, Low U

21. Demo Evaluating workload suitability

22. © SQLintersection. All rights reserved. http://www.SQLintersection.com Resource Requirements  Compression requires workspace, CPU, and I/O  The mechanism is the same as rebuilding an index  Resource requirements depend on  What is being compressed (heap, CI, NCI)  Sort_In_Tempdb on or off  Online option on or off  Recovery Model

23. © SQLintersection. All rights reserved. http://www.SQLintersection.com Resource Requirements Resources Affected During Compression  Workspace  Free workspace needed in user db, transaction log, tempdb  I/O  Proportional to workspace used  CPU  Row compression uses about 1.5 times the CPU used for an index rebuild  Page compression 2 – 5 times the CPU used for an index rebuild

24. © SQLintersection. All rights reserved. http://www.SQLintersection.com Compression Operations Options  Online vs. offline  Offline is faster, requires less resources, but locks table for duration  Sequential vs. concurrent  Concurrent requires greater resources (workspace, I/O, CPU)  Order of operations  Smallest to largest so more free disk space is available for larger objects  Sort_In_Tempdb On/Off  On is recommended, as requires less workspace in user databases

25. © SQLintersection. All rights reserved. http://www.SQLintersection.com Compression Side Effects  Compression rebuilds a partition or index so fragmentation is removed  Compressing a heap rebuilds any nonclustered indexes  When ONLINE is OFF, indexes are rebuilt sequentially  When ONLINE is ON, indexes are rebuilt simultaneously  Space for an uncompressed heap is not released until index rebuild is complete  Compression frees space in the data file  Managing free space considerations apply as normal

26. © SQLintersection. All rights reserved. http://www.SQLintersection.com Other Impacts  Workload performance  Index rebuilds are slower, require more CPU  Bulk inserts are slower  Partition manipulation  TDE not affected

27. © SQLintersection. All rights reserved. http://www.SQLintersection.com References  SQL Server Books Online https://msdn.microsoft.com/en-US/library/cc280449(v=sql.120).aspx https://msdn.microsoft.com/en-US/library/cc280449(v=sql.120).aspx  White Paper: Data Compression: Strategy, Capacity Planning and Best Practices http://technet.microsoft.com/en-us/library/dd894051(v=sql.100).aspx http://technet.microsoft.com/en-us/library/dd894051(v=sql.100).aspx  A Look Inside SQL Server Row and Page Compression – Jes Borland http://blogs.lessthandot.com/index.php/datamgmt/dbprogramming/how-sql- server-data-compression/ http://blogs.lessthandot.com/index.php/datamgmt/dbprogramming/how-sql- server-data-compression/  Applying Compression to the SQL Sentry Database – Melissa Connors http://blogs.sqlsentry.com/melissaconnors/sql-sentry-data-compression-1/ http://blogs.sqlsentry.com/melissaconnors/sql-sentry-data-compression-1/

28. © SQLintersection. All rights reserved. http://www.SQLintersection.com References  Data Compression and Backup Compression – Chad Boyd https://www.mssqltips.com/sqlservertip/2253/sql-server-2008-data- compression-and-backup-compression/ https://www.mssqltips.com/sqlservertip/2253/sql-server-2008-data- compression-and-backup-compression/  Estimating Data Compression Ratios for All Partitions https://www.mssqltips.com/sqlservertip/2219/estimating-data-compression- ratios-for-all/ https://www.mssqltips.com/sqlservertip/2219/estimating-data-compression- ratios-for-all/  Estimating Data Compression Savings in SQL Server 2012 – Glenn Berry http://www.sqlskills.com/blogs/glenn/estimating-data-compression-savings-in- sql-server-2012/ http://www.sqlskills.com/blogs/glenn/estimating-data-compression-savings-in- sql-server-2012/

29. © SQLintersection. All rights reserved. http://www.SQLintersection.com Review  Why compress data in SQL Server?  Data compression basics  Deciding what to compress  Planning your compression efforts  Resource requirements, options, and side effects  Summary

30. Don’t forget to complete an online evaluation on EventBoard! Your evaluation helps organizers build better conferences and helps speakers improve their sessions. Questions? Thank you! Putting the "Squeeze" on Large Tables Improve Performance and Save Space with Data Compression