1. Garrett Edmondson Data Warehouse Architect Blue Granite Inc. GEdmondson@blue-granite.com http://garrettedmondson.wordpress.com/

2. DW vs. OLTP Data Warehouse – Scan Centric (sequential reads/writes) MB/sec – Nonvolatile Data – nightly loads – Index – Light Few covering Clustered Indexes – Low Concurrency  OLTP  Seek Centric (Radom Read/Writes) IOPs  Volatile Data  Index Heavy  Many Heap Tables  High Concurrency

3. Enterprise Shared SAN Storage Dedicated Network Bandwidth Traditional SQL DW Architecture Shared Infrastructure Data Warehouse Speed = THROUGHPUT from storage to CPU DW

4. The DW Trinity Star Schema (Fact & Dimension) – The Data Warehouse Toolkit: The Complete Guide to Dimensional Modeling (Second Edition)The Data Warehouse Toolkit: The Complete Guide to Dimensional Modeling (Second Edition) Balanced Architecture – Parallel Data Warehouse ColumnStore Indexes - SQL 2012 SQL Server Columnstore Index FAQ – “Batch” Mode – Page Compression

5. Potential Performance Bottlenecks FC HBA FC HBA A A B B FC HBA FC HBA A A B B FC SWITCH STORAGE CONTROLLER STORAGE CONTROLLER A A B B A A B B CACHE SERVER CACHE SQL SERVER WINDOWS CPU CORES CPU Feed Rate HBA Port Rate Switch Port Rate SP Port Rate A A B B DISK LUN DISK LUN SQL Server Read Ahead Rate SQL Server Read Ahead Rate LUN Read Rate Disk Feed Rate

6. Balanced System IO Stack CPU Socket (4 Core) CPU Socket (4 Core) CPU Socket (4 Core) CPU Socket (4 Core) SMP Scale-Up for Data Growth!!! Complex Costly Obsolete Hardware Tune every IO Interface !

7. CPU Maximum Consumption Rate (MCR) Theoretical Max (MCR) – From Cache Query Set STATISTICS IO and STATISTICS TIME to ON MAXDOP = 4 ( [Logical reads] / [CPU time in seconds] ) * 8KB / 1024 = MCR per core Benchmark (BCR) – From Disk - Cache DBCC dropcleanbuffers Set STATISTICS IO and STATISTICS TIME to ON MAXDOP 8+ ( [Logical reads] / [CPU time in seconds] ) * 8KB / 1024

8. MB/sec Test Storage Throughput in MB/sec – SQLIO – Sequential Reads and Writes Important Establish real rather than rated, performance metrics for the key hardware components of the Fast Track reference architecture!!!!

9. LUN Configuration

10. Fast Track Data Striping ARY01D1v01ARY01D1v01 ARY01D2v02ARY01D2v02 ARY02D1v03ARY02D1v03 ARY02D2v04ARY02D2v04 ARY03D1v05ARY03D1v05 ARY03D2v06ARY03D2v06 ARY04D1v07ARY04D1v07 ARY04D2v08ARY04D2v08 ARY05v09ARY05v09 DB1-1.ndfDB1-1.ndf DB1-7.ndfDB1-7.ndf DB1-5.ndfDB1-5.ndf DB1-3.ndfDB1-3.ndf DB1-2.ndfDB1-2.ndf DB1-4.ndfDB1-4.ndfDB1-6.ndfDB1-6.ndfDB1-8.ndfDB1-8.ndf DB1.ldfDB1.ldf Storage Enclosure

11. Evaluating Page Fragmentation Average Fragment Size in Pages – This metric is a reasonable measure of contiguous page allocations for a table. Value should be >= 400 for optimal performance select db_name(ps.database_id) as database_name,object_name(ps.object_id) as table_name,ps.index_id,i.name,cast (ps.avg_fragment_size_in_pages as int) as [Avg Fragment Size In Pages],ps.fragment_count as [Fragment Count],ps.page_count,(ps.page_count * 8)/1024/1024 as [Size in GB] from sys.dm_db_index_physical_stats (DB_ID() --NULL for all DBs else run in context of DB, OBJECT_ID(‘dbo.lineitem’), 1, NULL, ‘SAMPLED’) AS ps --DETAILED, SAMPLED, NULL = LIMITED inner join sys.indexes AS i on (ps.object_id = i.object_id AND ps.index_id = i.index_id) where ps.database_id = db_id() and ps.index_level = 0

12. (c) 2011 Microsoft. All rights reserved. TRACE FLAG –E !!!!!! Trace FLAG -T1117

13. SQL Server Configurations Sequential scan performance starts with database creation and extent allocation Recall that the –E startup option is used – Allocate 64 extents at a time (4MB) Pre-allocation of user databases is strongly recommended Autogrow should be avoided if possible – If used, always use 4MB increments

14. Storage Layout Best Practices for SQL Server Create a SQL data file per LUN, for every filegroup TempDB filegroups share same LUNs as other databases Log on separate disks, within each enclosure – Striped using SQL Striping – Log may share these LUNs with load files, backup targets

15. Storage Layout Best Practices for SQL Server LUN16 LUN 2LUN 3 Local Drive 1 Log LUN 1 Permanent DB Log LUN 1 TempDB TempDB.mdf (25GB)TempDB_02.ndf (25GB)TempDB_03ndf (25GB)TempDB_16.ndf (25GB) Permanent FG Permanent_1.ndf Permanant_DB Stage Database Stage FG Stage_1.ndfStage_2.ndfStage_3.ndf Stage_16.ndf Stage DB Log Permanent_2.ndf Permanent_3.ndf Permanent_16.ndf

16. Techniques to Maximize Scan Throughput –E startup parameter (2MB Extents and not mixed extents) Minimize use of NonClustered indexes on Fact Tables Load techniques to avoid fragmentation – Load in Clustered Index order (e.g. date) when possible Index Creation always MAXDOP 1, SORT_IN_TEMPDB Isolate volatile tables in separate filegroup Isolate staging tables in separate filegroup or DB Periodic maintenance Turn on SQL Server Compression

17. Conventional data loads lead to fragmentation Bulk Inserts into Clustered Index using a moderate ‘batchsize’ parameter – Each ‘batch’ is sorted independently… causes fragmentation Overlapping batches lead to page splits 1:32 1:31 1:35 1:34 1:33 1:36 1:38 1:37 1:40 1:39 1:32 1:31 1:35 1:34 1:33 Key Order of Index

18. Best Practices for loading Use a heap – Practical if queries need to scan whole partitions or…Use a batchsize = 0 – Fine if no parallelism is needed during load or…Use a Two-Step Load 1.Load to a Staging Table (heap) 2.INSERT-SELECT from Staging Table into Target CI Resulting rows are not fragmented Can use Parallelism in step 1 – essential for large data volumes

19. Other fragmentation best practices Avoid Autogrow of filegroups – Pre-allocate filegroups to desired long-term size – Manually grow in large increments when necessary Keep volatile tables in a separate filegroup – Tables that are frequently rebuilt or loaded in small increments If historical partitions are loaded in parallel, consider separate filegroups for separate partitions to avoid extent fragmentation

20. Columnstore Indexes storage = Segments + Dictionaries CPU = Batch Mode (c) 2011 Microsoft. All rights reserved.

21. Row Storage Layout IDNameAddressCityStateBal Due 1Bob………3,000 2Sue………500 3Ann………1,700 4Jim………1,500 5Liz………0 6Dave………9,000 7Sue………1,010 8Bob………50 9Jim………1,300 1Bob………3,000 2Sue………500 3Ann………1,700 4Jim………1,500 5Liz………0 6Dave………9,000 7Sue………1,010 8Bob………50 9Jim………1,300 Customers Table page Extent

22. Column Storage Layout IDNameAddressCityStateBal Due 1Bob………3,000 2Sue………500 3Ann………1,700 4Jim………1,500 5Liz………0 6Dave………9,000 7Sue………1,010 8Bob………50 9Jim………1,300 Customers Table ID 1 2 3 4 5 6 7 8 9 Name Bob Sue Ann Jim Liz Dave Sue Bob Jim Address … … … … … … … … … City … … … … … … … … … State … … … … … … … … … Bal Due 3,000 500 1,700 1,500 0 9,000 1,010 50 1,300 Segment = 1 million row chunks

23. Run Length Encoding (RLE) Quarter Q1 … Q2 … QuarterStartCount Q11310 Q2311290 ……… ProdIDStartCount 115 263 ……… 1515 2563 ProdID 1 1 1 1 1 2 2 2 … 1 1 1 1 1 2 2 2 Price 100 120 315 100 315 198 450 320 150 256 450 192 184 310 251 266 Price 100 120 315 100 315 198 450 320 150 256 450 192 184 310 251 266 RLE Compression applied only when size of compressed data is smaller than original

24. xVelocity Store Dictionary Encoding Quarter Q1 Q2 … Q3 Q4 … Only 4 values. 2 bits are enough to represent it DISTINCT Q.IDQuarter 0Q1 1Q2 2Q3 3Q4 Q.ID 1 1 1 1 2 2 … 2 3 3 3 3 4 4 4 4 … R.L.E. Q.IDStartCount 114 2510 3114 415

25. CPU Architecture “Batch” Mode Modern CPUs have many Cores Cache Hierarchies: RAM L3,L2,L1 – Small L1 and L2 per core; L3 shared by socket – L1 faster than L2, L2 faster than L3 – CPUs stall when waiting for caches to load Batch Mode sizes instructions & data to fit into L2/L1 cache !

26. Parallel Data Warehouse Overview

27. Data Warehouse appliances A prepackaged or pre- configured balanced set of hardware (servers, memory, storage and I/O channels), software (operating system, DBMS and management software), service and support, sold as a unit with built-in redundancy for high availability positioned as a platform for data warehousing.

28. Control Node Failover Protection: Redundant Control Node Redundant Compute Node Cluster Failover Redundante Array of Inexpensive Databases Spare Node

29. Parallel Data Warehouse Appliance - Hardware Architecture Database Servers Dual Infiniband Control Nodes Active / Passive Landing Zone Backup Node Storage Nodes Spare Database Server Dual Fiber Channel Management Servers Client Drivers ETL Load Interface Corporate Backup Solution Data Center Monitoring Corporate Network Private Network

30. PDW Data Example Time Dim Date Dim ID Calendar Year Calendar Qtr Calendar Mo Calendar Day Date Dim ID Calendar Year Calendar Qtr Calendar Mo Calendar Day Store Dim Store Dim ID Store Name Store Mgr Store Size Store Dim ID Store Name Store Mgr Store Size Product Dim Prod Dim ID Prod Category Prod Sub Cat Prod Desc Prod Dim ID Prod Category Prod Sub Cat Prod Desc Mktg Campaign Dim Mktg Campaign Dim Mktg Camp ID Camp Name Camp Mgr Camp Start Camp End PDW Compute Nodes Sales Facts Date Dim ID Store Dim ID Prod Dim ID Mktg Camp Id Qty Sold Dollars Sold

31. Time Dim Date Dim ID Calendar Year Calendar Qtr Calendar Mo Calendar Day Date Dim ID Calendar Year Calendar Qtr Calendar Mo Calendar Day Sales Facts Date Dim ID Store Dim ID Prod Dim ID Mktg Camp Id Qty Sold Dollars Sold PDW Data Example Store Dim Store Dim ID Store Name Store Mgr Store Size Store Dim ID Store Name Store Mgr Store Size Product Dim Prod Dim ID Prod Category Prod Sub Cat Prod Desc Prod Dim ID Prod Category Prod Sub Cat Prod Desc Mktg Campaign Dim Mktg Campaign Dim Mktg Camp ID Camp Name Camp Mgr Camp Start Camp End PD TD MD SD PD TD MD SD PD TD MD SD PD TD MD SD Smaller Dimension Tables are Replicated on Every Compute Node

32. PDW Data Example Time Dim Date Dim ID Calendar Year Calendar Qtr Calendar Mo Calendar Day Date Dim ID Calendar Year Calendar Qtr Calendar Mo Calendar Day Store Dim Store Dim ID Store Name Store Mgr Store Size Store Dim ID Store Name Store Mgr Store Size Product Dim Prod Dim ID Prod Category Prod Sub Cat Prod Desc Prod Dim ID Prod Category Prod Sub Cat Prod Desc Sales Facts Date Dim ID Store Dim ID Prod Dim ID Mktg Camp Id Qty Sold Dollars Sold Mktg Campaign Dim Mktg Campaign Dim Mktg Camp ID Camp Name Camp Mgr Camp Start Camp End PD TD MD SD PD TD MD SD PD TD MD SD PD TD MD SD SF-1 SF-2 SF-3 SF-4 Larger Fact Table is Hash Distributed Across All Compute Nodes SF-1 SF-2 SF-3 SF-4

33. SMP: Scale-Up Complex – Tune Every IO interface Cost – exponential Obsolete Hardware MPP: Scale-Out Simple – Buy more Processing Nodes Cost – linear Keep all hardware investments SMP vs. MPP

34. SQL Server Parallel Data Warehouse A quick look at MPP query execution SQL Server PDW Appliance

35. Shuffle Movement DMS Redistributes the data by color values in parallel. Compute Node 1 Compute Node 2 Dealing with Distributions - Shuffling Example: Select [color], SUM([qty]) from [Store Sales] group by [color]; Example: Select [color], SUM([qty]) from [Store Sales] group by [color]; Return Ss_i d colorqty Store Sales 1 Red5 3 Blue11 5 Red12 7 Green7 Ss_i d colorqty Store Sales 2 Red8 4 Blue10 6 Yellow12 Distributed Table Temp_1 Red5 12 Red8 Green7 Temp_1 Blue11 Yellow12 Blue10 colorqty colorqty Hash Blue21 Red25 Green7 Yellow12 colorqty Hash Parallel Merge and Aggregate

36. Legend: Control Node Client Interface (JDBC, ODBC, OLE-DB, ADO.NET) Client Interface (JDBC, ODBC, OLE-DB, ADO.NET) DMS Manager PDW Engine … Compute Node 1 DMS Core PDW Agent Landing Zone Node Bulk Data Loader PDW Agent Management Node Active Directory PDW Agent Compute Node 2 DMS Core PDW Agent Compute Node 10 DMS Core PDW Agent PDW service Data Movement ServiceDMS= Parallel Data WarehousePDW= ETL Interface ETL Interface Data Rack (up to 4)Control Rack

37. (c) 2011 Microsoft. All rights reserved.

38. BI, Analytics, & ETL Integration Performance At Scale Broader functionality Full Alignment SQL Server Compatibility Less work for the same results Do the same work more efficiently Native Support for -Analysis Services -Reporting Services -PowerPivot Lay the foundation for broad connectivity support

39. Problem – Basic RDBMS functionality, that already exists in SQL Server, was re-built in PDW  Challenge for PDW AU3 release – Can we leverage SQL Server and focus on MPP related challenges? Control Node

40. SQL Server PDW AU3 Architecture PDW AU3 Architecture with Shell Appliance and Cost-Based Query Optimizer SQL Server runs a ‘Shell Appliance’ Every database exists as an empty ‘shell’ All objects, no user data DDL executes against both the shell and the compute nodes Large parts of basic RDBMS functionality now provided by the shell Authentication and authorization Schema binding Metadata catalog Shell Appliance (SQL Server) Shell Appliance (SQL Server) Engine Service Plan Steps Compute Node (SQL Server) Compute Node (SQL Server) Compute Node (SQL Server) Compute Node (SQL Server) Compute Node (SQL Server) Compute Node (SQL Server) Control Node SELECT foo

41. SQL Server PDW AU3 Architecture PDW AU3 Architecture with Shell Appliance and Cost-Based Query Optimizer 1.User issues a query 2.Query is sent to the Shell through sp_showmemo_xml stored procedure – SQL Server performs parsing, binding, authorization – SQL optimizer generates execution alternatives 3.MEMO containing candidate plans, histograms, data types is generated 4.Parallel execution plan generated 5.Parallel plan executes on compute nodes 6.Result returned to the user Shell Appliance (SQL Server) Shell Appliance (SQL Server) Engine Service Plan Steps MEMO Compute Node (SQL Server) Compute Node (SQL Server) Compute Node (SQL Server) Compute Node (SQL Server) Compute Node (SQL Server) Compute Node (SQL Server) Control Node SELECT Return

42. PDW Cost-Based Optimizer Optimizer lifecycle… 1. Simplification and space exploration – Query standardization and simplification (e.g. column reduction, predicates push- down) – Logical space exploration (e.g. join re-ordering, local/global aggregation) – Space expansion (e.g. bushy trees – dealing with intermediate resultsets) – Physical space exploration – Serializing MEMO into binary XML (logical plans) – De-serializing binary XML into PDW Memo 2. Parallel optimization and pruning – Injecting data move operations (expansion) – Costing different alternatives – Pruning and selecting lowest cost distributed plan 3. SQL Generation – Generating SQL Statements to be executed

43. PDW Cost-Based Optimizer … And Cost Model Details PDW cost model assumptions: – Costing only data movement operations (relational operations excluded) – Sequential step execution (no pipelined and independent parallelism) Data movement operations consist of multiple tasks Each task has Fixed and Variable overhead Uniform data distribution assumed (no data skew)

44. PDW Sales Test Workload AU2 to AU3 5x improvement in terms of total elapsed time out of the box Seconds Queries

45. Theme: Performance at Scale Zero data conversions in data movement Goal Eliminate CPU utilization spent on data conversions Further parallelize operations during data moves Functionality Using ODBC instead of ADO.NET for reading and writing data Minimizing appliance resource utilization for data moves Benefits Better resource, CPU, utilization 6x or more faster move operations Increased concurrency Mixed workload (loads + queries)

46. Theme: SQL Server Compatibility SQL Server Security and Metadata Security SQL Server security syntax and semantics Supporting user, roles and logins Fixed database roles Allows script re-use Allows well-known security methods Metadata PDW metadata stored in SQL Server Existing SQL Server metadata tables/views (e.g. security views) PDW distribution info as extended properties in SQL Server metadata Existing means and technology for persisting metadata Improved 3 rd party tool compatibility (BI, ETL)

47. Theme: SQL Server Compatibility Support for SQL Server (Native) Client Goal ‘Look’ just like a normal SQL Server Better integration with other BI tools Functionality Use existing SQL Server drivers to connect to SQL Server PDW Implement SQL Server TDS protocol Named Parameter support SQLCMD connectivity to PDW Benefits Use known tools and proven technology stack Existing SQL Server ’eco-system’ 2x performance improvement for return operations 5x reduction of connection time SQL PDW Clients (ODBC, OLE-DB, ADO.NET) SQL Server Clients (ADO.NET, ODBC, OLE-DB, JDBC) TDS Server: 10.217.165.13, 17001 Server: 10.217.165.13, 17000 SequeLink

48. Theme: SQL Server Compatibility Stored Procedure Support (Subset) Goal Support common scenarios of code encapsulation and reuse in Reporting and ETL Functionality System and user-defined stored procedures Invocation using RPC or EXECUTE Control flow logic, input parameters Benefits Enables common logic re-use Big impact for Reporting Services scenarios Allows porting existing scripts Increases compatibility with SQL Server Syntax CREATE { PROC | PROCEDURE } [dbo.]procedure_name [ { @parameter data_type } [ = default ] ] [,...n ] AS { [ BEGIN ] sql_statement [;] [...n ] [ END ] } [;] ALTER { PROC | PROCEDURE } [dbo.]procedure_name [ { @parameter data_type } [ = default ] ] [,...n ] AS { [ BEGIN ] sql_statement [;] [...n ] [ END ] } [;] DROP { PROC | PROCEDURE } { [dbo.]procedure_name } [;] [ { EXEC | EXECUTE } ] { { [database_name.][schema_name.]procedure_name } [{ value | @variable }] [,...n ] } [;] { EXEC | EXECUTE } ( { @string_variable | [ N ]'tsql_string' } [ +...n ] ) [;] Unsupported Functionality Stored Proc Nesting Output Params Return Try-Catch

49. Theme: SQL Server Compatibility Collations Goal Support local and international data Functionality Fixed server level collation User-defined column level collation Supporting all Windows collations Allow COLLATE clauses in Queries and DML Benefits Store all the data in PDW w/ additional querying flexibility Existing T-SQL DDL and Query scripts SQL Server alignment and functionality Syntax CREATE TABLE T ( c1 varchar(3) COLLATE traditional_Spanish_ci_ai, c2 varchar(10) COLLATE …) SELECT c1 COLLATE Latin1_General_Bin2 FROM T SELECT * FROM T ORDER BY c1 COLLATE Latin1_General_Bin2 Unsupported Functionality  Cannot specify DB collation during DB creation  Cannot alter column collations for existing tables

50. Theme: Improved Integration SQL Server PDW Connectors Connector for Hadoop Bi-directional (import/export) interface between MSFT Hadoop and PDW Delimited file support Adapter uses existing PDW tools (bulk loader, dwsql) Low cost solution that handles all the data: structured and unstructured Additional agility, flexibility and choice Connector for Informatica Connector providing PDW source and target (mappings, transformations) Informatica uses PDW bulk loader for fast loads Leverage existing toolset and knowledge Connector for Business Objects

51. Agenda Trends in the DW space How does SQL Server PDW fit in? SQL Server PDW AU3 – What’s new? Building BI Solutions with SQL Server PDW – Customer Successes – Using SQL Server PDW with Microsoft BI solutions – Using SQL Server PDW with third party BI solutions – BI solutions leveraging Hadoop integration What’s coming next in SQL Server PDW?

52. PDW Retail POS Workload Original Customer SMP solution vs. PDW AU3 (with cost-based query optimizer) Seconds Queries

53. Data Volume 80 TB data warehouse analyzing data from exchanges Existing system based on SQL SMP farm – 2 different clusters of 6 servers each Requirement Linear scalability with additional hardware Support hourly loads with SSIS – 300GB/day BI Integration: SSRS, SSAS and PowerPivot AU3 Feedback SP and increased T-SQL support was great Migrating SMP SSRS to PDW was painless 142x for scan heavy queries & no summary tables Enabled queries that do not run on existing system Portal ETL PDW Operational DB’s

54. Infiniban d GBit link

55. SSAS with SQL Server PDW Understanding the differences compared to ‘SMP world’

56. New Challenges for Business Analytics Huge amount of data born ‘unstructured’ Increasing demand for (near) real-time business analytics Pre-filtering of important from less relevant raw data required Applications Sensor networks & RFID Social networks & Mobile Apps Biological & Genomics Sensor/ RFID Data Blogs, Docs Web Data HADOOP

57. Fast ETL processing Active Archive Fast Refinery Cost-Optimal storage Hadoop as a Platform Solution In the context of ETL, BI, and DW Platform to accelerate ETL processes (not competing with current ETL software tools!) Flexible and fast development of ‘hand- written’ refining requests of raw data Active & cost effective data archive to let (historical) data ‘live forever’ Co-existence with a relational DW (not completely replacing it !)

58. Importing HDFS data into PDW for advanced BI HADOOP Sensor/ RFID Data Blogs, Docs Web Data SQL Server PDW Interactive BI/Data Visualization SQOOP Application Programmers DBMS Admin Power BI Users

59. Hadoop - PWD Integration via SQOOP (export) … Landing Zone Compute Node 1 Compute Node 8 HDFS … PDW- configuration file PDW Hadoop Connector SQOOP export with source (HDFS path) & target (PDW DB & table) 1. FTP Server Copies incoming data on Landing Zone 3.3. 2.2. Read HDFS data via mappers Invokes ‘DWLoader’ Telnet Server 4.4. Control Node Compute Nodes Window s/PDW Linux/ Hadoo p 5.5.

60. SQL Server PDW Roadmap What is coming next? Q1 Q2 Q3 Q4Q1 Q2 Improved node manageability Better performance and reduced overhead OEM requests Programmability Batches Control flow Variables Temp tables QDR infiniband switch Onboard Dell Columnar store index Stored procedures Integrated Authentication PowerView integration Workload management LZ/BU redundancy Windows 8 SQL Server 2012 Hardware refresh CALENDAR YEAR 2011 CALENDAR YEAR 2012 Cost based optimizer Native SQL Server drivers, including JDBC Collations More expressive query language Data Movement Services performance SCOM pack Stored procedures (subset) Half-rack 3rd party integration (Informatica, MicroStrategy, Business Objects, HADOOP) Q4 V-Next Appliance Update 3 Appliance Update 1 Shipped Appliance Update 2 Q3 Shippe d