1. Honest Bob’s Cube Processing Bob Duffy Database Architect Prodata SQL Centre of Excellence 18 th July, 2014

2. Bob Duffy 20 years in database sector, 250+ projects 20 years in database sector, 250+ projects Senior Consultant with Microsoft 2005-2008 Senior Consultant with Microsoft 2005-2008 One of about 25 MCA for SQL Server globally (aka SQL Ranger) One of about 25 MCA for SQL Server globally (aka SQL Ranger) SQL MCM on SQL 2005 and 2008 SQL MCM on SQL 2005 and 2008 SQL Server MVP 2009+ SQL Server MVP 2009+ SSAS Maestro SSAS Maestro Database Architect at Prodata SQL Centre of Excellence Database Architect at Prodata SQL Centre of Excellence http://blogs.prodata.ie/author/bob.aspx http://blogs.prodata.ie/author/bob.aspx http://blogs.prodata.ie/author/bob.aspx bob@Prodata.ie bob@Prodata.ie bob@Prodata.ie @bob_duffy

3. What we Will Cover How Processing Works Tips to Optimise Processing Benchmarking Tools Processing Options and Strategies Tips for Tabular Models

4. What's a cube ?

5. Dimension Processing

6. Measure Group Processing

7. Recap: Tuning Attribute Properties Key Column Name Column AttributeHierarchyEnabledAttributeHierarchyOptimizedAttributeHierarchyOrdered Add correct relationships Avoid Large Dimensions Or DELETE THEM

8. Determining Size of Objects  http://blogs.prodata.ie/post/Tool-for-baselining-MOLAP-Processing.aspx http://blogs.prodata.ie/post/Tool-for-baselining-MOLAP-Processing.aspx

9. Slow Execute SQL ? 1. Check WAIT Statistics What does ASYNC_NETWORK mean ? What does ASYNC_NETWORK mean ? 2. For Dimensions Optimise for Select Distinct Optimise for Select Distinct Or Switch “by Table Mode” Or Switch “by Table Mode” 3. For Facts 1. Ensure narrow base table (no joins!) 2. Optimise Scan speed 512k Read Ahead 512k Read Ahead Fast Disks Fast Disks Good File Group Layout Good File Group Layout No Extent Fragmentation (-E trace flag) No Extent Fragmentation (-E trace flag)

10. Processing Tuning Approach Measure Object Sizes Who is hogging space Measure Process Time Where is Timing Being Spent Attributes/Dimensions Process Data Bitmap Indexes Aggregations Optimise the Cube/Dimensions Tune the Engine/Hardware Repeat Baseline

11. Process Data Review Grain Increase BufferRecordLimit Or Aggregate in SQL Too many Measures ? Use Calculated Measures Do we have disk bottleneck Do we have Network bottleneck Are we consuming a lot of CPU Match SSAS to SQL Types Idle Server Resources Increase Parallelism

12. Optimising Bitmap Indexes 1. Optimise Attributes on Dimensions 2. Ensure good CPUs ! 3. Ensure max parallelism 4. Don’t over partition 5. Will sorting and Compression Help ?

13. Optimising Compression by Sorting Each Segment in MOLAP is 64k http://dennyglee.com/2013/09/30/analysis-services- multidimensional-it-is-the-order-of-things/ http://dennyglee.com/2013/09/30/analysis-services- multidimensional-it-is-the-order-of-things/

14. Optimising Aggregations Use correct Attribute Relationships Don’t over aggregate (say 20) Consider consolidating aggregations Avoid spilling to disk Ensure max parallelism CPU should be saturated!

15. Benchmarking Processing Time Use XMLA to Process interesting objects Use XMLA to capture a trace file Use logman to capture perfmon counters Disk throughput and latency CPU SSAS Threads Automate with a batch file

16. Analysing the Trace File  http://blogs.prodata.ie/post/Tool-for-baselining-MOLAP-Processing.aspx http://blogs.prodata.ie/post/Tool-for-baselining-MOLAP-Processing.aspx

17. Case Study – Tuning AW Size Before 8.8 GB Data, 1.2 GB Index Size After 8.5 GB, 0.8 GB Index

18. Dimension / Fact Structures ExtensionDescription Kstore, Asstore, Ahstore, bStore, Ahstore Attribute Stored DstoreHierarchy Decode Store lStoreHierarchy Child/Parent Structure SstoreSet Store. Ancestors of a level oStoreOrder Store. Contains Position of each level. BstoreBlob Store MapBitmap Index ExtensionDescription Fact.dataRaw Fact Data Map, HdrBitmap index Files Agg.flex.dataFlexible Aggregations Aff.Rigid.dataRigid Aggregations

19. Processing Options/Strategy Process Full Drops related Fact data! Process Update May drop flexible Aggregations and Indexes Will bloat over time Process Add Fastest Processing Option Not available via GUI Other Process Commands ProcessData/IndexProcessStructureProcessClear

20. Perfmon Counters ObjectCounter Processor% Processor Time Logical DiskAvg. Disk sec/Transfer Disk Transfers/Sec ProcessingRows Read/sec Proc IndexRows/sec Proc AggregationsRows Created/Sec Temp file bytes written/sec Current Partitions ThreadsProcessing pool idle threads Processing pool job queue length Processing pool busy threads MemoryMemory Usage Kb High/Low too

21. ENGINE TUNING TIPS

22. Connection String Check Data Source Try OLEDB driver instead of SqlClient Max TCP/IP Packet Size set to 32,756 Up default connections from 10 Don’t use TCP/IP on shared server

23. INI Settings See Analysis Services Performance Tuning paper INI SettingImpact OLAP\Process\ AggregationMemoryLimitMin AggregationMemoryLimitMax Increase Max if spill to disk Decrease Min if lots of partitions ThreadPool\Process\MaxThreadsIncrease if maxed out CoordinatorExecutionModeIncrease Threads per core BufferMemoryLimitImprove data grouping TempDirCan place on faster disk

24. Processing Options & Strategy Dimensions Process Full ProcessDataProcessIndex Process Update ProcessAdd Cube/Measure Group ProcessFullProcessDataProcessIndexProcessAdd Can we use ROLAP ?

25. THE TABULAR MODEL Cube Processing Deep Dive

26. Tabular Processing Events

27. Tabular Processing Types ProcessFullProcessDataProcessAddProcessRecalcProcessDefragProcessClear

28. Baselining Tabular Processing  http://blogs.prodata.ie/post/Baselining-Tabular-Model-Processing-with-a-trace-file.aspx http://blogs.prodata.ie/post/Baselining-Tabular-Model-Processing-with-a-trace-file.aspx

29. Sizing Tool  http://www.powerpivotblog.nl/what-is-using-all-that-memory-on-my-analysis-server-instance/ http://www.powerpivotblog.nl/what-is-using-all-that-memory-on-my-analysis-server-instance/

30. Optimising Columns Don’t store unused columns like fact keys Reduce Precision Split high cardinality columns Sort we improve sorting ProcessingTimeBoxSecPerMRow Review Segment Size DefaultSegmentRowCount Smaller may process faster in parallel Larger may compress better

31. Other Tips for tabular Replace simple calculated columns with DAX Optimise the ExecuteSQL for table scan Replace multiple ProcessFull with ProcessData and Process Recalc Avoid NUMA hardware

32. Should we Partition ? CANNOT be processed in parallel No support for partition elimination

33. Thank You

34. THE CRAIC SQL SATURDAY #310 DUBLIN STAY Hilton Hotel Charlemont – rooms from €159 B&B REGISTER http://www.sqlsaturday. com/310/ LEARN FROM INTERNATIONAL EXPERTS - -Sept 19th: 6 x Precons - -Sept 20th: 6 x Tracks – BI,DEV,DBA,2014,Cloud 20th Sept DUBLIN PARTY After party and entertainment during the event

36. Tabular Disk Structures ExtensionDescription XMLMeta Data IDFCompressed Column Store or Index DICTIONARYMap of DataID to Value HIDXAttribute Hierarchy

37. HARDWARE Selecting hardware for Analysis Servics

38. Considerations for HW SSAS does not perform well on NUMA Very sensitive to CPU and memory FE is single threaded Disk speed not as important for Tabular The intel E5/7 v2 chips are almost a must!  http://www.sqlperformance.com/2014/01/system-configuration/selecting-a-processor-for-sql-server-2014-1 http://www.sqlperformance.com/2014/01/system-configuration/selecting-a-processor-for-sql-server-2014-1  http://blogs.prodata.ie/post/Selecting-Hardware-for-Analysis-Services-(10GB-1TB-size).aspx http://blogs.prodata.ie/post/Selecting-Hardware-for-Analysis-Services-(10GB-1TB-size).aspx

39. Scale Out MOLAP does not really “scale up” for querying