Honest Bob’s Cube Processing Bob Duffy Database Architect

Thank you to our sponsors!

Speaker - Bob Duffy  20 years in database sector, 250+ projects  Senior Consultant with Microsoft  One of about 25 MCA for SQL Server globally (aka SQL Ranger)  SQL MCM on SQL 2005 and 2008  SQL Server MVP  SSAS Maestro  Database Architect at Prodata SQL Centre of Excellence  

What we Will Cover  Processing Internals  Dimensions Deep Dive  Measure Group Deep Dive  Tuning Patterns  Baseline Harness  Case Study

What's a cube ?

Dimension Processing

Measure Group Processing

Recap: Tuning Attribute Properties  Key Column  Name Column  AttributeHierarchyEnabled  AttributeHierarchyOptimized  AttributeHierarchyOrdered  Add correct relationships  Avoid Large Dimensions  Or  DELETE THEM

Dimension / Fact Structures ExtensionDescription Kstore, Asstore, Ahstore, bStore, Ahstore Attribute Stored DstoreHierarchy Decode Store lStoreHierarchy Child/Parent Structure SstoreSet Store. Ancestors of a level oStoreOrder Store. Contains Position of each level. BstoreBlob Store MapBitmap Index ExtensionDescription Fact.dataRaw Fact Data Map, HdrBitmap index Files Agg.flex.dataFlexible Aggregations Aff.Rigid.dataRigid Aggregations

Determining Size of Objects

Slow Execute SQL ? 1.Check WAIT Statistics What does ASYNC_NETWORK mean ? 2.For Dimensions Optimise for Select Distinct Or Switch “by Table Mode” 3.For Facts 1.Ensure narrow base table (no joins!) 2.Optimise Scan speed 512k Read Ahead Fast Disks Good File Group Layout No Extent Fragmentation (-E trace flag)

Process Data  Review Grain  Increase BufferRecordLimit  Or Aggregate in SQL  Too many Measures ?  Use Calculated Measures  Do we have disk bottleneck  Do we have Network bottleneck  Are we consuming a lot of CPU  Match SSAS to SQL Types  Idle Server Resources  Increase Parallelism

Optimising Bitmap Indexes 1.Optimise Attributes on Dimensions 2.Ensure good CPUs ! 3.Ensure max parallelism 4.Don’t over partition 5.Will sorting and Compression Help ?

Optimising Compression by Sorting  Each Segment in MOLAP is 64k

Optimising Aggregations  Use correct Attribute Relationships  Don’t over aggregate (say 20)  Consider consolidating aggregations  Avoid spilling to disk  Ensure max parallelism  CPU should be saturated!

Benchmarking Processing Time  Use XMLA to Process interesting objects  Use XMLA to capture a trace file  Use logman to capture perfmon counters  Disk throughput and latency  CPU  SSAS Threads  Automate with a batch file

Analysing the Trace File

Processing Options/Strategy  Process Full  Drops related Fact data!  Process Update  May drop flexible Aggregations and Indexes  Will bloat over time  Process Add  Fastest Processing Option  Not available via GUI  Other Process Commands  ProcessData/Index  ProcessStructure  ProcessClear

Perfmon Counters ObjectCounter Processor% Processor Time Logical DiskAvg. Disk sec/Transfer Disk Transfers/Sec ProcessingRows Read/sec Proc IndexRows/sec Proc AggregationsRows Created/Sec Temp file bytes written/sec Current Partitions ThreadsProcessing pool idle threads Processing pool job queue length Processing pool busy threads MemoryMemory Usage Kb High/Low too

ENGINE TUNING TIPS

Connection String  Check Data Source  Try OLEDB driver instead of SqlClient  Max TCP/IP Packet Size set to 32,756  Up default connections from 10  Don’t use TCP/IP on shared server

INI Settings (before 2012)  See Analysis Services Performance Tuning paper INI SettingImpact OLAP\Process\ AggregationMemoryLimitMin AggregationMemoryLimitMax Increase Max if spill to disk Decrease Min if lots of partitions ThreadPool\Process\MaxThreadsIncrease if maxed out CoordinatorExecutionModeIncrease Threads per core BufferMemoryLimitImprove data grouping TempDirCan place on faster disk

Partitioning Strategies  For Processing  At least one partition per core  Try and make even size  Ensure fact.data smaller than index/agg data  Use Partitions to avoid updating all data

Case Study – Tuning AW  Size Before  8.8 GB Data, 1.2 GB Index  Size After  8.5 GB, 0.8 GB Index

Case Study II – Retail Cube Baseline  2.5 Hours Starting Point for 350GB cube

Case Study II – Data Types  All Numeric Converted to money or float in sql view.  30% decrease in time for ProcessData

Case Study II – Attribute/Aggs  53% overall reduction in cube size  55% reduction in Aggregations  73% reduction in size of Bitmap Indexes

Case Study II – Tuning the TSQL  Need to denormalise Fact Table

References  Excel Tool for Analysing Process Time   Excel tool for MOLAP disk space analysis   Scripts for baselining Cube Processing 

Your feedback is important! Please let us know what you thought of this session

Thank You

Near Real Time Processing  Process recent partitions frequently  Queries can Block Process Commit  Consider adjusting “ForceCommitTimeout”  Or “Cube flipping”  Tip: Only Consider ROLAP as a last resort

Scale Out  MOLAP does not really “scale up” for querying