Indexing Your Data Warehouse Troy Gallant, MTA

Agenda  A little about me  Indexing review  Enterprise Data Warehouse (EDW) vs. OLTP  EDW structure  EDW indexing  Too many / too few  Considerations  Dimension / fact indexing  Maintenance

Bio  15 years as a database professional  Last 2 yrs in NYC, all previous in Jax  Microsoft MTA certified  Speaker – 16x SQL Saturday, 4x JSSUG  Working on MS in IT Mgmt  Twitter:  LinkedIn:  Website: 

Indexing Review  Broad definition  What an index DOES do.  What an index DOESN’T do.

Types of Indexes  Heap*  Clustered  Non-clustered  Non-clustered w/ included columns  Unique  Full-text  Spatial  Filtered  XML  Columnstore

EDW vs. OLTP (pt. 1)  EDW definition  Single, complete, consistent  Decision-support  Integrate divergent information  Historical

EDW vs. OLTP (pt. 2)  Comparisons  Integrated data vs. application-specific  Current/Historical data vs. current data  Non-volatile vs. updated  Encoded vs. descriptive  Detailed/summarized vs. raw

EDW Structure  Source  Staging  Storage  Dimensions  Fact tables  Presentation

EDW Indexing (pt. 1)  Too few indexes  Data loads quickly  QRT suffers  Too many indexes  Data loads slowly  QRT improves  Storage requirements increase

EDW Indexing (pt. 2)  Major considerations  Warehouse type  Size of tables  Access  How?  Who?  What?  Storage requirements  Response-time expectations

EDW Indexing (pt. 3)  Dimensions  Clustered Index on business/natural key  Identifier from the source system  Enhances response time when this business key is used in a WHERE clause  NCI(s)  Surrogate key  Usually the primary key  Meaningful only to the source system  Will expedite loads  Other columns found to be accessed frequently in searches, sorting, or grouping  Consider columns included in a hierarchy

EDW Indexing (pt. 4)  Date & time dimensions  No business key  Consider a smart PK and cluster on it  YYYYMMDD  HHMMSSSS  A smart key will retain proper order and range queries will be simplified as you will need one less join because the PK already contains the date/time

EDW Indexing (pt. 5)  Type 2 SCD  Consider adding a 4-pt NCI that includes…  The business key  The record begin date  The record end date  The surrogate key  CREATE NONCLUSTERED INDEX MyDim_CoveringIndex ON (NaturalKEY, RecordStartDate) INCLUDE ( RecordEndDate, SurrogateKEY)  Can be very useful during ETL as well as for historical queries

EDW Indexing (pt. 6)  Fact table  Similar to indexing a dimension with an eye towards partitioning  Usually best to cluster on the date key or date/time key  If table is partitioned on a date column, use that column as the clustering key  Create NCI’s on each of the FK’s in the fact table  Consider combining the FK and date key (in that order) to enhance query response  Watch storage requirements

Modifying the Scheme  Over time your data warehouse will change to accommodate what’s happening in your organization  Use tried-and-true transactional methods for tuning indexes…  DTA  Execution plans  DMV’s  sys.dm_db_index_usage_stats  sys.dm_db_index_operational_stats  sys.dm_db_missing_index_details  sys.dm_db_missing_index_columns  sys.dm_db_missing_index_group_stats  sys.dm_db_missing_index_groups

Thank you!!! Twitter: LinkedIn: Web: