1. Columnstore Index - is it the DW "Faster" switch you are looking for?
Johan Ludvig Brattås
Oslo – 22/2 2017

2. About me Johan Ludvig Brattås Johan-ludvig.brattas@capgemini.com
@intoleranse
Managing consultant, Capgemini Norway
SQLSaturday Oslo
Excel BI virtual chapter

3. Agenda Columnstore basics What’s new in 2016
What is Operational Analytics and In-Memory Analytics
When to use Clustered Columnstore
When to not use columnstore
Sofie

4. Columnstore basics First introduced in 2012. Version 0.4…
Current version 2016 = 0.9
Hybrid in-memory technology, based on xVelocity engine
Columnstore as opposed to rowstore.
Data grouped in rowgroups - 1,048,576 rows Minimum size 102,400 rows.
Columnsegments within each rowgroup 
Deltastore
Batchmode execution

5. What’s new in 2016?
Updateable nonclustered columnstore (a.k.a. Operational Analytics)
In-Memory Analytics
Nonclustered rowstore index on tables with Clustered Columnstore
Improved Batch mode functionality
Primary and Foreign Key support on tables with Clustered Columnstore Indexes
String Predicate Pushdown
Simple Aggregate Predicate Pushdown
Improved DMVs
Better index reorganzation
Batchmode-forbedringer: 1 core execution plans (for too low specced systems…)
Batch mode support for SORT operator Batch mode for Multiple Distinct Count
Support for LEFT SEMI ANTI JOINS (NOT EXISTS…)

6. What is Operational Analytics
Real-time analytics…
Only designed for non-clustered columnstore
The only real case for non-clustered columnstore usage
In-memory Analytics
Limitations: Adds 20% MORE memory usage
DEMO In-Memory Analytics

7. When to use Clustered Columnstore
Huge tables, like really gigantic. Ok?
Lots of columns
Star-schema/normalized fact tables
Tables you want to scan, not seek.
Demo

8. Some other examples on larger tables…
Existing index:
CREATE NONCLUSTERED INDEX [FactPR_Flybevegelse_ix] ON [dbo].[FactPunktlighetRegularitet] (
[FLYBEVEGELSE_ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF,
SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON [PRIMARY] GO
Result: ( row(s) affected) 00:00:27

9. Result: (154586 row(s) affected) 00:00:05
Reccomended index:
CREATE NONCLUSTERED INDEX [Test_Full]
ON [dbo].[FactPunktlighetRegularitet] ([PeriodeSk],[ISDELETED])
INCLUDE ([FactKey],[FlygningArtSk],[FlyplassSk],[FlyselskapSk],[FlytypeSk],[TimeSk],[ForsinkelseSk],
[OrganisasjonSk],[AvgAnkSk],[VektKlasseSk],[AvgAnkFlyplassSk],[KansellertSk],[GateSk],[HandlerSk],
[FlyoppstillingsplassSk],[AvgAnkFlyoppstillingsplassSk],[FLYBEVEGELSE_ID],[ACT_FLIGHT_KEY],[ACT_FLIGHT_LEG_KEY],
[FORSINKELSE_MIN],[FLYBEVEGELSE_DATO],[KALLESIGNAL],[STD],[STA],[ETD],[ETA],[OFF_BLOCK],[ON_BLOCK],[ATD],[ATA],
[REGMERKE],[ANTALL_SETER],[MTOW],[NOX_UTSLIPPSVERDI],[FRAKT],[POST],[FRAKT_POST],[FORSINKELSE_MIN_OVER_15_MIN],
[FORSINKELSE_MIN_UNDER_3_MIN],[Antall_forsinkelse_kjent_Tid],[Antall_forsinkelse_kjent_Tid_3],
[Antall_forsinkelse_totalt],[Totalt_Antall_kjent_tid],[Antall_kanselleringer],[FLIGHT_ID],
[FLYBEVEGELSE_ID_KANSELLERT],[FLYBEVEGELSE_ID_FORSINKELSE],[TOUCH_GO_COUNT],[FellesOrgSK],[motsattOrganisasjonSK],
[PeriodeSTDSk],[TimeSTDSk],[PeriodeSTASk],[TimeSTASk],[PeriodeATASk],[TimeATASk],[PeriodeATDSk],[TimeATDSk],
[PeriodeOnBlockSk],[TimeOnBlockSk],[PeriodeOffBlockSk],[TimeOffBlockSk],[KorrigeringPost],[KorrigeringFrakt],
[FirstBag],[Lastbag],[BagsKvalitetSk],[BusGateSk],[OmradeOSLSk]) GO
Result: ( row(s) affected) 00:00:05

10. With Clustered Columnstore:
create Clustered columnstore index CCI_FactPunktlighetRegularitet on [dbo].[FactPunktlighetRegularitet];
Result: ( row(s) affected) 00:00:09
With NONCLUSTERED Columnstore:
CREATE NONCLUSTERED COLUMNSTORE INDEX NCCI_FactPunktlighetRegularitet on [dbo].[FactPunktlighetRegularitet]
(FactKey, FlygningArtSk, FlyplassSk, FlyselskapSk, FlytypeSk, PeriodeSk, TimeSk, ForsinkelseSk, OrganisasjonSk,
AvgAnkSk, VektKlasseSk, AvgAnkFlyplassSk, KansellertSk, GateSk, HandlerSk, FlyoppstillingsplassSk,
AvgAnkFlyoppstillingsplassSk, FLYBEVEGELSE_ID, ACT_FLIGHT_KEY, ACT_FLIGHT_LEG_KEY, FORSINKELSE_MIN,
FLYBEVEGELSE_DATO, KALLESIGNAL, STD, STA, ETD, ETA, OFF_BLOCK, ON_BLOCK, ATD, ATA, REGMERKE, ANTALL_SETER,
MTOW, NOX_UTSLIPPSVERDI, FRAKT, POST, FRAKT_POST, BatchId, FORSINKELSE_MIN_OVER_15_MIN,
FORSINKELSE_MIN_UNDER_3_MIN, Antall_forsinkelse_kjent_Tid, Antall_forsinkelse_kjent_Tid_3,
Antall_forsinkelse_totalt, Totalt_Antall_kjent_tid, Antall_kanselleringer, FLIGHT_ID, FLYBEVEGELSE_ID_KANSELLERT,
FLYBEVEGELSE_ID_FORSINKELSE, TOUCH_GO_COUNT, FellesOrgSK, motsattOrganisasjonSK, PeriodeSTDSk, TimeSTDSk,
PeriodeSTASk, TimeSTASk, PeriodeATASk, TimeATASk, PeriodeATDSk, TimeATDSk, PeriodeOnBlockSk, TimeOnBlockSk,
PeriodeOffBlockSk, TimeOffBlockSk, PeriodeStopBeltSK, TimeStopBeltSK, PeriodeStartBeltSK, TimeStartBeltSK,
PARK_STA, Belt_Start_Time, Belt, KorrigeringPost, KorrigeringFrakt, ISDELETED, FirstBag, Lastbag, BagsKvalitetSk,
BusGateSk, OmradeOSLSk, Flygningsart)
Result: ( row(s) affected) 00:00:10
With both CCI and full row index:
Result: ( row(s) affected) 00:00:05

11. exec sp_spaceused 'FactPunktlighetRegularitet'
name
rows
reserved
data
index_size
unused
Current usage:
FactPunktlighetRegularitet KB KB KB
4696 KB
Med anbefalt rowindeks: KB KB
4768 KB
Med Clustered Columnstore: KB KB
0 KB
240 KB

12. When not to use Clustered Columnstore
Small and narrow tables
Tables used to search for single or small range sets
Tin can servers…
It really depends…

13. Fraud database. 40 columns, 15 Bill rows, 7 TB disk usage
Create Clustered Columnstore – took 9 hours
Database
Query
Comment
Feature
Svartid
MS SQL
Query 1 (MS SQL)
disk usage: 1,7TB
partisjonert + column store
44 sec
disk usage: 7,7TB
partisjonert + column store + index
4 sec
Oracle
Query 1 (Oracle)
index
21:70 sec
InMemory
6:13 sec

14. Links http://www.nikoport.com/columnstore