1. Agenda 10 Key SQL 2012 BI Innovations BI Semantic Model Project ‘Apollo’ Vertipaq xVelocity in SQL 2012

2. 10 Key SQL 2012 BI Innovations 1.BI Semantic Model 2.Analysis Services Tabular Mode 1.With BIDS support 3.PowerPivot “2” 4.Power View (SSRS) 5.Self-Service Data Alerts (SSRS) 6.Hadoop Big Data Integration 7.xVelocity–Columnstore Indexes 8.Geospatial Indexes 9.Unstructured Data Queries 10.Data Quality Services –Updated Master Data Services

3. Analysis Services: Tomorrow Build on the strengths and success of Analysis Services and expand its reach to a much broader user base Embrace the relational data model – well understood by developers and IT Pros Bring together the relational and multidimensional models under a single unified BI platform— best of both worlds! Provide flexibility in the platform to suit the diverse needs of BI applications

4. BI Semantic Model One Model for All User Experiences Visualise analysis using your favourite tools Model data the way you like Store analytical data however it is best done BISM is a concept, not a product –Can be hosted in PowerPivot or SSAS

5. BI Semantic Model One Model for All User Experiences BI Semantic Model Data Model Business Logic/Queries Data Access ROLAP MOLAP xVelocity Direct Query MDXDAX Multi- dimensional Tabular Your Apps Reporting Services & Power View Excel PowerPivot DatabasesLOB ApplicationsFilesOData FeedsCloud Services SharePoint Insights

6. BI Semantic Model What about existing Analysis Services applications? New applications New technology options Existing applications Based on Unified Dimensional Model Existing applications Every UDM becomes a BI Semantic Model Existing applications Based on Unified Dimensional Model

7. Third-party applications Reporting Services Excel PowerPivot DatabasesLOB ApplicationsFilesOData FeedsCloud Services SharePoint Insights BI Semantic Model: Architecture

8. Tabular Mode Scaling PowerPivot to Enterprise Needs Model in PowerPivot –PowerPivot as source of SSAS Tabular Models –Excel for browsing and testing in SSDT All new PowerPivot features: –Diagrams, Measure Grid, KPIs, Hierarchies, Perspectives, 30+ New DAX Functions …and, unique to SSAS Tabular Mode: –Row-level Security, Partitions, Large Tables (>2 billion rows), Images, Memory Paging

9. SQL ServerDynamics CRM End User Model Developer Example: Power View Over a Sales Model

10. SQL ServerDynamics CRM End User Model Developer Example: Power View Over a Sales Model

11. OracleSAP End User Model Developer Example: Excel Over a Finance Model

12. OracleSAP End User Model Developer Example: Excel Over a Finance Model

13. Demo…. Tour of SQL Server 2012 BISM

14. BI Semantic Model FlexibilityRichnessScalability

15. Data Model

16. Business Logic

17. Data Access and Storage

18. Analysis Services Architecture Excel Services PowerPivot for SharePoint (Analysis Services) PowerPivot for SharePoint (Analysis Services) Excel Browser Analysis Services BI Semantic Model SharePoint Reporting Services Third Party Apps PowerPivo t for Excel xlsx BI Development Studio Personal BI Team BI Organizational BI

19. SQL Server 2012 RTM – What’s In? Tabular modeling –Multiple relationships between tables –Hierarchies, KPIs, perspectives –Distinct count, parent child –Datetime, blobs, images –Reporting (Power View) properties –DAX extensions & queries –Drillthrough –Row level security –Large tables, partitions, paging –DirectQuery (SQL Server only) –Perf & scale Multidimensional modeling –Large dims (>4GB string store) –XEvents (monitoring) –Perf & scale improvements Pro Tools –New tabular project in BIDS –BIDS on VS 2010 shell –SSMS support for tabular models –OWC removal –PowerShell support –Profiler: MDX/DAX query plan events PowerPivot for Excel –Full support for tabular modeling features –Modeling diagram –Measure grid (calculations area) PowerPivot for SharePoint –PowerPivot Gallery integrated with Power View –Setup re-architecture, configuration tool –Perf, scale & reliability

20. Project ‘Apollo’ in SQL 2012

21. Apollo: A new column-oriented query accelerator What is Apollo? –What does column-oriented mean? –How does it accelerate queries? When to create a columnstore index How to use Apollo –Creating an index –Running queries –Loading data How does Apollo relate to VertiPaq and PowerPivot?

22. Apollo is the code name for new functionality that is available in SQL Server 2012 It will substantially accelerate common data warehouse queries Adds a column store option in SQL Server database engine – New index type in the database engine Advanced query processing algorithms – New batch mode processing What is Apollo?

23. When to use Apollo Data warehousing – Read-mostly workloads – Star joins – Process large amounts of data Generous amount of memory – Best performance when data fits in memory – Graceful degradation as fact table paged from disk – Under severe memory constraints, falls back to row-at-a-time processing

24. How does Apollo speed up queries? (1) Stores data column-wise Better compression Uses VertiPaq compression technology Less IO … C1 C2 C3 C5C6C4

25. How does Apollo speed up queries? (2) Fetches only needed columns from disk – Less IO – Better buffer hit rates C1 C2 C4 C5 C6 C3 SELECT region, sum (sales) …

26. How does Apollo speed up queries? (3) Advanced query processing technology Processes column data in batches Highly efficient algorithms Compact data representation Better parallelism

27. Columnstore Index Columnstore index is nonclustered (secondary) Base table can be clustered index or heap One CS index per table Multiple other nonclustered (B- tree) indexes allowed – But may not be needed CS index must be partition- aligned if table is partitioned Indexed view Filtered index Clustered index Heap Nonclustered index Nonclustered columnstore index - No CS index on indexed view - No CS as filtered index Base table OR

28. Using Apollo: Creating the index Create a columnstore index Create the table Load data into the table Create a non-clustered columnstore index on all, or some, columns CREATE NONCLUSTERED COLUMNSTORE INDEX ncci ON myTable(OrderDate, ProductID, SaleAmount) Object Explorer

29. Index dialog makes index creation easy

30. Using Apollo: Running queries Let the query optimizer do the work –Optimizer makes a cost-based decision –Data access method Columnstore index Clustered (row-based) index Nonclustered (row-based) index Heap Processing mode Batch mode Row mode

31. Using Apollo: Memory management Memory management is automatic Columnstore is persisted on disk Needed columns fetched into memory –Columnstore segments flow between disk and memory SELECT C2, SUM(C4) FROM T GROUP BY C2; T.C2 T.C4 T.C2 T.C4 T.C2 T.C1 T.C3 T.C4

32. Using Apollo: Running queries Optimizer allows you to use index hints –Can hint to use the columnstore index –Can hint to use a different index select distinct (SalesTerritoryKey) from dbo.FactResellerSales with (index (ci)) select distinct (SalesTerritoryKey) from dbo.FactResellerSales with (index (ncci))

33. Using Apollo: Restrictions –Create index: only on common business data types int, real, string, money, datetime, decimal <= 18 digits –Maintain table: limited operations Can read but not update the data Can switch partitions in and out –Process queries: all read-only T-SQL queries run Some queries are accelerated more than others

34. 34 Columnstore indexes provide an easy way to significantly improve data warehouse and decision support query performance against very large data sets Performance improvements for “typical” data warehouse queries from 10x to 100x Ideal candidates include queries against star schemas that use filtering, aggregations and grouping against very large fact tables Improved Data Warehouse Query performance

35. 35 You need to execute high performance DW queries against very large data sets? – In SQL Server 2008 and SQL Server 2008 R2 OLAP (SSAS) MDX solution ROLAP and T-SQL + intermediate summary tables, indexed views and aggregate tables – Inherently inflexible – In SQL Server 2012 You can create a columnstore index on a very large fact table referencing all columns with supporting data types – Utilizing T-SQL and core Database Engine functionality – Minimal query refactoring or intervention Upon creating the columnstore index, your table becomes “read only” – but you can still use partitioning to switch in and out data OR drop/rebuild indexes periodically What Happens When…

36. 36 Two complimentary technologies: –Storage Data is stored in a compressed columnar data format (stored by column) instead of row store format (stored by row). –Columnar storage allows for less data to be accessed when only a sub-set of columns are referenced –Data density/selectivity determines how compression friendly a column is – example “State” / “City” / “Gender” –Translates to improved buffer pool memory usage –New “batch mode” execution Data can then be processed in batches (1,000 row blocks) versus row-by-row Depending on filtering and other factors, a query may also benefit by “segment elimination” - bypassing million row chunks (segments) of data, further reducing I/O How Are These Performance Gains Achieved?

37. 37 OrderDat e 200107 01 … 200107 02 … … 200107 03 … … … … 200107 04 … Cost 2171.29 1912.15 2171.29 413.14 333.42 1295.00 4233.14 641.22 24.95 64.32 1111.25 ProductI D 310 311 312 313 314 315 316 317 318 319 320 321 Column Store (values compressed) ProductI D OrderDat e Cost 3102001070 1 2171.2 9 3112001070 1 1912.1 5 3122001070 2 2171.2 9 3132001070 2 413.14 Row Store (Heap / B-Tree) Column vs. Row Store data page 1000 ProductI D OrderDat e Cost 314200107 01 333.42 315200107 01 1295.0 0 316200107 02 4233.1 4 317200107 02 641.22 data page 1001 data page 2001 data page 2000 data page 2002

38. 38 Allows processing of 1,000 row blocks as an alternative to single row-by-row operations –Enables additional algorithms that can reduce CPU overhead significantly –Batch mode “segment” is a partition broken into million row chunks with associated statistics used for Storage Engine filtering Batch mode can work to further improve query performance of a columnstore index, but this mode isn’t always chosen: –Some operations aren’t enabled for batch mode: E.g. outer joins to columnstore index table / joining strings / NOT IN / IN / EXISTS / scalar aggregates –Row mode might be used if there is SQL Server memory pressure or parallelism is unavailable –Confirm batch vs. row mode by looking at the graphical execution plan Batch Mode

39. 39 Performance gains can come from a combination of: –Columnstore indexing alone + traditional row mode in QP –Columnstore indexing + batch mode in QP –Columnstore indexing + hybrid of batch and traditional row mode in QP Columnstore format + batch mode Variations

40. 40 SSMS T-SQL Creating a columnstore index

41. 41 Index type –Columnstore indexes are always non-clustered and non-unique –They cannot be created on views, indexed views, sparse columns –They cannot act as primary or foreign key constraints Column selection –Unlike other index types, there are no “key columns” Instead you choose the columns that you anticipate will be used in your queries Up to 1,024 columns – and the ordering in your CREATE INDEX doesn’t matter No concept of “INCLUDE” No 900 byte index key size limit Column ordering –Use of ASC or DESC sorting not allowed – as ordering is defined via columnstore compression algorithms Defining the Columnstore Index

42. Accelerating Data Warehouse Queries with SQL Server 2012 Columnstore Indexes Demo..

43. 43 Supported data types –Char / nchar / varchar / nvarchar (max) types, legacy LOB types and FILESTREAM are not supported –Decimal/numeric Precision greater than 18 digits NOT supported –Tinyint, smallint, int, bigint –Float/real –Bit –Money, smallmoney –Date and time data types Datetimeoffset with scale > 2 NOT supported Supported Data Types

44. 44 Columnstore indexes cannot be used in conjunction with –Change Data Capture and Change Tracking –Filestream columns (supported columns from same table are supported) –Page, row and vardecimal storage compression –Replication –Sparse columns Data type limitations –Binary / varbinary / ntext / text / image / varchar (max) / nvarchar (max) / uniqueidentifier / rowversion / sql_variant / decimal or numeric with precesion > 18 digits / CLR types / hierarchyid / xml / datetimeoffset with scale > 2 You can prevent a query from using the columnstore index using the IGNORE_NONCLUSTERED_COLUMNSTORE_INDEX query hint Limitations

45. Method 1: Disable the columnstore index Disable (or drop) the index ALTER INDEX my_index ON T DISABLE Update the table Rebuild the columnstore index ALTER INDEX my_index ON T REBUILD Adding data to a table with a columnstore index

46. Method 2: Use Partitioning Load new data into a staging table Build a columnstore index CREATE NONCLUSTERED COLUMNSTORE INDEX my_index ON StagingT(OrderDate, ProductID, SaleAmount) Switch the partition into the table ALTER TABLE StagingT SWITCH TO T PARTITION 5 Adding data to a table with a columnstore index

47. VertiPaq: – PowerPivot for Excel – PowerPivot for Sharepoint – Analysis Services – Database Engine – Apollo Use Apollo for relational data warehousing – Large fact tables – Ad hoc or reporting queries – When you don’t need MDX Apollo and VertiPaq

48. 1 TB version of the TPC-DS database 1.44 billion rows in catalog_sales fact table 32 logical processor machine with 256 GB RAM Warm start Query Performance example SELECT w_city, w_state, d_year, SUM(cs_sales_price) AS cs_sales_price FROM warehouse, catalog_sales, date_dim WHERE w_warehouse_sk = cs_warehouse_sk and cs_sold_date_sk = d_date_sk and w_state in ('SD','OH') and d_year in (2001,2002,2003) GROUP BY w_city, w_state, d_year ORDER BY d_year, w_state, w_city;

49. Performance example: Results Total CPU timeElapsed time No columnstore502 sec501 sec Columnstore31.0 sec1.10 sec Speedup16X455X

50. Summary: Apollo in a nutshell Astonishing speedup for DW queries Great compression Columnstore technology + Advanced query processing

51. 51 SQL Server 2012 offers significantly faster query performance for data warehouse and decision support scenarios –10x to 100x performance improvement depending on the schema and query I/O reduction and memory savings through columnstore compressed storage CPU reduction with batch versus row processing, further I/O reduction if segmentation elimination occurs –Easy to deploy and requires less management than some legacy ROLAP or OLAP methods No need to create intermediate tables, aggregates, pre- processing and cubes –Interoperability with partitioning –For the best interactive end-user BI experience, consider Analysis Services, PowerPivot and Crescent Summary: SQL 2012 ColumStore

52. xVelocity in SQL 2012 No more Vertipaq, it’s now called xVelocity in- memory technologies in SQL 2012

53. Q & A

54. Thank You