1. Sofia Event Center 21-22 November 2013 Margarita Naumova SQL Master Academy

3. Administrator pass@word1

5. Memory-optimized Table: Row Format Row headerPayload (table columns) Begin Ts End Ts StmtId IdxLinkCount 8 bytes 4 bytes 2 bytes 8 bytes * (IdxLinkCount – 1)

6. Key lookup: B-tree vs. Memory-optimized Table Matching index record Hash index on Name R1 R2 R3

7. Memory Optimized Tables and Indexes 50, ∞ JanePrague TimestampsNameChain ptrsCity Hash index on City Hash index on Name 90, ∞ Susan Bogota f(Jane) f(Susan) f(Prague) f(Bogota)

8. Memory Optimized Tables and Indexes 50, ∞ JanePrague TimestampsNameChain ptrsCity Hash index on City Hash index on Name T100: INSERT (John, Prague) 100, ∞ JohnPrague 90, ∞ Susan Bogota f(John) f(Prague)

9. 90, ∞ Susan Bogota Memory Optimized Tables and Indexes 50, ∞ JanePrague TimestampsNameChain ptrsCity Hash index on City Hash index on Name T150: DELETE (Susan, Bogota) 100, ∞ JohnPrague 90, 150

10. Memory Optimized Tables and Indexes 90, 150 Susan Bogota 50, ∞ JanePrague TimestampsNameChain ptrsCity Hash index on City Hash index on Name T200: UPDATE (John, Prague) to (John, Beijing) 100, ∞ JohnPrague 200, ∞ JohnBeijing 100, 200 f(Beijing) f(John)

11. Memory Optimized Tables and Indexes 90, 150 Susan Bogota 50, ∞ JanePrague TimestampsNameChain ptrsCity Hash index on City Hash index on Name T250: Garbage collection 100, 200 JohnPrague 200, ∞ JohnBeijing f(John) f(Jane) f(Beijing) f(Prague)

12. Storage: Data and Delta Files Data File Delta File 0100 TS (ins)RowIdTableId TS (ins)RowIdTableId TS (ins)RowIdTableId TS (ins)RowIdTS (del) TS (ins)RowIdTS (del) TS (ins)RowIdTS (del) Checkpoint File Pair Row pay load Data files ~128MB in size, write 256KB chunks at a time Stores only the inserted rows (i.e. table content) Chronologically organized streams of row versions Delta files File size is not constant, write 4KB chunks at a time. Stores IDs of deleted rows

13. Populating Data/Delta files Offline Checkpoint Thread Memory-optimized Table Filegroup Range 100-199Range 200-299 Range 300-399 Range 400-499 Range 500- New Inserts Delete 450 TS Delete 250 TS Delete 150 TS Data file with rows generated in timestamp range IDs of Deleted Rows (height indicates % deleted) Del Tran2 (TS 450) Del Tran3 (TS 250) Del Tran1(TS150 ) Insert into Hekaton T1 Log in disk Table Del Tran1 (row TS150) Del Tran2 (row TS 450) Del Tran3 (row TS 250) Insert into T1 SQL Transaction log

14. Merge Operation Memory-optimized data Filegroup Files as of Time 600 Range 100-199Range 200-299Range 300-399 Range 400-499 Data file with rows generated in timestamp range IDs of Deleted Rows (height indicates % deleted) Merge 200-399 Deleted Files Files Under Merge Files as of Time 500 Memory-optimized data Filegroup Range 100-199Range 200-299Range 300-399 Range 400-499 Range 500-599 Range 200-399 Range 200-299 Range 300-399

15. 15 What’s being delivered High-performance, memory-optimized OLTP engine integrated into SQL Server and architected for modern hardware trends Main benefits Optimized for in-memory data up to 20–30 times throughput Indexes (hash and range) exist only in memory; no buffer pool, B-trees T-SQL compiled to machine code via C code generator and Visual C compiler Core engine uses lock-free algorithms; no lock manager, latches, or spinlocks Multiversion optimistic concurrency control with full ACID support On-ramp existing applications Integrated experience with same manageability, administration, and development experience In-Memory OLTP summary

16. In-Memory DW

17. Columnstore Storage Model Data stored column- wise Each page stores data from a single column Highly compressed Each column can be accessed independently

18. In-Memory DW Index Processing an Example 18

19. Horizontally Partition Row Groups 19

20. Vertical Partition Segments 20

21. Compress Each Segment 21 Some Compress More than Others

22. Fetch Only Needed Columns Segment Elimination 22

23. Fetch Only Needed Segments Segment Elimination 23

24. Updatable Columnstore Index Table consists of column store and row store DML (update, delete, insert) operations leverage delta store SELECT Unifies data from Column and Row stores - internal UNION operation. “Tuple mover” converts data into columnar format once segment is full (1M of rows) REORGANIZE statement forces tuple mover to start. C1 C2 C3 C5C6C4 Column Store C1 C2 C3 C5C6C4 Delta (row) store tuple mover

25. Structure of In-Memory DW How It Works 25 CREATE CLUSTERED COLUMNSTORE ‐ Organizes and compresses data into columnstore BULK INSERT ‐ Creates new columnstore row groups INSERT ‐ Rows are placed in the row store (heap) ‐ When row store is big enough, a new columnstore row group is created ColumnStore Deleted Bitmap Row Store Partition

26. Structure of In-Memory DW How It Works (cont'd) DELETE ‐ Rows are marked in the deleted bitmap UPDATE ‐ Delete plus insert Most data is in columnstore format ColumnStore Deleted Bitmap Row Store Partition

27. Columnstore usage recommendations Use NCSI If you need constraints and forcing uniqueness Use CCI in other cases Use for big tables JOINs on string columns is still slow! Use integer keys instead Tune batch sizes for BULK INSERTS You don’t need any special maintenance if your data loading is tuned and you don’t have many deletes

28. Resource Governor

29. Resource Governor components

30. Resource Pools CREATE RESOURCE POOL pool_name [ WITH ( [ MIN_CPU_PERCENT = value ] [ [, ] MAX_CPU_PERCENT = value ] [ [, ] CAP_CPU_PERCENT = value ] [ [, ] AFFINITY {SCHEDULER = AUTO | (Scheduler_range_spec) | NUMANODE = (NUMA_node_range_spec)} ] [ [, ] MIN_MEMORY_PERCENT = value ] [ [, ] MAX_MEMORY_PERCENT = value ] [ [, ] MIN_IOPS_PER_VOLUME = value ] [ [, ] MAX_IOPS_PER_VOLUME = value ]) ]

31. Resource Pools

32. Create workload groups Create function to classify requests into workload group Register the classification function in the previous step with the Resource Governor Enable Resource Governor Monitor resource consumption for each workload group Use monitor to establish pools Assign workload group to pool Steps to implement Resource Governor

33. Scenario 1: I just got a new version of SQL Server and would like to make use of resource governor. How can I use it in my environment? Scenario 2 (based on Scenario 1): Based on monitoring results I would like to see an event any time a query in the ad-hoc group (groupAdhoc) runs longer than 30 seconds. Scenario 3 (based on Scenario 2): I want to further restrict the ad- hoc group so it does not exceed 50 percent of CPU usage when all requests are cumulated. Resource Governor scenarios

34. Monitoring Resource Governor

35. Complete and consistent data platform

36. Споделете вашата обратна връзка за тази сесия и за цялостната организация на конференцията http://aka.ms/incharge и участвайте в томболата за HTC 8S и други награди!