Download presentation
Presentation is loading. Please wait.
Published bySharlene Preston Modified over 6 years ago
1
TechEd 2013 9/6/2018 7:34 PM © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
2
SQL Server In-Memory OLTP: DBA Deep Dive
9/6/2018 7:34 PM DBI-B308 SQL Server In-Memory OLTP: DBA Deep Dive Jos de Bruijn © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
3
TechReady 16 9/6/2018 In-Memory OLTP Recap SQL Server 2014 adds in-memory technology to boost performance of OLTP workloads Memory optimized table and index structures Stream-based storage Native compilation of business logic in stored procedures Latch- and lock-free data structures Multi-versioning built into table structures Fully integrated into SQL Server Familiar management tools: SSMS/SMO/DMVs © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
4
In-Memory OLTP for Database Admins
Memory Usage and Management On Disk Storage for Durability
5
Memory Usage and Management
9/6/2018 7:34 PM Memory Usage and Management © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
6
In-Memory Data Structures
Rows New row format Structure of the row is optimized for memory residency and access One copy of row Indexes point to rows, they do not duplicate them Indexes Hash index for point lookups Memory-optimized nonclustered index for range and ordered scans (In CTP2) Do not exist on disk – recreated during recovery
7
MemOpt Table: Row Format
TechEd 2013 9/6/2018 7:34 PM MemOpt Table: Row Format Row header Payload (table columns) Begin Ts End Ts StmtId IdxLinkCount 8 bytes 4 bytes 2 bytes 8 bytes * (IdxLinkCount – 1) Key Points Begin/End timestamp determines row’s validity No data or index page; just rows Row size limited to 8060 bytes to allow data to be moved to disk-based table Not every SQL table schema is supported © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
8
Memory Optimized Tables and Indexes
Timestamps Chain ptrs Name City Hash index on City Hash index on Name 50, ∞ Jane Prague 100, ∞ John Prague 90, 150 90, ∞ Susan Bogota
9
Memory Management Data resides in memory at all times
TechEd 2013 9/6/2018 7:34 PM Memory Management Data resides in memory at all times Must configure SQL server with sufficient memory to store memory-optimized tables Failure to allocate memory will fail transactional workload at run-time Integrated with SQL Server memory manager and reacts to memory pressure where possible Integration with Resource Governor (Post CTP1) “Bind” a database to a resource pool Mem-opt tables in a database cannot exceed the limit of the resource pool Hard limit (80% of phys. memory) to ensure system remains stable under memory pressure © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
10
Memory Usage and Monitoring
TechReady 16 9/6/2018 Memory Usage and Monitoring Table Data Rule of thumb: 2 x data_size Index Data Hash index: Fixed size Nonclustered index: Variable size Considerations Configuration Use resource governor Monitoring Use DMVs, SMO, SSMS Management © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
11
Garbage Collection Stale Row Versions Garbage Collection (GC)
TechEd 2013 9/6/2018 7:34 PM Garbage Collection Stale Row Versions Updates, deletes, and aborted insert operations create row versions that (eventually) are no longer visible to any transaction Slow down scans of index structures Create unused memory that needs to be reclaimed (i.e. Garbage Collected) Garbage Collection (GC) Analogous to version store cleanup task for disk-based tables to support Read Committed Snapshot (RCSI) System maintains ‘oldest active transaction’ hint GC Design Non-blocking, Cooperative, Efficient, Responsive, Scalable A dedicated system thread for GC Active transactions work cooperatively and pick up parts of GC work © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
12
Cooperative Garbage Collection
TechEd 2013 9/6/2018 7:34 PM Cooperative Garbage Collection Index TX4: Begin = 210 Oldest Active Hint = 175 100 200 1 John Smith Kirkland Key Points Scanners can remove expired rows when found Offloads work from GC thread Ensures that frequently visited areas of the index are cleaned regularly A row needs to be removed from all indexes before memory can be freed Garbage collection is most efficient if all indexes are frequently accesses 200 ∞ 1 John Smith Redmond 50 100 1 Jim Spring Kirkland 300 ∞ 1 Ken Stone Boston © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
13
Demo Memory Management 9/6/2018 7:34 PM
© 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
14
On Disk Storage for Durability
9/6/2018 7:34 PM On Disk Storage for Durability © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
15
Durability Memory-optimized tables can be durable or non-durable
TechEd 2013 9/6/2018 7:34 PM Durability Memory-optimized tables can be durable or non-durable Default is ‘durable’ Non-durable tables are useful for transient data Durable tables are persisted in a single memory-optimized filegroup Storage used for memory-optimized has a different access pattern than for disk tables Filegroup can have multiple containers (volumes) Additional containers aid in parallel recovery; recovery happens at the speed of IO © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
16
On-disk Storage Filestream is the underlying storage mechanism
Checksums and single-bit correcting ECC on files Data files ~128MB in size, write 256KB chunks at a time Stores only the inserted rows (i.e. table content) Chronologically organized streams of row versions Delta files File size is not constant, write 4KB chunks at a time. Stores IDs of deleted rows
17
Storage: Data and Delta Files
TechEd 2013 9/6/2018 7:34 PM Storage: Data and Delta Files 100 Transaction Timestamp Range Data File TS (ins) RowId TableId Row pay load TS (ins) RowId TableId Row pay load TS (ins) RowId TableId Row pay load Data file contains rows inserted within a given transaction range Checkpoint File Pair Delta File TS (ins) RowId TS (del) Delta file contains deleted rows within a given transaction range © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
18
Populating Data/Delta files
TechEd 2013 Populating Data/Delta files 9/6/2018 7:34 PM SQL Transaction log Del Tran1 (row TS150) Del Tran1(TS150) Log in disk Table Del Tran2 (row TS 450) Del Tran2 (TS 450) Del Tran3 (row TS 250) Del Tran3 (TS 250) Insert into Hekaton T1 Insert into T1 Offline Checkpoint Thread Data file has pre-allocated size (128 MB) Engine switches to new data file when the current file is full Transaction does not span data files Once a data file is closed, it becomes read-only Row deletes are tracked in delta file Files are append only Delete 150 TS Delete 250 TS Delete 450 TS New Inserts Memory-optimized Table Filegroup Range Range Range Range Range 500- Data file with rows generated in timestamp range IDs of Deleted Rows (height indicates % deleted) © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
19
Merge Operations What is a Merge Operation? Need for Merge
Merges 2 or more adjacent data/delta files pairs into 1 pair Need for Merge Deleting rows causes data files to have stale rows DMV: sys.dm_xtp_checkpoint_files can be used to find inserted/deleted rows and freespace Benefits of Merge Reduces storage (i.e. fewer data/delta files) required to store active data rows Improves the recovery time as there will be fewer files to load Merge is an ONLINE background Operation Merge does not block concurrent deletes in the affected file pairs
20
Merge Operation Files as of Time 500 Files as of Time 600
TechEd 2013 9/6/2018 7:34 PM Merge Operation Files as of Time 500 Files as of Time 600 Memory-optimized data Filegroup Memory-optimized data Filegroup Range Range Range Range Range Range Range Range Range Range Range Range Merge Data file with rows generated in timestamp range IDs of Deleted Rows (height indicates % deleted) Files Under Merge Deleted Files © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
21
Logging for Memory-Optimized Tables
TechEd 2013 9/6/2018 7:34 PM Logging for Memory-Optimized Tables Uses SQL transaction log to store content Each HK log record contains a log record header followed by opaque memory optimized-specific log content. All logging for memory-optimized tables is logical No log records for physical structure modifications. No index-specific / index-maintenance log records. No UNDO information is logged Recovery Models All three recovery models are supported © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
22
Storage Considerations
TechReady 16 9/6/2018 Storage Considerations Capacity needed is 2-3 x size of durable memory optimized tables Use sequential bandwidth sufficient to meet RTO Spinning media Data Log size is similar or less than disk based tables Latency is important SSDs Log © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
23
Backup for Memory-Optimized Tables
TechEd 2013 Backup for Memory-Optimized Tables 9/6/2018 7:34 PM Integrated with SQL Database Backup Memory-Optimized file group is backed up as part SQL database backup Existing backup scripts work with minimal or no changes Not supported Differential backup Transaction log backup includes memory-optimized log records CTP1 limitation: log backup includes data and delta files © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
24
Demo Logging and Storage 9/6/2018 7:34 PM
© 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
25
Recovery for Memory-Optimized Tables
Analysis Phase Finds the last completed checkpoint Data Load Load from set of data/delta files from the last completed checkpoint Parallel Load by reading data/delta files using 1 thread / file Redo phase to apply tail of the log Apply the transaction log from last checkpoint Concurrent with REDO on disk-based tables No UNDO phase for mem-opt tables Only committed transactions are logged
26
Recovery: Parallel load
Impact on RTO Load speed of data Size of durable tables Memory Optimized Tables Recovery Data Loader Recovery Data Loader Recovery Data Loader filter filter filter Delta map Delta map Delta map Data File1 Delta File1 Data File2 Delta File2 Data File3 Delta File3 Memory Optimized Container - 1 Memory Optimized Container - 2
27
Summary In-memory Data Structures
Managing Memory and Garbage Collection Persistence and Storage Management Logging and Recovery
28
Related content Breakout Sessions
9/6/2018 7:34 PM Related content Breakout Sessions DBI-B204 Microsoft SQL Server 2014 In-Memory OLTP: Overview DBI-B307 Microsoft SQL Server 2014 In-Memory OLTP: DB Developer Deep Dive Hands-On Lab: SQL Server Project “Hekaton” Find Me Later At… Ask The Experts…..this evening MSE – Server Cloud Tools…. Friday © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
29
Resources Learning TechNet msdn http://channel9.msdn.com/Events/TechEd
9/6/2018 7:34 PM Resources Learning Sessions on Demand Microsoft Certification & Training Resources TechNet msdn Resources for IT Professionals Resources for Developers © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
30
Evaluate this session Scan this QR code to evaluate this session.
9/6/2018 7:34 PM Required Slide *delete this box when your slide is finalized Your MS Tag will be inserted here during the final scrub. Evaluate this session Scan this QR code to evaluate this session. © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
31
9/6/2018 7:34 PM © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.