Real world In-Memory OLTP Ned Otter| SQL Strategist #575 | Providence 2016

Passionate about SQL Server Obsessed with In-Memory OLTP About me SQL Server DBA since 1995 MCSE Data Platform Passionate about SQL Server Obsessed with In-Memory OLTP

KEY TAKEAWAYS ARCHITECTURE RTO RESOURCES

Express (352 MB) Web (16 GB) Standard (23 GB) Enterprise (OS limit) Editions, as of SQL 2016/SP1 Express (352 MB) Web (16 GB) Standard (23 GB) Enterprise (OS limit)

Desired workload characteristics Concurrency High Write level Intensive Latency/Contention “localized” Hot data Bounded Business logic in Database

Use cases Workload won’t scale Intensive business logic Low latency Session state TempDB bottleneck Interpreted TSQL bottleneck ETL staging

ARCHITECTURE

Lock free != WAIT FREE

Data/delta files 001 to 100 101 to 200 201 to 300 301 to 400

Data/delta files DATABASE FILEGROUP CONTAINER (FOLDER)

IN-MEM DB ARCHITECTURE FILEGROUPS MEMORY-OPTIMIZED Container1 DATA/DELTA Container2 PRIMARY MDF LDF OFFLINE CKPT THREAD

Data/delta files Shared CFP IFI

RECOVERY

Why is the number and placement of containers absolutely critical?

MEMORY-OPTIMIZED TABLES Recovery process MEMORY-OPTIMIZED TABLES CONTAINER 1 CONTAINER 2 CONTAINER 3 CONTAINER 4

Recovery events SQL service restart Server reboot RESTORE OFFLINE/ONLINE SQL service restart RESTORE Server reboot

Potential recovery issues In-Mem FG CPU bound FCI/AG

RESTORE

RESTORE (1)

RESTORE (2)

RESTORE FILELISTONLY RESTORE will fail without enough memory LogicalName PhysicalName Type FileGroupName Size MyDBData h:\SQLData\MyDB.mdf D PRIMARY 104857600 MyDBlog h:\SQLData\MyDB.ldf L NULL MyDBFG1 h:\SQLDATA\FG1 S FG1 30945575359 MyDBFG2 h:\SQLDATA\FG2 FG2 61891150718

CORRUPTION

Corruption: Detection and Recovery (1) Method Harddrive Memory CHECKTABLE YES NO CHECKDB Page-level restore (mirroring/AG) n/a Recover data from NC indexes Hack attach RESTORE … REBUILD LOG

Corruption: Detection and Recovery (2) BLOCK CHECKSUM

Corruption: Detection and Recovery (3) CHECKDB Alternative: BACKUP <mem-optimized FG> TO DISK = ‘nul’

ROW VERSIONS

Row versions (1) Memory-optimized Harddrive-based

Row versions (2) Row = version!!! INDEX MEMORY VERSION 1 V1 VERSION 2

Row versions (3) UPDATE DELETE

EXISTING ROW VERSIONS = 1 begin1 end1 201 to 300 DELETE EXISTING ROW VERSIONS = 1

Row versions (5) 201 to 300 DELETE TOTAL ROW VERSIONS = 1 VERSION 1 begin1 end1 2016-08-06 13:56:35 201 to 300 DELETE TOTAL ROW VERSIONS = 1

EXISTING ROW VERSIONS = 1 UPDATE VERSION 1 begin1 end1 EXISTING ROW VERSIONS = 1

EXISTING ROW VERSIONS = 1 INSERT DELETE = + VERSION 1 begin1 end1 UPDATE EXISTING ROW VERSIONS = 1

EXISTING ROW VERSIONS = 1 INSERT = + VERSION 1 begin1 end1 UPDATE DELETE 2016-08-06 13:56:35 EXISTING ROW VERSIONS = 1

Row versions (9) UPDATE = DELETE + INSERT TOTAL ROW VERSIONS = 2 begin1 end1 begin2 end2 UPDATE DELETE 2016-08-06 13:56:35 TOTAL ROW VERSIONS = 2

Row versions (10) … What could possibly go wrong???? VERSION 1 begin1 end1 VERSION 2 begin2 end2 … What could possibly go wrong????

Buffer Pool

RESOURCE GOVERNOR Initial binding Resource pool % Monitoring

GARBAGE COLLECTION

Garbage Collection: Identify ROWS IN MEMORY VERSION 1 VERSION 2 VERSION 3 INDEX V1 V3 V2 SCAN SELECT * FROM table WHERE col = x

Garbage Collection: Unlink ROWS IN MEMORY VERSION 1 VERSION 2 VERSION 3 INDEX V3 SCAN SELECT * FROM table WHERE col = x

Garbage Collection: Schedule SELECT * FROM table WHERE col = x SELECT * FROM table WHERE col = y SCH0 1..16 SCH1

Garbage Collection: Schedule SELECT * FROM table WHERE col = x SELECT * FROM table WHERE col = y SCH0 SCH1

Garbage Collection – review Background Self-throttling Shared workload Long/cancelled transactions Table variables

Memory usage COMPRESSION PAGING

Capacity planning MEMORY

Capacity planning, Memory Indexes Row versions Table variables Temporal/Staging Index/data growth Sorting (NC)

Capacity planning STORAGE

Data/delta files 001 to 100 101 to 200 201 to 300 301 to 400

Capacity planning, Storage FOOTPRINT GROWTH BACKUPS CONTAINERS WORKLOAD IO RTO

MIGRATING DATA

Loading data Item Hard Drive Memory- optimized Recovery model BULK n/a Create clustered index before load YES Drop nonclustered indexes before load NO Multiple clients to load

Considerations for migrating data SELECT INTO TLOG BUCKET COUNT DURABILITY

Benchmark - source table on SSD Sessions Rows in batch Duration Rows inserted 1 3 minutes 1,248,433 10 100 seconds 5 million 100 24 seconds 1000 13 seconds 2 17 seconds 10 million 3 15 million

Migrating data: cross-database (1) Source Destination Cross-database Disk table Memory table Not directly Disk*

Migrating data: cross-database (2)

Migrating data: cross-database (3)

ALTER TABLE

ALTER TABLE Can be single-threaded/fully logged Executes OFFLINE

HOT/COLD

Hot/Cold data requirements Hot data in-memory Cold data on harddrive Referential integrity

Referential Integrity requirements VALIDATE PARENTID NO ORPHANS

Sample tables CUSTOMERS ORDERS

CUSTOMERS (in-mem) HOT: ORDERS (in-mem) COLD: ORDERS (harddrive) MEMORY CUSTOMERS (in-mem) HOT: ORDERS (in-mem) COLD: ORDERS (harddrive) DISK

HOT: ORDERS (in-mem) CUSTOMERS (harddrive) COLD: ORDERS (harddrive) MEMORY HOT: ORDERS (in-mem) CUSTOMERS (harddrive) COLD: ORDERS (harddrive) DISK

Scenario 1 PARENT TABLE IN-MEMORY

Sample scenario (1) A CUSTOMERS HOT: ORDERS (in-mem) (in-mem) MEMORY CUSTOMERS (in-mem) HOT: ORDERS (in-mem) COLD: ORDERS (harddrive) DISK

Sample scenario (1) B CUSTOMERS FK HOT: ORDERS (in-mem) (in-mem) MEMORY CUSTOMERS (in-mem) HOT: ORDERS (in-mem) FK COLD: ORDERS (harddrive) DISK

Sample scenario (1) C CUSTOMERS FK HOT: ORDERS (in-mem) (in-mem) MEMORY CUSTOMERS (in-mem) HOT: ORDERS (in-mem) FK COLD: ORDERS (harddrive) DISK

Sample scenario (1) D CUSTOMERS FK HOT: ORDERS (in-mem) (in-mem) FK MEMORY CUSTOMERS (in-mem) HOT: ORDERS (in-mem) FK FK COLD: ORDERS (harddrive) DISK

Sample scenario (1) E CUSTOMERS FK HOT: ORDERS (in-mem) (in-mem) FK MEMORY CUSTOMERS (in-mem) HOT: ORDERS (in-mem) FK FK COLD: ORDERS (harddrive) DISK

Scenario 2 PARENT TABLE IN-MEMORY

Sample scenario (2) A CUSTOMERS FK HOT: ORDERS (in-mem) (in-mem) MEMORY CUSTOMERS (in-mem) HOT: ORDERS (in-mem) FK COLD: ORDERS (harddrive) DISK

Sample scenario (2) B CUSTOMERS FK HOT: ORDERS (in-mem) (in-mem) MEMORY CUSTOMERS (in-mem) HOT: ORDERS (in-mem) FK COLD: ORDERS (harddrive) DISK TRIGGER

Sample scenario (2) C CUSTOMERS FK HOT: ORDERS (in-mem) (in-mem) MEMORY CUSTOMERS (in-mem) HOT: ORDERS (in-mem) FK COLD: ORDERS (harddrive) DISK TRIGGER

Sample scenario (2) D CUSTOMERS FK HOT: ORDERS (in-mem) (in-mem) MEMORY CUSTOMERS (in-mem) HOT: ORDERS (in-mem) FK COLD: ORDERS (harddrive) DISK TRIGGER

Sample scenario (2) E CUSTOMERS FK HOT: ORDERS (in-mem) (in-mem) MEMORY CUSTOMERS (in-mem) HOT: ORDERS (in-mem) FK COLD: ORDERS (harddrive) DISK TRIGGER

Scenario 3 PARENT TABLE HARDDRIVE-BASED

Sample scenario (3) A HOT: ORDERS (in-mem) CUSTOMERS (harddrive) MEMORY HOT: ORDERS (in-mem) CUSTOMERS (harddrive) COLD: ORDERS (harddrive) DISK FK

Sample scenario (3) B HOT: ORDERS (in-mem) TRIGGER CUSTOMERS MEMORY HOT: ORDERS (in-mem) TRIGGER CUSTOMERS (harddrive) COLD: ORDERS (harddrive) DISK FK

Sample scenario (3) C HOT: ORDERS (in-mem) TRIGGER CUSTOMERS MEMORY HOT: ORDERS (in-mem) TRIGGER CUSTOMERS (harddrive) COLD: ORDERS (harddrive) DISK FK

Sample scenario (3) D DML HOT: ORDERS (in-mem) TRIGGER CUSTOMERS MEMORY HOT: ORDERS (in-mem) TRIGGER CUSTOMERS (harddrive) COLD: ORDERS (harddrive) DISK FK

Sample scenario (3) E DML HOT: ORDERS (in-mem) TRIGGER CUSTOMERS MEMORY HOT: ORDERS (in-mem) INTEROP PROC TRIGGER CUSTOMERS (harddrive) COLD: ORDERS (harddrive) DISK FK

Sample scenario (3) F DML HOT: ORDERS (in-mem) TRIGGER CUSTOMERS MEMORY HOT: ORDERS (in-mem) INTEROP PROC TRIGGER CUSTOMERS (harddrive) COLD: ORDERS (harddrive) DISK FK

Scenario 4 TEMPORAL

Sample scenario 4a: Temporal MEMORY CUSTOMERS (in-mem) Temporal ORDERS (in-mem) Temporal FK CUSTOMERS (harddrive) History ORDERS (harddrive) History DISK FK

Sample scenario 4b: Temporal MEMORY CUSTOMERS (in-mem) Temporal ORDERS (in-mem) Temporal FK HOT AND COLD DATA IN MEMORY CUSTOMERS (harddrive) History ORDERS (harddrive) History DISK FK

Scenario 5 DUPLICATE PARENT

Sample scenario 5: dupe parent MEMORY CUSTOMERS (in-mem) HOT: ORDERS (in-mem) FK TRIGGER CUSTOMERS (harddrive) COLD: ORDERS (harddrive) DISK FK

Hot/Cold data – possible approaches Partitioning: NO FK: YES, but… Triggers: YES , but… Temporal: NO

Hardware Log volume Data volume NUMA Soft NUMA

KEY TAKEAWAYS ARCHITECTURE RTO RESOURCES

Connect items Unknown memory requirements for restoring a backup containing memory-optimized data: http://bit.ly/1QntGy6 Excessive memory usage for memory- optimized table variable with RANGE index: http://bit.ly/28Nvm7t

Thanks! t: @NedOtter e: ned@nedotter.com http://www.nedotter.com Ned Otter | SQL Strategist #575 | Providence 2016