1. Real world In-Memory OLTP
Ned Otter| SQL Strategist
#575 | Providence 2016

2. Passionate about SQL Server Obsessed with In-Memory OLTP
About me
SQL Server DBA since 1995
MCSE Data Platform
Passionate about SQL Server
Obsessed with In-Memory OLTP

3. KEY TAKEAWAYS
ARCHITECTURE
RTO
RESOURCES

4. Express (352 MB) Web (16 GB) Standard (23 GB) Enterprise (OS limit)
Editions, as of SQL 2016/SP1
Express (352 MB)
Web (16 GB)
Standard (23 GB)
Enterprise (OS limit)

5. Desired workload characteristics
Concurrency
High
Write level
Intensive
Latency/Contention
“localized”
Hot data
Bounded
Business logic in
Database

6. Use cases Workload won’t scale Intensive business logic Low latency
Session state
TempDB bottleneck
Interpreted TSQL bottleneck
ETL staging

7. ARCHITECTURE

8. Lock free != WAIT FREE

9. Data/delta files 001 to 100 101 to 200 201 to 300 301 to 400

10. Data/delta files DATABASE FILEGROUP CONTAINER (FOLDER)

11. IN-MEM DB ARCHITECTURE
FILEGROUPS
MEMORY-OPTIMIZED
Container1
DATA/DELTA
Container2
PRIMARY
MDF
LDF
OFFLINE CKPT THREAD

12. Data/delta files
Shared
CFP
IFI

13. RECOVERY

14. Why is the number and placement of containers absolutely critical?

15. MEMORY-OPTIMIZED TABLES
Recovery process
MEMORY-OPTIMIZED TABLES
CONTAINER 1
CONTAINER 2
CONTAINER 3
CONTAINER 4

16. Recovery events SQL service restart Server reboot RESTORE
OFFLINE/ONLINE
SQL service restart
RESTORE
Server reboot

17. Potential recovery issues
In-Mem FG
CPU bound
FCI/AG

18. RESTORE

19. RESTORE (1)

20. RESTORE (2)

21. RESTORE FILELISTONLY RESTORE will fail without enough memory
LogicalName
PhysicalName
Type
FileGroupName
Size
MyDBData
h:\SQLData\MyDB.mdf D
PRIMARY
MyDBlog
h:\SQLData\MyDB.ldf L
NULL
MyDBFG1
h:\SQLDATA\FG1 S
FG1
MyDBFG2
h:\SQLDATA\FG2
FG2

22. CORRUPTION

23. Corruption: Detection and Recovery (1)
Method
Harddrive
Memory
CHECKTABLE
YES NO
CHECKDB
Page-level restore (mirroring/AG)
n/a
Recover data from NC indexes
Hack attach
RESTORE … REBUILD LOG

24. Corruption: Detection and Recovery (2)
BLOCK
CHECKSUM

25. Corruption: Detection and Recovery (3)
CHECKDB Alternative: BACKUP <mem-optimized FG> TO DISK = ‘nul’

26. ROW VERSIONS

27. Row versions (1)
Memory-optimized
Harddrive-based

28. Row versions (2) Row = version!!! INDEX MEMORY VERSION 1 V1 VERSION 2

29. Row versions (3)
UPDATE
DELETE

30. EXISTING ROW VERSIONS = 1
begin1
end1
201 to 300
DELETE
EXISTING ROW VERSIONS = 1

31. Row versions (5) 201 to 300 DELETE TOTAL ROW VERSIONS = 1 VERSION 1
begin1
end1
13:56:35
201 to 300
DELETE
TOTAL ROW VERSIONS = 1

32. EXISTING ROW VERSIONS = 1
UPDATE
VERSION 1
begin1
end1
EXISTING ROW VERSIONS = 1

33. EXISTING ROW VERSIONS = 1
INSERT
DELETE = +
VERSION 1
begin1
end1
UPDATE
EXISTING ROW VERSIONS = 1

34. EXISTING ROW VERSIONS = 1
INSERT = +
VERSION 1
begin1
end1
UPDATE
DELETE
13:56:35
EXISTING ROW VERSIONS = 1

35. Row versions (9) UPDATE = DELETE + INSERT TOTAL ROW VERSIONS = 2
begin1
end1
begin2
end2
UPDATE
DELETE
13:56:35
TOTAL ROW VERSIONS = 2

36. Row versions (10) … What could possibly go wrong???? VERSION 1 begin1
end1
VERSION 2
begin2
end2 …
What could possibly go wrong????

37. Buffer Pool

38. RESOURCE GOVERNOR
Initial binding
Resource pool %
Monitoring

39. GARBAGE COLLECTION

40. Garbage Collection: Identify
ROWS IN MEMORY
VERSION 1
VERSION 2
VERSION 3
INDEX V1 V3 V2
SCAN
SELECT *
FROM table
WHERE col = x

41. Garbage Collection: Unlink
ROWS IN MEMORY
VERSION 1
VERSION 2
VERSION 3
INDEX V3
SCAN
SELECT *
FROM table
WHERE col = x

42. Garbage Collection: Schedule
SELECT *
FROM table
WHERE col = x
SELECT *
FROM table
WHERE col = y
SCH0
1..16
SCH1

43. Garbage Collection: Schedule
SELECT *
FROM table
WHERE col = x
SELECT *
FROM table
WHERE col = y
SCH0
SCH1

44. Garbage Collection – review
Background
Self-throttling
Shared workload
Long/cancelled transactions
Table variables

45. Memory usage
COMPRESSION
PAGING

46. Capacity planning
MEMORY

47. Capacity planning, Memory
Indexes
Row versions
Table variables
Temporal/Staging
Index/data growth
Sorting (NC)

48. Capacity planning
STORAGE

49. Data/delta files 001 to 100 101 to 200 201 to 300 301 to 400

50. Capacity planning, Storage
FOOTPRINT
GROWTH
BACKUPS
CONTAINERS
WORKLOAD IO
RTO

51. MIGRATING DATA

52. Loading data Item Hard Drive Memory- optimized Recovery model BULK n/a
Create clustered index before load
YES
Drop nonclustered indexes before load NO
Multiple clients to load

53. Considerations for migrating data
SELECT INTO
TLOG
BUCKET COUNT
DURABILITY

54. Benchmark - source table on SSD
Sessions
Rows in batch
Duration
Rows inserted 1
3 minutes
1,248,433 10
100 seconds
5 million
100
24 seconds
1000
13 seconds 2
17 seconds
10 million 3
15 million

55. Migrating data: cross-database (1)
Source
Destination
Cross-database
Disk table
Memory table
Not directly
Disk*

56. Migrating data: cross-database (2)

57. Migrating data: cross-database (3)

58. ALTER TABLE

59. ALTER TABLE
Can be single-threaded/fully logged
Executes OFFLINE

60. HOT/COLD

61. Hot/Cold data requirements
Hot data in-memory Cold data on harddrive Referential integrity

62. Referential Integrity requirements
VALIDATE PARENTID
NO ORPHANS

63. Sample tables
CUSTOMERS
ORDERS

64. CUSTOMERS (in-mem) HOT: ORDERS (in-mem) COLD: ORDERS (harddrive)
MEMORY
CUSTOMERS
(in-mem)
HOT: ORDERS
(in-mem)
COLD: ORDERS
(harddrive)
DISK

65. HOT: ORDERS (in-mem) CUSTOMERS (harddrive) COLD: ORDERS (harddrive)
MEMORY
HOT: ORDERS
(in-mem)
CUSTOMERS
(harddrive)
COLD: ORDERS
(harddrive)
DISK

66. Scenario 1
PARENT TABLE
IN-MEMORY

67. Sample scenario (1) A CUSTOMERS HOT: ORDERS (in-mem) (in-mem)
MEMORY
CUSTOMERS
(in-mem)
HOT: ORDERS
(in-mem)
COLD: ORDERS
(harddrive)
DISK

68. Sample scenario (1) B CUSTOMERS FK HOT: ORDERS (in-mem) (in-mem)
MEMORY
CUSTOMERS
(in-mem)
HOT: ORDERS
(in-mem) FK
COLD: ORDERS
(harddrive)
DISK

69. Sample scenario (1) C CUSTOMERS FK HOT: ORDERS (in-mem) (in-mem)
MEMORY
CUSTOMERS
(in-mem)
HOT: ORDERS
(in-mem) FK
COLD: ORDERS
(harddrive)
DISK

70. Sample scenario (1) D CUSTOMERS FK HOT: ORDERS (in-mem) (in-mem) FK
MEMORY
CUSTOMERS
(in-mem)
HOT: ORDERS
(in-mem) FK FK
COLD: ORDERS
(harddrive)
DISK

71. Sample scenario (1) E CUSTOMERS FK HOT: ORDERS (in-mem) (in-mem) FK
MEMORY
CUSTOMERS
(in-mem)
HOT: ORDERS
(in-mem) FK FK
COLD: ORDERS
(harddrive)
DISK

72. Scenario 2
PARENT TABLE
IN-MEMORY

73. Sample scenario (2) A CUSTOMERS FK HOT: ORDERS (in-mem) (in-mem)
MEMORY
CUSTOMERS
(in-mem)
HOT: ORDERS
(in-mem) FK
COLD: ORDERS
(harddrive)
DISK

74. Sample scenario (2) B CUSTOMERS FK HOT: ORDERS (in-mem) (in-mem)
MEMORY
CUSTOMERS
(in-mem)
HOT: ORDERS
(in-mem) FK
COLD: ORDERS
(harddrive)
DISK
TRIGGER

75. Sample scenario (2) C CUSTOMERS FK HOT: ORDERS (in-mem) (in-mem)
MEMORY
CUSTOMERS
(in-mem)
HOT: ORDERS
(in-mem) FK
COLD: ORDERS
(harddrive)
DISK
TRIGGER

76. Sample scenario (2) D CUSTOMERS FK HOT: ORDERS (in-mem) (in-mem)
MEMORY
CUSTOMERS
(in-mem)
HOT: ORDERS
(in-mem) FK
COLD: ORDERS
(harddrive)
DISK
TRIGGER

77. Sample scenario (2) E CUSTOMERS FK HOT: ORDERS (in-mem) (in-mem)
MEMORY
CUSTOMERS
(in-mem)
HOT: ORDERS
(in-mem) FK
COLD: ORDERS
(harddrive)
DISK
TRIGGER

78. Scenario 3
PARENT TABLE
HARDDRIVE-BASED

79. Sample scenario (3) A HOT: ORDERS (in-mem) CUSTOMERS (harddrive)
MEMORY
HOT: ORDERS
(in-mem)
CUSTOMERS
(harddrive)
COLD: ORDERS
(harddrive)
DISK FK

80. Sample scenario (3) B HOT: ORDERS (in-mem) TRIGGER CUSTOMERS
MEMORY
HOT: ORDERS
(in-mem)
TRIGGER
CUSTOMERS
(harddrive)
COLD: ORDERS
(harddrive)
DISK FK

81. Sample scenario (3) C HOT: ORDERS (in-mem) TRIGGER CUSTOMERS
MEMORY
HOT: ORDERS
(in-mem)
TRIGGER
CUSTOMERS
(harddrive)
COLD: ORDERS
(harddrive)
DISK FK

82. Sample scenario (3) D DML HOT: ORDERS (in-mem) TRIGGER CUSTOMERS
MEMORY
HOT: ORDERS
(in-mem)
TRIGGER
CUSTOMERS
(harddrive)
COLD: ORDERS
(harddrive)
DISK FK

83. Sample scenario (3) E DML HOT: ORDERS (in-mem) TRIGGER CUSTOMERS
MEMORY
HOT: ORDERS
(in-mem)
INTEROP PROC
TRIGGER
CUSTOMERS
(harddrive)
COLD: ORDERS
(harddrive)
DISK FK

84. Sample scenario (3) F DML HOT: ORDERS (in-mem) TRIGGER CUSTOMERS
MEMORY
HOT: ORDERS
(in-mem)
INTEROP PROC
TRIGGER
CUSTOMERS
(harddrive)
COLD: ORDERS
(harddrive)
DISK FK

85. Scenario 4
TEMPORAL

86. Sample scenario 4a: Temporal
MEMORY
CUSTOMERS
(in-mem)
Temporal
ORDERS
(in-mem)
Temporal FK
CUSTOMERS
(harddrive)
History
ORDERS
(harddrive)
History
DISK FK

87. Sample scenario 4b: Temporal
MEMORY
CUSTOMERS
(in-mem)
Temporal
ORDERS
(in-mem)
Temporal FK
HOT AND COLD DATA IN MEMORY
CUSTOMERS
(harddrive)
History
ORDERS
(harddrive)
History
DISK FK

88. Scenario 5
DUPLICATE PARENT

89. Sample scenario 5: dupe parent
MEMORY
CUSTOMERS
(in-mem)
HOT: ORDERS
(in-mem) FK
TRIGGER
CUSTOMERS
(harddrive)
COLD: ORDERS
(harddrive)
DISK FK

90. Hot/Cold data – possible approaches
Partitioning: NO
FK: YES, but…
Triggers: YES , but…
Temporal: NO

91. Hardware
Log volume
Data volume
NUMA
Soft NUMA

92. KEY TAKEAWAYS
ARCHITECTURE
RTO
RESOURCES

93. Connect items
Unknown memory requirements for restoring a backup containing memory-optimized data:
Excessive memory usage for memory- optimized table variable with RANGE index:

94. Thanks! t: @NedOtter e: ned@nedotter.com http://www.nedotter.com
Ned Otter | SQL Strategist
#575 | Providence 2016