1. Super Scaling
The LMAX Queue Pattern

2. Thanks you our PLATINUM sponsors

3. Thanks you our GOLD and SILVER sponsors

4. About Me 15+ years plus database experience
Speaker at the last three SQL Bits and at Pass events around Europe
Some of my material on spinlocks is referenced by SQL Skills

5. I can do this purely in .Net why do this in the database ?
Memory Cache Lines
Building A High Performance Queue With SQL Server
I can do this purely in .Net why do this in the database ?

6. Building A High Performance Queue With SQL Server
C P U
C P U
A quote taken from Microsoft distinguished engineer Jim Gray out of the abstract from this article:
“Queues need security, configuration, performance monitoring, recovery, and reorganization utilities. Database systems already have these features. A full-function MOM system duplicates these database features. Queue managers are simple TP-monitors managing server pools driven by queues. Database systems are encompassing many server pool features as they evolve to TP-lite systems.”

7. Backup and recovery tools Performance monitoring tools
.Net Provides Concurrent Dictionaries, But Do You Get . . .
C P U
C P U
Backup and recovery tools
Performance monitoring tools
High availability
Natively compiled code
Seamless integration with the database engine

8. High Performance Queueing: The Naïve Approach
C P U
C P U
PUSH = Insert into a clustered index
POP = DELETE with OUTPUT clause

9. Pop Push The LMAX Disruptor Queue Pattern To The Rescue FIFO Queue
C P U
C P U
Pop
Push
Message
Message
Message
Message
Message
FIFO Queue
CREATE TABLE dbo.MyQLMax (
[slot] [bigint] NOT NULL
,[message_id] [bigint] NOT NULL
,[time] [datetime] NOT NULL
,[message] [char](300) NOT NULL
,[reference_count] [tinyint] NOT NULL )

10. The First Test Run C P U C P U CREATE PROCEDURE [dbo].[LMaxPush] AS
BEGIN
DECLARE @PushedMessageCount [bigint] = 0
[bigint] =
[datetime]
[int] = 1;
SET NOCOUNT ON;
= NEXT VALUE FOR dbo.PushSequence;
UPDATE dbo.MyQLMax
SET [time] = GETDATE()
,[message] = 'Hello world'
,[message_id]
,[reference_count] = [reference_count] + 1
WHERE slot
+= 1;
END;

11. FIFO Queue But . . . C P U C P U PAGELATCH_EX Same Page Push thread 1
Push thread N
Message
Message
Message
PAGELATCH_EX
FIFO Queue
Message
Message
Same Page

12. Scalability up to 9 threads 
The Solution !
C P U
C P U
Scalability up to 9 threads 
Stop logically contiguous slots from being in the same page

13. Where Is The Bottleneck
C P U
C P U

14. Scalability up to 14 threads 
Scalable Sequence Generation
C P U
C P U
CREATE TABLE [dbo].[NonBlockingSequence] (
[ID] [bigint] IDENTITY (1, 1) NOT NULL
,PRIMARY KEY NONCLUSTERED HASH ( [ID] )
WITH ( BUCKET_COUNT = )
) WITH ( MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA )
BEGIN TRANSACTION
INSERT INTO [dbo].[NonBlockingSequence]
DEFAULT VALUES;
= SCOPE_IDENTITY()
ROLLBACK TRANSACTION;
Scalability up to 14 threads 

15. FIFO Queue The “Last page” Problem Killed The Naïve Approach C P U
Push thread 1
Push thread N
Message
Message
Message
PAGELATCH_EX
FIFO Queue
Message
Message
Same Page
What about the in-memory OLTP engine ?, after all it is 100% lock and latch free . . .

16. Hash Index Versus Range Index
Memory Optimised Table Hash Versus Range Table Scalability
C P U
C P U
CREATE TABLE [dbo].[MyQLmaxImOltp] (
[Slot] [bigint] IDENTITY(1,1) NOT NULL
,[message_id] [bigint] NULL
,[time] [datetime] NOT NULL
,[message] [char](300)
COLLATE Latin1_General_CI_AS NOT NULL
,[reference_count] [tinyint] NOT NULL
,PRIMARY KEY NONCLUSTERED HASH (
[Slot]
) WITH ( BUCKET_COUNT = )
) WITH ( MEMORY_OPTIMIZED = ON
,DURABILITY = SCHEMA_AND_DATA )
CREATE TABLE [dbo].[MyQLmaxImOltp] (
[Slot] [bigint] IDENTITY(1,1) NOT NULL
,[message_id] [bigint] NULL
,[time] [datetime] NOT NULL
,[message] [char](300)
COLLATE Latin1_General_CI_AS NOT NULL
,[reference_count] [tinyint] NOT NULL
,PRIMARY KEY NONCLUSTERED (
[Slot] )
) WITH ( MEMORY_OPTIMIZED = ON
,DURABILITY = SCHEMA_AND_DATA )
Hash Index Versus Range Index

17. This Is How A Singleton Insert Workload Scales With These Tables
C P U
C P U
Even with zero locking and latching, the naïve approach will not scale that well with a range index . . .

18. What About The In-Memory OLTP Engine ?
C P U
C P U
The in-memory OLTP engine is lock and latch free, no performance bottlenecks to see here or are there ?

19. LMAX Queue In Memory Code V1: Queue Table
C P U
C P U
CREATE TABLE [dbo].[MyQLmaxImOltp] (
[Slot] [bigint] NOT NULL
,[message_id] [bigint] NULL
,[time] [datetime] NOT NULL
,[message] [char](300) COLLATE Latin1_General_CI_AS NOT NULL
,[reference_count] [tinyint] NOT NULL
PRIMARY KEY NONCLUSTERED HASH ( [Slot] )
WITH ( BUCKET_COUNT = )
) WITH ( MEMORY_OPTIMIZED = ON , DURABILITY = SCHEMA_AND_DATA )

20. LMAX Queue In Memory Code V1: Slot Id Generation
C P U
C P U
CREATE TABLE [dbo].[NonBlockingSequence](
[ID] [bigint] IDENTITY(1, 1) NOT NULL
,PRIMARY KEY NONCLUSTERED HASH ( [ID] )
WITH ( BUCKET_COUNT = )
) WITH ( MEMORY_OPTIMIZED = ON
,DURABILITY = SCHEMA_AND_DATA )
CREATE PROCEDURE [dbo].[GetSlotId] @Slot int OUTPUT
int
WITH NATIVE_COMPILATION, SCHEMABINDING AS
BEGIN ATOMIC
WITH ( TRANSACTION ISOLATION LEVEL = SNAPSHOT
,LANGUAGE = N'us_english')
INSERT INTO dbo.NonBlockingSequence
DEFAULT VALUES;
= SCOPE_IDENTITY();
END;

21. LMAX Queue In Memory Code V1: Slot Id Generation
C P U
C P U
CREATE TABLE [dbo].[NonBlockingSequence](
[ID] [bigint] IDENTITY(1, 1) NOT NULL
,PRIMARY KEY NONCLUSTERED HASH ( [ID] )
WITH ( BUCKET_COUNT = )
) WITH ( MEMORY_OPTIMIZED = ON
,DURABILITY = SCHEMA_AND_DATA )
CREATE PROCEDURE [dbo].[GetSlotId] @Slot int OUTPUT
int
WITH NATIVE_COMPILATION, SCHEMABINDING AS
BEGIN ATOMIC
WITH ( TRANSACTION ISOLATION LEVEL = SNAPSHOT
,LANGUAGE = N'us_english')
INSERT INTO dbo.NonBlockingSequence
DEFAULT VALUES;
= SCOPE_IDENTITY();
END;

22. LMAX Queue In Memory Code V1: Push Message Procs
C P U
C P U
CREATE PROCEDURE [dbo].[LmaxPushImOltp] AS
BEGIN
DECLARE @QueueSize int =
int
int
int = 0;
BEGIN TRAN
EXEC
ROLLBACK TRAN;
EXEC dbo.PushMessageImOltp @Slot
OUTPUT;
= 0
END
ELSE
+= 1;
END;
CREATE PROCEDURE [dbo].[PushMessageImOltp] @Slot int
int
OUTPUT
WITH NATIVE_COMPILATION, SCHEMABINDING AS
BEGIN ATOMI
WITH ( TRANSACTION ISOLATION LEVEL = SNAPSHOT
,LANGUAGE = N'us_english')
int = ;
UPDATE [dbo].[MyQLmaxImOltp]
SET time = GETDATE()
,message = 'Hello world'
,message_id
,reference_count = reference_count + 1
WHERE Slot
AND reference_count = 0;
SET @MessagePushed =
END;

23. How Well Does This Scale ?
C P U
C P U

24. Where Is The CPU Time Going ?

25. The push and get slot id procedures rolled into one
Lets Try Reducing The Number Of Procedures Called
C P U
C P U
ALTER PROCEDURE PushMessageImOltp @Slot int
int OUTPUT
WITH NATIVE_COMPILATION, SCHEMABINDING AS
BEGIN ATOMIC
WITH ( TRANSACTION ISOLATION LEVEL = SNAPSHOT
,LANGUAGE = N'us_english')
int = ;
INSERT INTO [dbo].[NonBlockingSequence]
DEFAULT VALUES;
= SCOPE_IDENTITY()
UPDATE [dbo].[MyQLmaxImOltp]
SET time = GETDATE()
,message = 'Hello world'
,message_id
,reference_count = reference_count + 1
WHERE Slot
AND reference_count = 0;
SET @MessagePushed =
END;
The push and get slot id procedures rolled into one

26. This Is The New Throughput Graph
C P U
C P U
44% improvement !!!

27. 41 % of the total CPU time is being expended on one spinlock !!!
When Throughput Falls Off A Cliff, Where Is The CPU Time Going ?
C P U
C P U
41 % of the total CPU time is being expended on one spinlock !!!

28. What Do The .DLLs In The Call Stack Represent ?
C P U
C P U
Language Processing T-SQL Interpreter, Statistics Collection, result set rendering and Query Optimization
SQLLANG.dll
Iterators, Memory Grants, Latches, Spinlocks, Column Store Batch Engine
Query Execution
In Memory OLTP Engine
Expression Service
Expression evaluation, basic compression, data type handling and conversion
Query Data Store
Hekaton.dll, <native compiled proc.dll>,
<in memory table.dll>
SQLMIN.dll
SQLTST.dll
QDS.dll
Storage Engine
T-SQL Interpreter, Statistics Collection and Query Optimization
SQLMIN.dll
SQL OS
Threads, Memory management framework, synchronization
SQLDK.dll, SQLOS.dll
The in-memory OLTP engine breaks this clean layering model . . .

29. What Hekaton.dll uses from SQLMIN.dll
Hekaton.dll and Its Host Engine SQLMIN.dll
C P U
C P U
Hekaton.dll
What Hekaton.dll uses from SQLMIN.dll
The metadata cache spinlock cmedhashset
Backup and restore
The logging infrastructure for the logging of undo
The management of ‘Joint’ transactions, transactions which span both engines
The allocation of large pages (64 ~ 256Kb)

30. What Is Spinlock<62,16,1> Doing ?
C P U
C P U

31. The Killer Metadata Cache Protection Spinlock !!!
C P U
C P U
CMED_HASH_SET !!!
Synchronises access to the meta data cache
Used to check that objects have not been dropped prior to query execution
Other than to use a fast CPU, there is little that can be done about this

32. Spinlock Activity Looks And The Disk Row-store Engine
C P U
C P U

33. Spinlock Activity Looks And The In-Memory Engine
C P U
C P U

34. Do Not Try This At Home !!! The database engine is coded
C P U
C P U
The database engine is coded
such that the spinlock to
protect metadata cache is not
taken out if you run a workload
inside a system database

35. What About Popping Messages Off The Queue ?
C P U
C P U
ALTER PROCEDURE PushMessageImOltp @MessagePushed int OUTPUT
WITH NATIVE_COMPILATION, SCHEMABINDING AS
BEGIN ATOMIC
WITH ( TRANSACTION ISOLATION LEVEL = SNAPSHOT
,LANGUAGE = N'us_english')
int = ;
INSERT INTO [dbo].[NonBlockingPushSequence]
DEFAULT VALUES;
= SCOPE_IDENTITY()
UPDATE [dbo].[MyQLmaxImOltp]
SET time = GETDATE()
,message = 'Hello world'
,message_id
,reference_count = reference_count + 1
WHERE Slot
AND reference_count = 0;
SET @MessagePushed =
END;

36. What About Popping Messages Off The Queue ?
C P U
C P U
CREATE PROCEDURE PopMessageImOltp @MessagePopped CHAR(300) OUTPUT
WITH NATIVE_COMPILATION, SCHEMABINDING AS
BEGIN ATOMIC
WITH ( TRANSACTION ISOLATION LEVEL = SNAPSHOT
,LANGUAGE = N'us_english')
DECLARE int =
@Slot int;
INSERT INTO [dbo].[NonBlockingPopSequence]
DEFAULT VALUES;
= SCOPE_IDENTITY()
UPDATE [dbo].[MyQLmaxImOltp]
SET time = GETDATE()
= message
,message_id
,reference_count = 0
WHERE Slot
AND reference_count = 1;
END;

37. How The Push and Pop Working Together Scale . . .
C P U
C P U

38. What Have We Learned ? C P U C P U
The naïve approach to implementing a queue is throttled by the “Last page problem”
We can overcome the last page problem using the LMAX disruptor pattern, but we need to craft a scalable sequence generator and avoid push and pop threads from hitting the same page when using the legacy engine
We pay a performance penalty by switching between the ‘Host’ and in-memory engines
Using the in-memory engine can still result in spinlock pressure due its use of the legacy engine as a “Host engine”

39. Please review the event and sessions
11/8/2018 |
Footer Goes Here

40. My Contact Details C P U C P U chris@exadat.co.uk
ChrisAdkin8