SQL Server Internals & Architecture Naomi Williams, SQL DBA LinkedIn
2 Tips After Party / Networking Event 5:30pm -8:30pm at Jefferson’s 618 Georgia Avenue Chattanooga, TN #SQLHELP Get a copy, study it and become a master!
3 Agenda ACID SQLOS Components of database engine Components of storage engine Cache aging Differences of Hekaton
Why is there ACID in my SQL Server? ACID properties of Transactions Atomic Consistent Isolated Durable 4
Welcome to SQL Server! Where does the path begin? 5
6 SELECT SQL Server Interface aka API TDS packets SNI (SQL Server Client Network Interface) SNI (SQL Server Server Network Interface) Protocols TCP/IP Shared Memory Named Pipes Virtual Interface Adapter (VIA) SQLOS
SQLOS, Workers and Schedulers 7 SQLOS creates a set of schedulers upon start equal to the number of logical CPU’s. Each scheduler manages workers, for every task to execute it’s assigned a worker that is in an idle state. Workers do not move between schedulers, tasks are never moved between workers. However SQLOS can create child tasks and assign them different workers. Tasks can have one of six states Pending; task is waiting for an available worker Done; task is completed Running; task is currently executing Runnable; task is waiting for the scheduler Suspended; task is waiting for external event or resource Spinloop; task is processing a spinlock
SQLOS, Workers and Schedulers, cont Running state Suspended queue and suspended state Runnable queue and runnable state 8
9 SELECT SQL Server Interface aka API TDS packets SNI (SQL Server Client Network Interface) SNI (SQL Server Server Network Interface) Protocols TCP/IP Shared Memory Named Pipes Virtual Interface Adapter (VIA) Database Engine Cmd Parser Optimizer Query Plan Optimizer Query Plan Query Tree Language Event Query Executor Buffer Pool Data Cache Language Hash of execution plan Does the hash exist in the Plan Cache? If query hash does not exist in Plan Cache YES! Query has plan exists! Plan Cache SQLOS
Project (*) Filter (o.date=‘ ’) Inner Join c.cid=o.cid Get (customers) as C Get (orders) as O Logical Query Tree Select * from Customers c inner join Orders o on c.cid=o.cid where o.date=‘ ’; 10
11 SELECT SQL Server Interface aka API TDS packets SNI (SQL Server Client Network Interface) SNI (SQL Server Server Network Interface) Protocols TCP/IP Shared Memory Named Pipes Virtual Interface Adapter (VIA) Database Engine Cmd Parser Optimizer Query Plan Optimizer Query Plan Query Tree Language Event Query Executor Buffer Pool Data Cache Plan Cache SQLOS
Query Optimizer Parse and logical query tree Does it make sense and how can I get there? Binding, do they exist? Do the objects exist? Create path in binary from algebrizer for optimizer. Generate physical query plan Most efficient plan (path) to the data. 12
Physical Query Plan 13
14 SELECT SQL Server Interface aka API TDS packets SNI (SQL Server Client Network Interface) SNI (SQL Server Server Network Interface) Protocols TCP/IP Shared Memory Named Pipes Virtual Interface Adapter (VIA) Database Engine Cmd Parser Optimizer Query Plan Optimizer Query Plan Query Tree Language Event Query Executor Data File Transaction Log File Storage Engine Buffer Manager Access Methods Transaction Manager Buffer Pool Data Cache OLE DB Does Data exist in Cache? Retrieves data from disk to cache Plan Cache SQLOS
15 Insert, Update, Delete SQL Server Interface aka API TDS packets SNI (SQL Server Client Network Interface) SNI (SQL Server Server Network Interface) Protocols TCP/IP Shared Memory Named Pipes Virtual Interface Adapter (VIA) Database Engine Cmd Parser Optimizer Query Plan Optimizer Query Plan Query Tree Language Event Query Executor Data File Transaction Log File Storage Engine Buffer Manager Access MethodsTransaction Manager Lock and Log manager Buffer Pool Data Cache OLE DB Plan Cache SQLOS CheckPoint
16 WAIT, WAIT WAIT SOS_Scheduler_Yield Async_Network_IO Writelog, LogBuffer PageIOLatch_x, Async_IO_Completion, IO_Completion Pagelatch_x, Latch_x, Resource_Semaphore Locks LCK_x, LCK_M_x Latches OLEDB BackupIOCXPacket IO_Completion MSQL_XP MSQL_DQ PreEmptive_OS_ ThreadPool
17 WAIT, WAIT WAIT SQL Server Interface aka API TDS packets SNI (SQL Server Client Network Interface) SNI (SQL Server Server Network Interface) Protocols TCP/IP Shared Memory Named Pipes Virtual Interface Adapter (VIA) Database Engine Cmd Parser Optimizer Query Plan Optimizer Query Plan Query Tree Language Event Query Executor Data File Transaction Log File Storage Engine Buffer Manager Access MethodsTransaction Manager Lock and Log manager Buffer Pool Data Cache OLE DB Plan Cache SOS_Scheduler_Yield, CXPacket Async_Network_IO SQLOS Writelog, LogBuffer PageIOLatch_x, Async_IO_Completion, IO_Completion Pagelatch_x, Latch_x, Resource_Semaphore Locks LCK_x, LCK_M_x Latches
Data Cache and LRU-K 18 ABC AB C LAZY WRITER Data File A BC Buffer Pool Data Cache Lazy Writer 0 0 SQLOS Free Buffer List Every buffer has a header Last two times the page was referenced Status information
Plan Cache and LRU 19 ABC AB C Buffer Pool Plan Cache Resource Monitor 0 0 SQLOS ABC
20 Hekaton SQL Server Interface aka API TDS packets SNI (SQL Server Client Network Interface) SNI (SQL Server Server Network Interface) Protocols TCP/IP Shared Memory Named Pipes Virtual Interface Adapter (VIA) Database Engine Cmd Parser Optimizer Query Plan Optimizer Query Plan Query Tree Language Event Query Executor Data File Transaction Log File Storage Engine Buffer Manager Access MethodsTransaction Manager Lock and Log manager Buffer Pool Data Cache OLE DB Plan Cache SQLOS Natively compiled stored procedures and schema In-Memory OLTP Compiler Memory Optimized Tables and Indexes
Summary 21 ACID SQLOS Components of database engine Components of storage engine Cache aging Differences of Hekaton