Applications hitting a wall today with SQL Server Locking/Latching Scale-up Throughput or latency SLA Applications which do not use SQL Server.

Slides:



Advertisements
Similar presentations
Yukon – What is New Rajesh Gala. Yukon – What is new.NET Framework Programming Data Types Exception Handling Batches Databases Database Engine Administration.
Advertisements

SQL Server performance tuning basics
new database engine component fully integrated into SQL Server 2014 optimized for OLTP workloads accessing memory resident data achive improvements.
6 SQL Server Integration Same manageability, administration & development experience Integrated queries & transactions Integrated HA and backup/restore.
SQL Performance 2011/12 Joe Chang, SolidQ
Project “Hekaton” adds in-memory technology to boost performance of OLTP workloads in SQL Server.
Dos and don’ts of Columnstore indexes The basis of xVelocity in-memory technology What’s it all about The compression methods (RLE / Dictionary encoding)
Meanwhile RAM cost continues to drop Moore’s Law on total CPU processing power holds but in parallel processing… CPU clock rate stalled… Because.
Applications hitting a wall today with SQL Server Locking/Latching Scale-up Throughput or latency SLA Applications which do not use SQL Server today.
Convergence /20/2017 © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks.
Database Systems Design, Implementation, and Management Coronel | Morris 11e ©2015 Cengage Learning. All Rights Reserved. May not be scanned, copied or.
Physical Database Design & Performance. Optimizing for Query Performance For DBs with high retrieval traffic as compared to maintenance traffic, optimizing.
Module 7 Reading SQL Server® 2008 R2 Execution Plans.
Ashwani Roy Understanding Graphical Execution Plans Level 200.
SQL Server 2014: In In-memory OLTP for Database Developers.
Primary Key, Cluster Key & Identity Loop, Hash & Merge Joins Joe Chang
SQL Server 2014 adds in-memory technology to boost performance of OLTP workloads.
6 1 Lecture 8: Introduction to Structured Query Language (SQL) J. S. Chou, P.E., Ph.D.
Parallel Execution Plans Joe Chang
IN-MEMORY OLTP By Manohar Punna SQL Server Geeks – Regional Mentor, Hyderabad Blogger, Speaker.
Database Systems Design, Implementation, and Management Coronel | Morris 11e ©2015 Cengage Learning. All Rights Reserved. May not be scanned, copied or.
Drsql.org How In-Memory Affects Database Design Louis Davidson Certified Nerd.
Srik Raghavan Principal Lead Program Manager Kevin Cox Principal Program Manager SESSION CODE: DAT206.
Meet Kevin Liu Principal Lead Program Manager Kevin Liu has been with Microsoft and the SQL Server engine team for 7 years, working on key projects like.
Ἑ κατόν by Niko Neugebauer. Niko Neugebauer PASS EvangelistPASS Evangelist SQL Server MVPSQL Server MVP SQLPort ( founder & leaderSQLPort.
AL-MAAREFA COLLEGE FOR SCIENCE AND TECHNOLOGY INFO 232: DATABASE SYSTEMS CHAPTER 7 (Part II) INTRODUCTION TO STRUCTURED QUERY LANGUAGE (SQL) Instructor.
Indexes and Views Unit 7.
Moore’s Law means more transistors and therefore cores, but… CPU clock rate stalled… Meanwhile RAM cost continues to drop.
CS 257 Chapter – 15.9 Summary of Query Execution Database Systems: The Complete Book Krishna Vellanki 124.
Applications hitting a wall today with SQL Server Locking/Latching Scale-up Throughput or latency SLA Applications which do not use SQL Server today.
1 Copyright © 2005, Oracle. All rights reserved. Following a Tuning Methodology.
Last Updated : 27 th April 2004 Center of Excellence Data Warehousing Group Teradata Performance Optimization.
2010 Ami Levin. SQL Server implements three different physical operators to perform joins. In this session we will see how each of these three operators.
IMS 4212: Database Implementation 1 Dr. Lawrence West, Management Dept., University of Central Florida Physical Database Implementation—Topics.
Query Processing – Implementing Set Operations and Joins Chap. 19.
7 1 Database Systems: Design, Implementation, & Management, 7 th Edition, Rob & Coronel 7.6 Advanced Select Queries SQL provides useful functions that.
1 Overview of Query Evaluation Chapter Outline  Query Optimization Overview  Algorithm for Relational Operations.
How to kill SQL Server Performance Håkan Winther.
In-Memory OLTP The faster is now simpler in SQL Server 2016.
SQL Server Statistics DEMO SQL Server Statistics SREENI JULAKANTI,MCTS.MCITP,MCP. SQL SERVER Database Administration.
Execution Plans Detail From Zero to Hero İsmail Adar.
Vedran Kesegić. About me  M.Sc., FER, Zagreb  HRPro d.o.o. Before: Vipnet, FER  13+ years with SQL Server (since SQL 2000)  Microsoft Certified.
SQL Server Statistics DEMO SQL Server Statistics SREENI JULAKANTI,MCTS.MCITP SQL SERVER Database Administration.
SQL Server 2014: In-Memory OLTP Adoption Considerations Mike
SAP Tuning 실무 SK㈜ ERP TFT.
Redmond Protocols Plugfest 2016 Jos de Bruijn, Borko Novakovic SQL In-Memory OLTP Senior Program Manager.
Oracle Announced New In- Memory Database G1 Emre Eftelioglu, Fen Liu [09/27/13] 1 [1]
SQL Server 2016: Real-time operational analytics
Indexing strategies and good physical designs for performance tuning Kenneth Ureña /SpanishPASSVC.
Use Cases for In-Memory OLTP Warner Chaves SQL MCM / MVP SQLTurbo.com Pythian.com.
SQL Server Statistics and its relationship with Query Optimizer
In-Memory Capabilities
Indexes By Adrienne Watt.
Stored Procedures – Facts and Myths
UFC #1433 In-Memory tables 2014 vs 2016
Database Performance Tuning and Query Optimization
මොඩියුල විශ්ලේෂණය Buffer Pool Extension භාවිතය.
Super Scaling The LMAX Queue Pattern.
Migrating a Disk-based Table to a Memory-optimized one in SQL Server
Physical Join Operators
Real world In-Memory OLTP
SQL 2014 In-Memory OLTP What, Why, and How
Steve Hood SimpleSQLServer.com
Statistics for beginners – In-Memory OLTP
In-Memory OLTP for Database Developers
Four Rules For Columnstore Query Performance
Chapter 8 Advanced SQL.
Chapter 11 Database Performance Tuning and Query Optimization
CS222/CS122C: Principles of Data Management UCI, Fall 2018 Notes #03 Row/Column Stores, Heap Files, Buffer Manager, Catalogs Instructor: Chen Li.
SQL Server 2016 High Performance Database Offer.
Presentation transcript:

Applications hitting a wall today with SQL Server Locking/Latching Scale-up Throughput or latency SLA Applications which do not use SQL Server today Key/Value pair where desire relational characteristics Scenarios where previously might not have implemented database in the critical path Common Patterns o High Data Input Rate – “Shock Absorber” o Read Performance and Scale o Compute Heavy Data Processing o Latency critical OLTP o Session State

Implementation Scenario Pattern Characteristics and Challenge Main Benefits of In-Memory OLTP High Data Insert Rate  Primarily append-only store  Inability to ingest write workload  Eliminate contention  Minimize IO logging Read Performance and Scale  High performance read operations  Unable to meet scale-up requirements  Eliminate contention  Efficient data retrieval  Minimize code execution time  CPU efficiency for scale

Implementation Scenario Pattern Characteristics and ChallengeMain Benefits of In-Memory OLTP Compute Heavy Data Processing  Insert/Update/Delete workload  Heavy computation inside database  Read and write contention  Eliminate contention  Minimize code execution time  Efficient data processing Low Latency  Require low latency business transactions  High concurrency exacerbates latency  Eliminate contention  Minimize code execution time  Efficient data retrieval Session State Management  Heavy insert, update, point lookups  User scale under load from multiple stateless web servers  Eliminate contention  Efficient data retrieval  IO reduction/removal

Client Connectivity Query Execution Data Access (Buffer Pool) Transaction Logging Client Connectivity Procedure Execution Data Access (Memory) Transaction Logging No Improvement Same Latency Potentially Less Volume for Durable I/O Eliminated for Non-Durable Traditional Execution StackIn-Memory OLTP Execution Stack Gains 2X to 30X Improvement *Including concurrency gains

SELECT TOP (5) b.name AS TableName, a.database_id, a.singleton_lookup_count, a.range_scan_count FROM sys.dm_db_index_operational_stats(DB_ID(), NULL, NULL, NULL) AS a INNER JOIN sys.objects b on a.object_id = b.object_id WHERE b.type <> 'S‘ AND (a.singleton_lookup_count > 0 OR a.range_scan_count > 0) ORDER BY a.singleton_lookup_count DESC GO Manual Analysis if MDW is too heavy SELECT TOP (5)a.database_id, so.object_id, so.name AS TableName, a.page_la tch_wait_count, a.page_latch_wait_in_ms, a.page_lock_wait_count, a.page_lock_wait_in_ms FROM sys.dm_db_index_operational_stats(DB_ID(), NULL, NULL, NULL) a INNER JOIN sys.objects AS so ON a.object_id = so.object_id WHEREso.type = 'U' AND a.page_io_latch_wait_coun t > 0 ORDER BY a.page_latch_wait_count DESC SELECT TOP (10) sp.database_id, so.name AS StoredProcName, sp.total_worker_time, sp.execution_count, sp.total_logical_reads, sp.total_logical_writes, sp.total_logical_reads FROM sys.dm_exec_procedure_stats AS sp INNER JOIN sys.objects AS so ON (sp.object_id = so.object_id) WHEREso.type = 'P' AND sp.database_id = DB_ID() ORD ER BY sp.total_worker_time DESC; T-SQL latch

ostress.exe -S. –E -dAdventureWorks2012 -Q"EXEC =, order_count=100000" – n

Identify bottlenecks in tables Address unsupported constructs and migrate data Perform minimal changes for interpreted access Identify performance critical transaction code Address surface area limitations and migrate code Taking an Iterative Approach to Migration

Memory Size = Table Size + SUM(Index Size) Table Size = Row Size * Row Count Row Size = SUM(Column Sizes) + Row Header Size Row Header Size = * Index Count Column Size = Dependent on column type and associated padding/overhead Hash Index Size = Bucket_Count * 8 bytes Nonclustered Index Size = Row Count * (Key Size + 8) bytes Details available in Books Online

OperationNonclusteredNonclustered Hash Specify Bucket CountNot RequiredRequired Index Scan, retrieve all table rowsSupported Index seek on equality predicate(s) (=) Supported*Performs better than Nonclustered. Predicate must include all columns in hash key, otherwise will result in scan Index seek on inequality predicates (>, =), BETWEEN Performs better than Nonclustered Hash* Will result in scan Retrieve rows in a sort-order matching the index definition SupportedNot Supported Retrieve rows in a sort-order matching the reverse of the index definition Not Supported * Predicate does not need to include all key columns but must include at least leading key columns, otherwise will result in scan

Lack of parallelism can be problematic for some queries Consider trade-off between faster execution (native) vs. parallel operation Joins between memory-optimized table and columnstore can be problematic Statistics creation and updates can be problematic for large tables Requirement of FULLSCAN can take a long time If query has many attributes individual column create stats can have an impact Consider impact of no auto-update statistics For InterOp queries: determining how often and when can affect plans For native procedures: Do data load, execute statistics update, create native compiled procedure A few difficult scenarios to convert to native compiled queries Disjunction (NOT/OR) Outer Join

Interpreted T-SQL Access – Access both memory- and disk-based tables – Less performant – Virtually full T-SQL surface When to use – Ad hoc queries – Surface area not available in native – Compilation at runtime can impact plan (positively) Natively Compiled Procs ‒ Access only memory- optimized tables ‒ Maximum performance ‒ Limited T-SQL surface area When to use ‒ Well defined OLTP-style operations ‒ Optimize performance critical business logic ‒ Query plan unlikely to change based on statistics

Review