In the name of the resources - aggregating data Dejan Dular In the name of the resources - aggregating data

Sponsors

About me Dejan Dular .NET Developer SQL Developer Dynamics CRM specialist IoT fan DIY & eco maniac Twitter: @dejandular Mail: dejan@majolka.com LinkedIn: dejandular

Execution plan is your friend Include actual execution plan (seek for seek) Review suggested missing index – CHANGE THE NAME! Compare client statistics Discard results after query execution [Tools-Options-Query results-Results to grid] Use external tools SSMS Tools Pack www.ssmstoolspack.com SQL Sentry Plan Explorer www.sentryone.com/plan-explorer Test hypothetical indexes WITH (STATISTICS_OLNY = 1) and AUTOPILOT https://sqlperformance.com/2014/07/sql-indexes/new-index-columns-key-vs-include https://technet.microsoft.com/en-us/library/ms172984(v=sql.110).aspx https://www.simple-talk.com/sql/database-administration/hypothetical-indexes-on-sql-server/

CTE vs @temp vs #temp SPEED CTE "Sub-query" No index or statistics In memory Re-run possible SPEED * most of the time Table variable In memory* No index or statistics Treated as one row Reusable Temporary table has statistics can be indexed https://blogs.msdn.microsoft.com/craigfr/2007/10/18/ctes-common-table-expressions/ http://thinknook.com/sql-server-table-variable-vs-temporary-table-vs-cte-2012-01-12/

Getting OVER() window functions SELECT s.TerritoryID, p.FirstName, p.LastName, s.SalesYTD ROW_NUMBER() OVER(PARTITION BY s.TerritoryId ORDER BY s.SalesYTD) FROM Sales.SalesPerson s JOIN Person.Person p ON s.BusinessEntityID = p.BusinessEntityID ORDER TerritoryID FirstName LastName SalesYTD Row Number 1 Pamela Ansman-Wolfe 1.352.577,13 David Campbell 1.573.012,94 2 Tete Mensa-Annan 1.576.562,20 3 Michael Blythe 3.763.178,18 Jillian Carson 3.189.418,37 4 Shu Ito 2.458.535,62 Linda Mitchell 4.251.368,55 PARTITION http://blogs.lessthandot.com/index.php/DataMgmt/DBProgramming/MSSQLServer/t-sql-window-functions-part1/ http://blogs.lessthandot.com/index.php/DataMgmt/DBProgramming/MSSQLServer/t-sql-window-functions-part-2/ http://blogs.lessthandot.com/index.php/DataMgmt/DBProgramming/MSSQLServer/t-sql-window-functions-part-03/ http://blogs.lessthandot.com/index.php/DataMgmt/DBProgramming/MSSQLServer/t-sql-window-functions-part-4/ http://www.vertabelo.com/blog/technical-articles/oracle-sql-analytical-functions-for-beginners-a-gentle-introduction-to-common-sql-window-functions http://www.vertabelo.com/blog/technical-articles/exploring-window-functions-in-sql-server-part-two-aggregating-data

CROSS APPLY Get these rows, use (apply) them to this set and display the data Like a JOIN. Same same, but different. In most cases the same execution plan when only joining the tables Possible to use the rowset on a table valued UDF SELECT P.Id, P.FirstName, P.LastName, O.OrderDate, O.OrderStatus FROM Person P CROSS APPLY ( SELECT TOP 1 OrderDate, OrderStatus FROM SalesOrder SO WHERE SO.CustomerId = P.ID ORDER BY OrderDate DESC ) O WHERE P.zip = '94604' https://explainextended.com/2009/07/16/inner-join-vs-cross-apply/ http://sqlblog.com/blogs/alexander_kuznetsov/archive/2009/07/07/using-cross-apply-to-optimize-joins-on-between-conditions.aspx

Aggregate on the fly How many products do we have with Tax 0%, 9% and 22% in three price ranges <100€, 100-500€, >500€? Tax <100€ 100 – 500€ >=500€ 0% 2263 9201 11726 9% 7081 28700 36114 22% 10347 42001 52567

Aggregate, Then Join JOIN is slow Aggregate if possible to get less rows, then join https://www.periscopedata.com/blog/use-subqueries-to-count-distinct-50x-faster.html

Intervals ... WHERE Price BETWEEN 100 AND 500 What does BETWEEN x AND y mean? [x,y] | (x,y) | (x,y] | [x,y) Sum of sales over the years in Q1 Year Sales in Q1 2012 64422,11 2013 708132,64 2014 154322,22 2015 201678,12 2016 235213,17 https://sqlperformance.com/2013/01/t-sql-queries/generate-a-set-1

Anti-patterns OR between columns in different tables T1 JOIN T2… WHERE T1.col1 = x OR T2.col2 = y WHERE col1 = CASE @var WHEN 1 THEN 'val1' WHEN 2 THEN 'val2' ELSE col2 https://blogs.msdn.microsoft.com/sqlcat/2013/09/09/when-to-break-down-complex-queries/

Simulating production environment DBCC OPTIMIZER WHAT_IF (undocumented) DBCC Optimizer_WhatIf (1,32) --set 32 CPUs GO DBCC Optimizer_WhatIf (2,131072) --set 128 GB of RAM DBCC Optimizer_WhatIf (3,64) -- 64 bit system -- Execute query DBCC Optimizer_WhatIf (1, 0) --clear CPU DBCC Optimizer_WhatIf (2, 0) --clear RAM DBCC Optimizer_WhatIf (3, 0) --clear 64 bits SET OPTIONS Statistics Numbers of rows and pages in the table Available physical memory Number of CPUs 32bit / 64 bit Export metadata from production database https://support.microsoft.com/en-us/kb/914288 https://www.simple-talk.com/sql/database-administration/using-optimizer_whatif-and-statsstream-to-simulate-a-production-environment/

Very important slide Use correct data types https://youtu.be/ZV06H6yvC-g?t=294

Very important slide Use correct data types Avoid correlated sub-queries SELECT T1.Column1, Column2 = (SELECT T2.Column2 FROM T2 WHERE T2.Col2 = T1.Column2 ) FROM T1

Very important slide Use correct data types Avoid correlated sub-queries Avoid using functions within ON, WHERE or GROUP BY Remove unnecessary tables Foreign keys are not indexed by default NOT IN vs NOT EXISTS Query plan numbers are only estimations – NOT! Use query hints with caution If you are using cursors, you have to buy me a beer! WHERE Column1 = dbo.SomeFunction(Column2) SELECT Car.Id FROM Car JOIN Service ON Car.Id = Service.CarId * ISNULL() is also a function!

Another important slide Do not use query builders Investigate execution plans, check indexes, include selected columns Make as little scans as possible, seek is what you are looking for Limit the size of the working data set Only select fields you really need WITH(NOLOCK), WITH(READPAST) or ISOLATION LEVEL READ UNCOMMITED is OK only when you know what you are doing Configure Index and Table file partitions Upgrade SQL server to newest version

while (tooSlow) { Change(); Test(); Evaluate(); }

Que?

Resources Almost every page has at least one link in the notes section. Use it to learn more about that topic.

Session Evaluations: Event Evaluations: http://www.sqlsaturday.com/644/Sessions/EventEval.aspx Event Evaluations: http://www.sqlsaturday.com/644/EventEval.aspx

Window functions are your friends This room at 4 PM Unicorns and rainbows included!