Thinking in Sets and SQL Query Logical Processing

Step-by-step flow Code where to find info Code how to find info Procedural thinking

Set thinking Tell SQl what to find Result = unordered set To sort it use ORDER BY

What is a set? SQL Server is based on the Relational Model, which is based on set theory and predicate logic. By a ‘set’ we mean any collection M into a whole of definite, distinct objects m (which are called the ’elements’ of M) of our perceptions or of our thought. Joseph W. Dauben and Georg Cantor Whole: A set is considered as a single entity, not individual objects Distinct:Every object in a set is unique SQL deviates from SETS in that it allows duplicates SQL translates to a GROUP BY Perception: The definition of a set is subjective

Set ASet B 0 Set Theory – Venn Diagram Domain {0,1,2,3,4,5,6,7,8,9} A = {1,2,3,4,5,6} B = {2,5,6,7,8,9} A ∩ B = {2,5,6} A U B = {1,2,3,4,5,6,7,8,9} A’ = {0,7,8,9} B’ = {0,1,3,4} 4

WHAT DECLARE Sets are not ordered T-SQL uses a declarative programming paradigm Tell it WHAT you want, not HOW to get it. SQL Server figures out how. as opposed to procedural languages which use the imperative paradigm When dealing with one row at a time – you have a CURSOR and it is not set-based WHILE loops & CURSORS are very inefficient The cost of the query increases with a PER ROW overhead Thinking in Sets

SQL Query Logical Processing Elements of the SELECT Statement & Order in which the Query Clauses are logically processed (not physically) FROM WHERE GROUP BY HAVING SELECT ORDER BY Anatomy of a SQL query

SQL Query Logical Processing FROM NAME of tables TABLE OPERATORS JOIN (INNER, OUTER (LEFT, RIGHT & FULL) APPLY (CROSS, OUTER) PIVOT UNPIVOT Best Practices SCHEMA Based (reduces name resolution time) Not to use Delimiters so avoid SPACES….. EmployeeName is better than [Employee Name] Alias (AS ME)

SQL Query Logical Processing WHERE Predicate Logic Normal Seek 3-Valued Predicate Logic: Must Evaluate to TRUE TRUE FALSE UNKNOWN – SQL Doesn’t LIKE NULLs Best Practise Filter the SET – keep it skinny Future proof your SQL Use the most appropriate SEEK predicate to ensure performance that does not degrade over time INDEX Definition Cant have too many (INSERT, DELETE & UPDATE – pay for each index) LEFT MOST Column Cardinality Estimation based on STATISTICS

GROUP BY Arrange rows from previous phase in GROUPs Determined by elements that you specify Produces a group for each UNIQUE combination All subsequent phases must now operate on the groups and not the rows Elements not in GROUP BY are allowed only as inputs to an aggregation function – COUNT, SUM, MIN, MAX Example: GROUP BY YEAR(InsertDateTime), MONTH(InsertDateTime) HAVING, SELECT & ORDER BY can’t refer to InsertDateTime No guarantees SINGLE row for the group All aggregate Answers relate to the TOP row All aggregate function ignore NULL – except COUNT(*) I.To get DISTINCT values, use COUNT(DISTINCT x) SQL Query Logical Processing

HAVING FILTER Groups instead of Rows Only takes TRUE, FALSE & UNKNOWN excluded COUNT(*) > 1 – Easy to find duplicates

SQL Query Logical Processing SELECT SQL allows you to return columns with no names – not relational Use Alias to ensure that you have a relational table Processed after the FROM, WHERE, GROUP, HAVING Means aliases defined at this level CAN NOT be used by previous phases “All-at-once” Operations Best Practice No missing Column Names Each column new line – if comma is left out the 2 nd part is assumed to be an alias: “EmployeeID EmployeeName” will be returned as EmployeeName Don’t use the “*” – performance cost; can force a PK lookup if not part of the include section of the index Aim to return a SET – increases Reuse Unique rows with no guaranteed order Duplicates give you a BAG Will not fail when used with an INTO

SQL Query Logical Processing Proof the SELECT statement is the last to be processed Using EmployeeCount in the HAVING causes a compiler error

SQL Query Logical Processing ORDER BY Presentation purposes Very last phase as part of the logical processing Result can not qualify as a table – it is a CURSOR Reduces reuse TOP & OFFSET / FETCH Filters Deterministic & Non-Deterministic TOP without ORDER BY gives no guarantee Best practises Do not use ORDINAL Order By (ORDER BY 1,2) Order by what is SELECTED otherwise the DISTINCT will fail Important for TDD

Join Theory and Relational Algebra A join merges rows from one table with rows from another table creating a new set of rows that includes columns from both. The SELECT statement and WHERE clause can access these columns and rows from both tables. ∩ NameCode Smith101 NameCodeOrder Smith1011 CodeOrder 1011

Types of Joins Self Join A join that refers back to the same table Recursive Like a table joined with a temporary copy Union Addition Merges two tables vertically by stacking one table above another table and lining up the columns Appends the results of one SELECT at the end of the results of another SELECT

Joins Join TypeQuery Designer Symbol Definition Inner JoinIncludes only matching rows Left Outer Join All rows from the LEFT table regardless of whether a match exists, and matching rows from the right table Full Outer Join Includes all rows from both tables regardless of whether a match exists

Inner Joins The intersection of 2 tables Default Usually Primary Key and Foreign Key match, but can be any column as long as the data types match When a row from Table1 matches a row from Table2, the result is a new row with data from both tables Can appear to multiply rows SELECT * FROM Table1 INNER JOIN Table2 ON Table1.column = Table2.column

Table1 Table2 Table1 Table2 Right Outer Join (Not used in MoBatch) All data from both tables Like a union of results from LEFT OUTER JOIN and RIGHT OUTER JOIN Matches where it can, adds NULLs where it can’t Used for understanding all data or cleaning out bad data Left Outer Join Full Outer Join SELECT *

Thank you