There’s a particular style to it… Thinking in SQL There’s a particular style to it… Rob Hatton Email: rob@convitali.com Blog: RobertJHatton.com

SQL is not a complete language No way to build a user interface No way to read or write to a file Can’t print Can’t control how things happen Doesn’t even know about variables It’s awesome at retrieving data!

SQL Server has 2 languages ANSI SQL Declarative Based on Sets TSQL Procedural We’ll be talking about ANSI SQL

Types Of SQL Statements Data Definition Language (DDL) Create (Table, View, Proc) Alter (Table, View, Proc) Drop (Table, View, Proc) Data Manipulation Language (DML) Select Insert Delete Update We’re just talking about SELECT Database definition is stored in a database. DDL manipulates the definition. We’re not going to focus on DDL.

SQL is a different kind of language Most languages are procedural Like telling a story It’s the reason for TSQL SQL is a declarative language Like painting a picture The complete solution does both It’s a comic book Sets are key to understanding SQL

SQL & Relational Databases They work together! SQL works on sets Relational Databases house sets Both physical and virtual Sample database ERD: Fruit Fruit Color

Sets They’re just collections of values: Database Examples: Customers that owe you money Students & grades Cities and temperatures Database Examples: Tables Views Function return Virtual Sets Sub-queries Common Table Expressions

Entity Relationship Diagrams (ERD) Essential for understanding database Key to building joins Database has tables & columns that describe database Pick up Database Spelunking scripts RobertJHatton.com select * from sys.tables t inner join sys.columns c on t.object_id = c.object_id where c.name = 'salesOrderID'

Demo Database Fruit Database Colors table Fruit table Red Yellow Green Orange Fruit table Banana Grape Apple Eaten

Select Syntax SELECT field_list [ FROM set_source ] (includes joins) [ WHERE search_expression ] (record based) [ GROUP BY group_expression ] [ HAVING aggregate_expression ] [ ORDER BY order_expression [ ASC | DESC ] ] Where clause logic is record based, not set based! ‘Where eyeColor = ‘blue’ and eyeColor = ‘green’ returns no records. ‘Group by’ requires an aggregate function in select list. ‘Having’ only works with group by. Where clause logic is record based, not set based! ‘Where eyeColor = ‘blue’ and eyeColor = ‘green’ returns no records. ‘Group by’ requires an aggregate function in select list. ‘Having’ only works with group by.

Functions and Expressions UPPER() - Converts a field to upper case SUBSTR() - Returns part of a character expression LTRIM() – Remove spaces from left end of text field LEN() - Returns the length of a text field GETDATE() - Returns the system date and time CHARINDEX() – Returns start of string in another string Case When Beware of misuse

Comparison Operators = < <= > <> ! (not) Between Exists In Is [not] null Remember, they are record based not set based!

The importance of NULL NULL is missing data NULL is not the same as zero, or an empty string Needed to calculate valid aggregates Example: When calculating average dress size, it keeps the guys from spoiling the average Anything combined with NULL is NULL Any relational operator using a NULL returns NULL The ISNULL() function helps deal with NULLs

Aliasing select fis.orderDateKey , max(fis.salesAmount) as maxAmt from factInternetSales fis group by fis.orderDateKey Table Alias: fis Field Alias: maxAmt Use an acronym or abbreviation Don’t use ‘a’, ‘b’, ‘c’!

Select examples * Intrinsic sets From a table From a view select 'just some text' as txt select getdate() as dt From a table select dG.City from DimGeography dG From a view select tm.lastName from vTargetMail tm From a subquery select color from (select 'blue' as color) as colors --Select * is bad, it eats memory! use [AdventureWorksDW2012] go select 'just some text' as txt select getdate() as dt select City from DimGeography select lastName from vTargetMail select color from (select 'blue' as color) as colors *

Joins * Require a comparison operator between 2 sets Types Inner Outer Left Right Cross Full Self Exists Joins work with any comparison operator! Joins aren’t as much of a problem as data size! --Example Queries go use Fruit select * from fruit select * from colors from fruit select * --Use 'where' clause where colorKey = 1 --Dealing with null where colorKey is null --where colorKey = null --Wildcard --where fruitName like '%e ' where fruitName like '%e%' --where rtrim(fruitName) like '%e' --'In' clause where fruitName in ('apple','grape') --In with a subquery where fruitName in (select fruitName from fruit where rtrim(fruitName) like '%e') --Everything from inner join from fruit f --Relevent columns from inner join on f.colorKey = c.colorKey inner join colors c select f.fruitName, c.color --Everything from left outer join left outer join colors c --Everything from right outer join right outer join colors c --Everything from Cross join (match everything with everything - no 'on' condition) --on f.colorKey = c.colorKey cross join colors c --Everything from full join (combination of left & right outer joins) full join colors c select fruitName --Check for Exists (select * where exists from colors c --Alternate version of Exists where c.colorKey = f.colorKey) where f.colorKey is not null on c.colorKey = f.colorKey select colorKey, count(*) as cnt --Aggregations having count(*) > 1 group by colorKey use [AdventureWorksDW] --Self join from DimAccount c select c.AccountDescription, p.AccountDescription order by c.accountKey on p.ParentAccountKey = c.AccountKey left outer join DimAccount p *

Common Table Expressions CTE Create a set on the fly Can only be used in a single query Doesn’t actually create the set Think of it like an include Optimizer doesn’t see a separate set --Common Table Expressions with northAmerica as (select [SalesTerritoryKey], [SalesTerritoryRegion] from DimSalesTerritory where [SalesTerritoryGroup] = 'North America') select top 3 na.SalesTerritoryRegion, fis.SalesAmount from FactInternetSales fis inner join northAmerica na on na.SalesTerritoryKey = fis.SalesTerritoryKey *

Union * Combine 2 sets with matching fields Useful in replacing Case Field count must match exactly Data types must be implicitly convertible First query dictates names Useful in replacing Case Implicitly removes duplicates Use ‘Union All’ to preserve duplicates --Union use [AdventureWorksDW2012] Go select distinct st.SalesTerritoryRegion from FactInternetSales fis inner join DimSalesTerritory st on fis.SalesTerritoryKey = st.SalesTerritoryKey union from FactResellerSales frs on frs.SalesTerritoryKey = st.SalesTerritoryKey *

Case when Two versions Alternative Case Case when Union When expression Case when Expression Alternative Union --Case When use [AdventureWorksDW2012] Go select c.AccountType , (case c.AccountType when 'Assets' then 'Balance Sheet' when 'Liabilities' then 'Balance Sheet' when 'Revenue' then 'Income Statement' when 'Expense' then 'Income Statement' end) as 'journal' from DimAccount c left outer join DimAccount p on p.ParentAccountKey = c.AccountKey --Union instead of case go select distinct AccountType, 'Balance Sheet' as Jounal from DimAccount where AccountType = 'Assets' union where AccountType = 'Liabilities' select distinct AccountType, 'Income Statement' as Jounal where AccountType = 'Revenue' where AccountType = 'Expense'

Correlated Subquery * A necessary evil Alternatives Appears in select list Subquery contains a join referencing the outer query Runs once for each record! Alternatives Join Creating a virtual record set (CTE) use [AdventureWorksDW] go --Correlated Subquery -- Note that not all employees have sales, so maxSales may be null select e.lastName ,(select max(SalesAmount) from FactResellerSales frs where frs.EmployeeKey = e.EmployeeKey group by frs.EmployeeKey) as maxSales from DimEmployee e order by e.LastName --Alternate version using join , max(frs.salesAmount) as maxSales left outer join FactResellerSales frs on frs.EmployeeKey = e.EmployeeKey group by e.LastName *

Cursors * Another necessary evil Procedural Slow More code than SQL Sometime may be the only solution use [AdventureWorks2012] go --Separate products into free, cheap and expensive select top 10 * from Production.Product --cursor version--------------------------------------------------------------------------------------------------------------------------- declare @prod nvarchar(50) declare @price money declare @priceGroup char(10) if object_id('tempdb..#priceGroups') is not null drop table #priceGroups create table #priceGroups (prod nvarchar(50), priceGroup char(10)) declare prices cursor for select p.Name, p.ListPrice from [Production].Product p open prices fetch next from prices into @prod, @price while @@FETCH_STATUS = 0 begin if @price = 0 insert into #priceGroups values (@prod, 'free') if @price > 0 and @price < 100 insert into #priceGroups values (@prod, 'cheap') if @price >= 100 insert into #priceGroups values (@prod, 'expensive') end close prices deallocate prices select * from #priceGroups --case version--------------------------------------------------------------------------------------------------------------------------- select Name, (case when listPrice = 0 then 'free' when listPrice > 0 and listPrice < 100 then 'cheap' when listPrice >= 100 then 'expensive' else 'priceless' end) as priceGroup from Production.Product --union version--------------------------------------------------------------------------------------------------------------------------- select Name, 'free' as priceGroup from Production.Product where ListPrice = 0 union select Name, 'cheap' as priceGroup from Production.Product where ListPrice between 0 and 100 select Name, 'expensive' as priceGroup from Production.Product where ListPrice > 100 *

Recursive SQL * Use a CTE Create an anchor condition Union All Query again calling anchor Choose what you want outside the CTE use AdventureWorksDW2012 go --Recursive CTE with r as (select c.AccountKey as ChildAccountKey, c.ParentAccountKey, 0 as lvl from DimAccount c where c.ParentAccountKey is null union all select p.AccountKey, p.ParentAccountKey, lvl + 1 from DimAccount p inner join r on p.ParentAccountKey = r.ChildAccountKey ) select parent.AccountDescription as parent, child.AccountDescription as child, lvl from r inner join DimAccount child on r.ChildAccountKey = child.AccountKey inner join DimAccount parent on r.ParentAccountKey = parent.AccountKey order by lvl, parent *

It’s important to understand data! Plan joins by drawing ERDs! Query system tables To find tables with matching foreign keys Profile data --Profile declare @sqlStmt varchar(700) declare @col varchar(50) declare @tbl varchar(50) declare @val varchar(100) set @fldSample = '' set @val = '' declare @fldSample varchar(200) DROP TABLE [dbo].[dbProfile] IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[dbProfile]') AND type in (N'U')) --Recreate the profile table CREATE TABLE [dbo].[dbProfile]( [recordCnt] [int] NULL, [ColumnName] [varchar](50) NOT NULL, [TableName] [varchar](50) NOT NULL, [distinctCnt] [int] NULL, [fldSample] varchar(300) [NullCnt] [int] NULL, [nonNullCnt] [int] NULL, ) ON [PRIMARY] SELECT t.name AS table_name, declare dbSchema cursor for c.name AS column_name ORDER BY table_name INNER JOIN sys.columns c ON t.OBJECT_ID = c.OBJECT_ID FROM sys.tables AS t while @@FETCH_STATUS = 0 fetch next from dbSchema into @tbl, @col open dbSchema begin EXEC (@sqlStmt) set @sqlStmt = 'declare fldList cursor for select distinct ' + @col + ' from ' + @tbl open fldList fetch next from fldList into @val while @@FETCH_STATUS = 0 and len(@fldSample) < 200 set @fldSample = @fldSample + rtrim(ltrim(@val)) + ', ' end deallocate fldList close fldList set @sqlStmt = 'insert into dbProfile (TableName, ColumnName, recordCnt, distinctCnt, nonNullCnt, nullCnt, fldSample) ' set @sqlStmt = @sqlStmt +'count(distinct ['+@col+']) as distinctCnt, ' set @sqlStmt = @sqlStmt +'count(*) as recordCnt, ' set @sqlStmt = @sqlStmt +'select '''+@tbl+''' as TableName ,'''+@col+''' as ColumnName, ' set @sqlStmt = @sqlStmt + '''' + left(@fldSample,200) + '''' set @sqlStmt = @sqlStmt +'sum(case len(rtrim(['+@col+'])) when 0 then 1 else 0 end) as NullCnt, ' set @sqlStmt = @sqlStmt +'sum(case len(rtrim(['+@col+'])) when 0 then 0 else 1 end) as nonNullCnt, ' set @sqlStmt = @sqlStmt +' from ' + @tbl begin try --select @sqlStmt end try end catch print(@sqlStmt) begin catch deallocate dbSchema close dbSchema select * from dbProfile *

There’s a particular style to it… Thinking in SQL There’s a particular style to it… Rob Hatton Email: rob@convitali.com Blog: RobertJHatton.com