1. Sorting and Filtering Data
Module 5
Sorting and Filtering Data

2. Working with Unknown Values
Module Overview
5: Sorting and Filtering Data
Working with Unknown Values

3. Demonstration: Sorting Data
Lesson 1: Sorting Data
5: Sorting and Filtering Data
Demonstration: Sorting Data
Question
If you declare an alias for a column in the SELECT clause, you cannot use that alias in the WHERE clause—but you can use it in the ORDER BY clause. Why is this?
Answer
This is because of the order in which clauses of a SELECT query are processed. The WHERE clause is processed before the SELECT column list; therefore, column aliases are not available for sorting. The ORDER BY clause is processed last, so column aliases are available and can be used without errors.

4. Using the ORDER BY Clause
5: Sorting and Filtering Data
ORDER BY sorts rows in results for presentation purposes
No guaranteed order of rows without ORDER BY
Use of ORDER BY guarantees the sort order of the result
Last clause to be logically processed
Sorts all NULLs together
ORDER BY can refer to:
Columns by name, alias or ordinal position (not recommended)
Columns not part of SELECT list
Unless DISTINCT specified
Declare sort order with ASC or DESC
ORDER BY determines the order of the data and does not affect the set of data selected.
Remind students that, without an ORDER BY, the lab exercises may return results in a different order from the supplied lab answers. If they want to check results, they can add an ORDER BY clause, both to their solution and the provided one. This will affect all labs!
ORDER BY sorts all NULLs together. See the next lesson for more information on NULL.
Use of ordinal position (for example, ORDER by 1,2) is not recommended as, if the SELECT clause changes the position of columns, the ORDER BY clause is not automatically updated to refer to new positions. This may lead to errors in client applications.
ASC is the default sort.
Note: ORDER BY may also include a COLLATE clause, which helps sort by a specific collation, not the collation of the column in the table. Collations will be discussed further in Module 6 of this course.

5. Writing ORDER BY using column names:
ORDER BY Clause Syntax
5: Sorting and Filtering Data
Writing ORDER BY using column names:
Writing ORDER BY using column aliases:
Specifying sort order in the ORDER BY clause:
SELECT <select list>
FROM <table source>
ORDER BY <column1_name>, <column2_name>;
SELECT <column> AS <alias>
FROM <table source>
ORDER BY <alias1>, <alias2>;
SELECT <column> AS <alias>
FROM <table source>
ORDER BY <column_name|alias> ASC|DESC;

6. ORDER BY Clause Examples
5: Sorting and Filtering Data
ORDER BY with column names:
ORDER BY with column alias:
ORDER BY with descending order:
SELECT orderid, custid, orderdate
FROM Sales.Orders
ORDER BY orderdate;
SELECT orderid, custid, YEAR(orderdate) AS orderyear
FROM Sales.Orders
ORDER BY orderyear;
SELECT orderid, custid, orderdate
FROM Sales.Orders
ORDER BY orderdate DESC;

7. Demonstration: Sorting Data
5: Sorting and Filtering Data
In this demonstration, you will see how to:
Sort data using the ORDER BY clause
Preparation Steps
All demonstrations in this module use a Microsoft Azure SQL Database, running a copy of the AdventureWorksLT database.
Before attempting to run these demos, ensure you have a copy of AdventureWorksLT running on an Azure instance.
For detailed steps on creating a copy of the AdventureWorksLT database in Azure see: D:\Creating an AdventureWorks Database on Azure.docx
Start the MSL-TMG1, 20761A-MIA-DC, and 20761A-MIA-SQL virtual machines.
Demonstration Steps
Ensure that the 20761A-MIA-DC, and 20761A-MIA-SQL virtual machines are running, and then log on to 20761A-MIA-SQL as ADVENTUREWORKS\Student with the password Pa$$w0rd.
Start SQL Server Management Studio and connect to your Azure instance of the AdventureWorksLT database engine instance using SQL Server authentication.
If the Microsoft SQL Server Management Studio dialog box appears, click OK.
Open the Demo.ssmssln solution in the D:\Demofiles\Mod05\Demo folder.
If the Solution Explorer pane is not visible, on the View menu, click Solution Explorer.
Expand Queries, and double-click the 11 – Demonstration A.sql script file.
In the Available Databases list, click ADVENTUREWORKSLT.
Select the code under the comment Step 1, and then click Execute.
Select the code under Step 2, and then click Execute.
Select the code under the comment Step 3 and then click Execute.
Select the code under the comment Step 4, and then click Execute.
Select the code under the comment Step 5, and then click Execute.
Select the code under the comment Step 6, and then click Execute.
Keep SQL Server Management Studio open for the next demonstration.

8. Lesson 2: Filtering Data with Predicates
5: Sorting and Filtering Data
Demonstration: Filtering Data with Predicates
Explain that the WHERE clause restricts the rows selected by the SELECT statement, working as a filter. The WHERE clause does not refer to location, but could be defined in ordinary English as ‘in those cases in which’.
Question
You have a table named Employees that includes a column named StartDate. You want to find who started in any year other than What query would you use?
Answer
Multiple answers are possible. For example:
SELECT FirstName, LastName
FROM Employees
WHERE Employees.StartDate < ' ' OR Employees.StartDate >= ' ';

9. Filtering Data in the WHERE Clause with Predicates
5: Sorting and Filtering Data
WHERE clauses use predicates
Must be expressed as logical conditions
Only rows for which predicate evaluates to TRUE are accepted
Values of FALSE or UNKNOWN filtered out
WHERE clause follows FROM, precedes other clauses
Can’t see aliases declared in SELECT clause
Can be optimized by SQL Server to use indexes
Data filtered server-side
Can reduce network traffic and client memory usage

10. Filter rows for customers from Spain
WHERE Clause Syntax
5: Sorting and Filtering Data
Filter rows for customers from Spain
Filter rows for orders after July 1, 2007
Filter orders within a range of dates
SELECT contactname, country
FROM Sales.Customers
WHERE country = N'Spain';
SELECT orderid, orderdate
FROM Sales.Orders
WHERE orderdate > ' ';
SELECT orderid, custid, orderdate
FROM Sales.Orders
WHERE orderdate >= ' ' AND orderdate < ' ';

11. Demonstration: Filtering Data with Predicates
5: Sorting and Filtering Data
In this demonstration, you will see how to:
Filter data in a WHERE clause
Preparation Steps
Complete the previous demonstration in this module. Alternatively, start the MSL-TMG1, 20761A-MIA-DC, and 20761A-MIA-SQL virtual machines, log on to 20761A-MIA-SQL as ADVENTUREWORKS\Student with the password Pa$$w0rd, and connect to your Azure copy of the AdventureWorksLT database.
Demonstration Steps
Filter data in a WHERE clause
Ensure that you have completed the previous demonstration in this module. Alternatively, start the MSL-TMG1, 20761A-MIA-DC, and 20761A-MIA-SQL virtual machines, log on to 20761A-MIA-SQL as ADVENTUREWORKS\Student with the password Pa$$w0rd.
If SQL Server Management Studio is not already open, start it and connect to your Azure instance of the AdventureWorksLT database engine instance using SQL Server authentication, and then open the Demo.ssmssln solution in the D:\Demofiles\Mod05\Demo folder.
In Solution Explorer, open the 21 – Demonstration B.sql script file.
In the Available Databases list, click ADVENTUREWORKSLT.
Select the code under the comment Step 1, and then click Execute.
Select the code under the comment Step 2, and then click Execute.
Select the code under the comment Step 3, and then click Execute.
Select the code under the comment Step 4, and then click Execute.
Select the code under the comment Step 5, and then click Execute.
Select the code under the comment Step 6, and then click Execute.
Select the code under the comment Step 7, and then click Execute.
Select the code under the comment Step 8, and then click Execute.
Select the code under the comment Step 9, and then click Execute.
Select the code under the comment Step 10, and then click Execute.
Select the code under the comment Step 11, and then click Execute.
Keep SQL Server Management Studio open for the next demonstration.

12. Lesson 3: Filtering Data with TOP and OFFSET-FETCH
5: Sorting and Filtering Data
Demonstration: Filtering Data with TOP and OFFSET-FETCH
Question
You have a table named Products in your Sales database. You are creating a paged display of products in an application that shows 20 products on each page, ordered by name. Which of the following queries would return the third page of products?
( )Option 1: SELECT ProductID, ProductName, ProductNumber
FROM Sales.Products
ORDER BY ProductName ASC
OFFSET 60 ROWS FETCH NEXT 20 ROWS ONLY
( )Option 2: SELECT ProductID, ProductName, ProductNumber
OFFSET 40 ROWS FETCH NEXT 20 ROWS ONLY;
( )Option 3: SELECT TOP (20) ProductID, ProductName, ProductNumber
( )Option 4: SELECT TOP (20) WITH TIES ProductID, ProductName, ProductNumber
Answer
Option 1 returns the fourth page of 20 products. Options 3 and 4 return the first page of 20 products, assuming there are no duplicate product names.

13. Filtering in the SELECT Clause Using the TOP Option
5: Sorting and Filtering Data
TOP allows you to limit the number or percentage of rows returned by a query
Works with ORDER BY clause to limit rows by sort order:
If ORDER BY list is not unique, results are not deterministic (no single correct result set)
Modify ORDER BY list to ensure uniqueness, or use TOP WITH TIES
Added to SELECT clause:
SELECT TOP (N) | TOP (N) Percent
With percent, number of rows rounded up (nondeterministic)
SELECT TOP (N) WITH TIES
Retrieve duplicates where applicable (deterministic)
TOP is proprietary to Microsoft SQL Server
Note that, if TOP is used, ORDER BY fills two purposes—first, to support the TOP operator in the SELECT clause, then again to determine the output order of rows for display purposes.
If ORDER BY list is not unique, the results are not deterministic, which means there is not just one correct set of rows. SQL Server will return rows as they are accessed. To address this, either add expressions to the ORDER BY list to ensure uniqueness, or use TOP (N) with TIES.

14. Filtering in the ORDER BY Clause Using OFFSET-FETCH
5: Sorting and Filtering Data
OFFSET-FETCH is an extension to the ORDER BY clause:
Allows filtering a requested range of rows
Dependent on ORDER BY clause
Provides a mechanism for paging through results
Specify number of rows to skip, number of rows to retrieve:
Available in SQL Server 2012, 2014, and 2016
Provides more compatibility than TOP
While TOP may be used without ORDER BY (with unpredictable results), OFFSET/FETCH, without an ORDER BY, will return an error.
Note: The code example is a partial representation of the OFFSET/FETCH syntax only. More complete syntax and examples are presented on the next slide.
ORDER BY <order_by_list>
OFFSET <offset_value> ROW(S)
FETCH FIRST|NEXT <fetch_value> ROW(S) ONLY

15. OFFSET-FETCH Syntax OFFSET value must be supplied
5: Sorting and Filtering Data
OFFSET value must be supplied
May be zero if no skipping is required
The optional FETCH clause allows all rows following the OFFSET value to be returned
Natural Language approach to code:
ROW and ROWS interchangeable
FIRST and NEXT interchangeable
ONLY optional – makes meaning clearer to human reader
OFFSET value and FETCH value may be constants or expressions, including variables and parameters
Note that the interchangeability of ROW for ROWs and FIRST for NEXT allows for English language-like code.
In an application that calls these queries in sequence, each query is independent and not related to the other. No state is maintained on the server (unlike a cursor).
In paging scenarios like the ones mentioned above, it is a best practice to use unique ordering, avoiding the chance of the same row appearing in multiple pages.
Unlike those found in any previous modules, examples of OFFSET/FETCH must be executed by SQL Server 2012, 2014 or OFFSET/FETCH was not supported in SQL Server 2008 R2 or earlier.
OFFSET <offset_value> ROW|ROWS
FETCH FIRST|NEXT <fetch_value> ROW|ROWS [ONLY]

16. Demonstration: Filtering Data with TOP and OFFSET-FETCH
5: Sorting and Filtering Data
In this demonstration, you will see how to:
Filter data using TOP and OFFSET-FETCH
Preparation Steps
Complete the previous demonstration in this module. Alternatively, start the MSL-TMG1, 20761A-MIA-DC, and 20761A-MIA-SQL virtual machines, log on to 20761A-MIA-SQL as ADVENTUREWORKS\Student with the password Pa$$w0rd, and connect to your Azure copy of the AdventureWorksLT database.
Demonstration Steps
Filter data using TOP and OFFSET-FETCH
Ensure that you have completed the previous demonstration in this module. Alternatively, start the A-MIA-DC, and 20761A-MIA-SQL virtual machines, log on to 20761A-MIA-SQL as ADVENTUREWORKS\Student with the password Pa$$w0rd.
If SQL Server Management Studio is not already open, start it and connect to your Azure instance of the AdventureWorksLT database engine instance using SQL Server authentication, and then open the Demo.ssmssln solution in the D:\Demofiles\Mod05\Demo folder.
In Solution Explorer, open the 31 – Demonstration C.sql script file.
In the Available Databases list, ensure ADVENTUREWORKSLT is selected.
Select the code under the comment Step 1, and then click Execute.
Select the code under the comment Step 2, and then click Execute.
Select the code under the comment Step 3, and then click Execute.
Select the code under the comment Step 4, and then click Execute.
Select the code under the comment Step 5, and then click Execute.
Select the code under the comment Step 6, and then click Execute.
Select the code under the comment Step 7, and then click Execute.
Select the code under the comment Step 8, and then click Execute.
Keep SQL Server Management Studio open for the next demonstration.

17. Lesson 4: Working with Unknown Values
5: Sorting and Filtering Data
Demonstration: Working with NULL
Explain the fundamental concept of NULL, and that it is different from blank.
Question
You have the following query:
SELECT e.Name, e.Age
FROM HumanResources.Employees AS e
WHERE YEAR(e.Age) < 1990;
Several employees have asked for their age to be removed from the Human Resources database, and this requested action has been applied to the database. Will the above query return these employees?
Answer
No.

18. SQL Server uses NULLs to mark missing values
Three-Valued Logic
5: Sorting and Filtering Data
SQL Server uses NULLs to mark missing values
NULL can be "missing but applicable" or "missing but inapplicable"
Customer middle name: Not supplied, or doesn’t have one?
With no missing values, predicate outputs are TRUE or FALSE only ( 5 > 2, 1=1)
With missing values, outputs can be TRUE, FALSE or UNKNOWN (NULL > 99, NULL = NULL)
Predicates return UNKNOWN when comparing missing value to another value, including another missing value
Missing but applicable: License plate number of an employee who owns an automobile.
Missing but inapplicable: License plate number of an employee who is a pedestrian.
Celko’s joke: Hair color of a bald man? (He’s bald).
Predicates should be written to handle not only TRUE or FALSE, but also UNKNOWN.

19. Handling NULL in Queries
5: Sorting and Filtering Data
Different components of SQL Server handle NULL differently
Query filters (ON, WHERE, HAVING) filter out UNKNOWNs
CHECK constraints accept UNKNOWNS
ORDER BY, DISTINCT treat NULLs as equals
Testing for NULL
Use IS NULL or IS NOT NULL rather than = NULL or <> NULL
SELECT custid, city, region, country
FROM Sales.Customers
WHERE region IS NOT NULL;

20. Demonstration: Working with NULL
5: Sorting and Filtering Data
In this demonstration, you will see how to:
Test for NULL
Preparation Steps
Complete the previous demonstration in this module. Alternatively, start the MSL-TMG1, 20761A-MIA-DC, and 20761A-MIA-SQL virtual machines, log on to 20761A-MIA-SQL as ADVENTUREWORKS\Student with the password Pa$$w0rd, and connect to your Azure copy of the AdventureWorksLT database.
Demonstration Steps
Test for Null
Ensure that you have completed the previous demonstration in this module. Alternatively, start the A-MIA-DC, and 20761A-MIA-SQL virtual machines, log on to 20761A-MIA-SQL as ADVENTUREWORKS\Student with the password Pa$$w0rd.
If SQL Server Management Studio is not already open, start it and connect to your Azure instance of the AdventureWorksLT database engine instance using SQL Server authentication, and then open the Demo.ssmssln solution in the D:\Demofiles\Mod05\Demo folder.
In Solution Explorer, open the 41 – Demonstration D.sql script file.
In the Available Databases list, ensure ADVENTUREWORKSLT is selected.
Select the code under the comment Step 2, and then click Execute.
Select the code under the comment Step 3, and then click Execute.
Select the code under the comment Step 4, and then click Execute.
Select the code under the comment Step 5, and then click Execute.
Select the code under the comment Step 6, and then click Execute.
Select the code under the comment Step 7, and then click Execute.
Close SQL Server Management Studio.

21. Lab: Sorting and Filtering Data
Exercise 4: Write Queries that Filter Data Using the OFFSET-FETCH Clause
Remind students that, without an ORDER BY, the lab exercises may return results in a different order than the supplied lab answers. If they want to check results, they can add an ORDER BY clause, both to their solution and the provided solution. This will affect all labs!
Exercise 1: Write Queries that Filter Data Using a WHERE Clause
The marketing department is working on several campaigns for existing customers and staff need to obtain different lists of customers, depending on several business rules. Based on these rules, you will write the SELECT statements to retrieve the needed rows from the Sales.Customers table.
Exercise 2: Write Queries that Sort Data Using an ORDER BY Clause
The sales department would like a report showing all the orders with some customer information. An additional request is that the result be sorted by the order dates and the customer IDs. From previous modules, remember that the order of the rows in the output of a query without an ORDER BY clause is not guaranteed. Because of this, you will have to write a SELECT statement that uses an ORDER BY clause.
Exercise 3: Write Queries that Filter Data Using the TOP Option
The sales department wants to have some additional reports that show the last invoiced orders and the top 10 percent of the most expensive products being sold.
Exercise 4: Write Queries that Filter Data Using the OFFSET-FETCH Clause
In this exercise, you will implement a paging solution for displaying rows from the Sales.Orders table because the total number of rows is high. In each page of a report, the user should only see 20 rows.
Logon Information
Virtual machine: 20761A-MIA-SQL
User name: ADVENTUREWORKS\Student
Password: Pa$$w0rd
Estimated Time: 60 minutes

22. 20761A
Lab Scenario
5: Sorting and Filtering Data
You are an Adventure Works business analyst who will be writing reports using corporate databases stored in SQL Server. You have been provided with a set of data business requirements and will write T-SQL queries to retrieve the specified data from the databases. You will need to retrieve only some of the available data, and return it to your reports in a specified order.

23. 20761A
Lab Review
5: Sorting and Filtering Data
What is the difference between filtering using the TOP option, and filtering using the WHERE clause?
Question
What is the difference between filtering using the TOP option, and filtering using the WHERE clause?
Answer
The TOP option can only be used to filter results based on the columns specified in the ORDER BY clause; and then it can only be used to return a count of rows (or a percentage of rows) from the top of that result set. There is no support for more complex filters that cannot be expressed in terms of sorting. TOP has no facility for filtering NULL values—NULL values in the columns included in the ORDER BY clause are always returned first.
Filters in the WHERE clause have no such limitations; you can specify complex filters and filters that handle NULL.

24. Module Review and Takeaways
5: Sorting and Filtering Data
Review Question(s)
Review Question(s)
Question
Does the physical order of rows in an SQL Server table guarantee any sort order in queries using the table?
Answer
No.
You have the following query:
SELECT p.PartNumber, p.ProductName, o.Quantity
FROM Sales.Products AS p
LEFT OUTER JOIN Sales.OrderItems AS o
ON p.ID = o.ProductID
ORDER BY o.Quantity ASC
You have one new product that has yet to receive any orders. Will this product appear at the top or the bottom of the results?
At the top of the results.