SQL: Structured Query Language Part -2 Instructor: Mohamed Eltabakh meltabakh@cs.wpi.edu

Aggregation + GroupBy

Restrictions of GROUP BY If you group by A1, A2, …An, then any other column projected in SELECT clause must be inside an aggregation function SELECT pNumber, address, count(sName) AS CNT, sum(sNumber) AS SUM FROM Student WHERE sNumber > 1 GROUP BY pNumber, address; SELECT pNumber, address, sName, sum(sNumber) AS SUM FROM Student WHERE sNumber > 1 GROUP BY pNumber, address; X SELECT pNumber, count(sName) AS CNT, sum(sNumber) AS SUM FROM Student WHERE sNumber > 1 GROUP BY pNumber, address;

HAVING Clause: Putting Condition on Groups How to add conditions on each group? Select only the groups where the COUNT > 5 These conditions are after you build the groups (not before) Remember: WHERE conditions are executed before the groups are formed New optional clause called “HAVING”, added after the GROUP BY clause SELECT pNumber, COUNT (sName) FROM Student GROUP BY pNumber HAVING SUM(sNumber) > 2; Can reference aggregation inside HAVING

HAVING Clause: Example Student sNumber sName address pNumber 1 Dave 320FL 2 Greg 3 Matt 4 Jan 500MA  (SUM> 3) (pNumber,address, CNT  count(sName), SUM  sum(sNumber) ( (sNumber > 1) (Student))) SELECT pNumber,address, count(sName) AS CNT, sum(sNumber) AS SUM FROM Student WHERE sNumber > 1 GROUP BY pNumber, address HAVING sum(sNumber) > 3; Applied before the grouping Applied after the grouping on each group pNumber address CNT SUM 2 500MA 1 4

Example Queries Find customers having loans with sum > 20,000. Report the customer names and the total loans amount SELECT customer_name, sum(amount) AS SUM FROM Loan L, Borrower B WHERE L.loan_number = B. loan_number GROUP BY customer_name HAVING sum(amount) > 20,000;

Example Queries Find the cities with more than 3 branches SELECT branch_city FROM Branch GROUP BY branch_city HAVING count(branch_name) > 3;

Example Queries Find the customer names who have accounts with balance > 100,000 in more than two branches. Report the customer names & number of branches SELECT customer_name, count(branch_name) As CNT FROM Account A, Depositor D WHERE A.account_number = D.account_number AND A.balance > 100,000 GROUP BY customer_name HAVING count(branch_name) > 2;

Example Queries Report the largest and smallest loans given by each branch in NY city along with the branch name. SELECT branch_name, Max(amount) As MaxLoan, Min(amount) As MinLoan FROM Loan L, Branch B Where L.branch_name = B.branch_name AND branch_city = ‘NY’ GROUP BY branch_name;

SELECT Statement Clauses SELECT <projection list> FROM <relation names> WHERE <conditions> GROUP BY <grouping columns> HAVING <grouping conditions> ORDER BY <order columns>; optional Optional clauses if added must be in the order above Order of execution FROM  Check which relations are used WHERE  Filter records based on conditions GROUP BY  Form groups HAVING  Filter groups based on conditions ORDER BY  Sort the data SELECT  Form the projection list (output columns)

Example: Order of Execution Student sNumber sName address pNumber 1 Dave 320FL 2 Greg 3 Matt 4 Jan 500MA 5 SELECT pNumber,address, count(sName) AS CNT, sum(sNumber) AS SUM FROM Student WHERE sNumber > 1 GROUP BY pNumber, address HAVING sum(sNumber) > 3; 1 2 3 4 pNumber address CNT SUM 2 500MA 1 4

Questions optional SELECT <projection list> FROM <relation names> WHERE <conditions> GROUP BY <grouping columns> HAVING <grouping conditions> ORDER BY <order columns>; optional

Example Queries Report the branch-cities having more than 5 branches, and order them by the number branches Report the output relation O(CustomerName, Num_Loans, Num_Accounts) only for customers who have loans or accounts

Example Queries Report the branch-cities having more than 5 branches, and order them by the number branches SELECT branch_city, count(branch_name) FROM Branch GROUP BY branch_city Having count(branch_name) > 5 Order By count(branch_name);

Example Queries Report the output relation O(CustomerName, Num_Loans, Num_Accounts) only for customers who have loans or accounts SELECT customer_name, count(loan_number), count(account_number) FROM Depositor D, Borrower B WHERE D.customer_name = B.customer_name GROUP BY customer_name; X Join will cause values to replicate

Fixing Previous Example The join keeps only customers Who appear in both sides Report the output relation O(CustomerName, Num_Loans, Num_Accounts) only for customers who have loans and accounts SELECT customer_name, count(Distinct loan_number), count(Distinct account_number) FROM Depositor D, Borrower B WHERE D.customer_name = B.customer_name GROUP BY customer_name;  Applies the aggregation over the distinct values  Can be used with others, e.g., Min, Max, Avg, …

Fixing Previous Example In this case, we need Outer Join Report the output relation O(CustomerName, Num_Loans, Num_Accounts) only for customers who have loans or accounts SELECT customer_name, count(Distinct loan_number), count(Distinct account_number) FROM Depositor D Full Outer Join Borrower B On D.customer_name = B.customer_name GROUP BY customer_name;  Notice the syntax is slightly different

More in SELECT Statement Special handling for NULL values Queries inside Insert/Update/Delete Nested subqueries

Null Values Null means ‘unknown’ value Any expression containing Null returns Null 5 + null  null ‘ABC’ || null  null Null in predicates returns UNKNOWN Predicates usually return TRUE or FALSE

Having Null in the data is problematic and needs special care… Example Student sNumber sName address pNumber 1 Dave 320FL 2 Greg null 3 Matt 4 Jan 500MA sNumber 1 2 3 SELECT sNumber FROM Student WHERE address = ‘320FL’; May or may not appear Having Null in the data is problematic and needs special care…

Use of “IS NULL” or “IS NOT NULL” Check if a value is null or not SELECT sNumber FROM Student WHERE address is null; Select student numbers where the address is null SELECT sNumber FROM Student WHERE address is not null AND address = ‘320FL’; Remember: SELECT sNumber FROM Student WHERE address = null; X The returned value here is unknown

Use of “NVL” Function NVL( exp1, exp2) If exp1 is null return exp2, otherwise return expr1 Can be used in projection list or in predicates SELECT sNumber FROM Student WHERE nvl(address, ‘n/a’) <> ‘n/a’ AND address ‘320FL’; SELECT sNumber, nvl(address, ‘N/A’) FROM Student; sNumber address 1 320FL 2 N/A 3 4 500MA

Null with Grouping & Aggregation Null is ignored with all aggregates, e.g., SUM, AVG, MIN, MAX except COUNT() Grouping Null is considered as a separate group

Example Student SELECT address, sum(pNumber) as sum, count(*) as cnt sNumber sName address pNumber 1 Dave 320FL 2 Greg null 3 Matt 4 Jan 500MA SELECT address, sum(pNumber) as sum, count(*) as cnt FROM Student GROUP BY address; address sum cnt 320FL 1 null 2 500MA

More in SELECT Statement Special handling for NULL values Queries inside Insert/Update/Delete Nested subqueries

Reminder About: Insert, Update, Delete This is performed using Data Manipulation Language of SQL (DML) Insertion Insert into Students values (‘1111’, …); Deletion Delete from Students; Delete from Students Where sid = ‘1111’; Update Update Students Set GPA = GPA + 0.4; Update Students Set GPA = GPA + 0.4 Where sid = ‘1111’;

Use of Select Inside Insert All records returned from the Select will be inserted Notice that there is no keyword “values” in this case INSERT INTO suppliers (supplier_id, supplier_name) SELECT account_no, name FROM externals Where code = 1; Number of columns and data types should match

Use of Select Inside Delete Student Registration sNumber sName address 1 Dave 320FL 2 Greg null 3 Matt 4 Jan 500MA sNumber courseID grade 4 DB1 A 2 F 3 DB2 Delete from Student all those who have grade ‘F’ Delete From Student Where sNumber in (Select sNumber from Registration Where grade = ‘F’); Students number 2 & 4 will be deleted…

Use of Select Inside Delete Student Registration sNumber sName address 1 Dave 320FL 2 Greg null 3 Matt 4 Jan 500MA sNumber courseID grade 4 DB1 A 2 F 3 DB2 Delete from Student all those who do not have registration Delete From Student Where sNumber not in (Select sNumber from Registration); Student number 1 will be deleted…

Use of Select Inside Update Student Registration sNumber sName address 1 Dave 320FL 2 Greg null 3 Matt 4 Jan 500MA sNumber courseID grade 4 DB1 A 2 F 3 DB2 Update the grades of student ‘Matt’ to B Update Registration Set grade = ‘B’ Where sNumber in (Select sNumber from Student Where sName =‘Matt’);

Remember… In Delete and Update From Student Where sNumber not in (Select sNumber from Registration); Update Registration Set grade = ‘B’ Where sNumber in (Select sNumber from Student Where sName =‘Matt’); In Delete and Update - The ‘From’ clause always has one table - The subquery can be only added in the ‘Where’ clause

More in SELECT Statement Special handling for NULL values Queries inside Insert/Update/Delete Nested subqueries

Nested Subquery SQL provides a mechanism for the nesting of subqueries. A subquery is a SELECT statement expression that is nested within another query Subquery can appear in FROM or WHERE clauses

Nested Subquery in FROM Clause SELECT * FROM Student, (inner SELECT) AS q WHERE … Table built on the fly Use the inner SELECT like any other table It is just built on the fly Inner SELECT can be a full statement with all clauses ORDER BY clause does not make sense in the inner select

Example Subquery 1 is computed on the fly It is treated as a normal table after that

Back to this Example Find another way ?? Report the output relation O(CustomerName, Num_Loans, Num_Accounts) only for customers who have loans and accounts SELECT customer_name, count(Distinct loan_number), count(Distinct account_number) FROM Depositor D, Borrower B WHERE D.customer_name = B.customer_name GROUP BY customer_name;

Back to this Example Report the output relation O(CustomerName, Num_Loans, Num_Accounts) only for customers who have loans and accounts SELECT D.customer_name, Num_Loan, Num_Acc FROM ( Select customer_name, count(*) As Num_Acc From Depositor Group By customer_name) As D, ( Select customer_name, count(*) As Num_Loan From Borrower Group By customer_name) As B WHERE D.customer_name = B.customer_name;

Back to this Example Find another way ?? Report the output relation O(CustomerName, Num_Loans, Num_Accounts) only for customers who have loans Or accounts SELECT customer_name, count(Distinct loan_number), count(Distinct account_number) FROM Depositor D, Borrower B WHERE D.customer_name = B.customer_name GROUP BY customer_name;

Back to this Example Report the output relation O(CustomerName, Num_Loans, Num_Accounts) only for customers who have loans Or accounts SELECT D.customer_name, Num_Loan, Num_Acc FROM ( Select customer_name, count(*) As Num_Acc From Depositor Group By customer_name) As D Full Outer Join ( Select customer_name, count(*) As Num_Loan From Borrower Group By customer_name) As B On D.customer_name = B.customer_name;

Nested Subquery in WHERE Clause 2- Then, execute this statement once pNumber from the first step is known (outer SELECT) SELECT * FROM Student WHERE pNumber = (SELECT pNumber FROM Professor WHERE pName = ‘Mike’); 1- Execute this statement first to get the pNumber (inner SELECT) Since the predicates has = : The inner statement must return one record with one column In this case, DBMS will automatically convert the relation to a single scalar value Otherwise an error is generated

Example: Subqueries Retuning Scalar Value Student Professor sNumber sName address pNum 1 Dave 320FL 2 Greg 3 Matt pNumber pName address 1 MM 141FL 2 ER 201FL Select students of professor ‘MM’ SELECT sNumber, sName FROM Student WHERE pNum = (SELECT pNumber FROM Professor WHERE pName=‘MM’); sNumber sName 1 Dave 2 Greg CS3431

SubQuery Returning a Relation (General Case) Outer Select (S) SELECT sNumber, sName FROM Student WHERE pNum OP (SELECT pNumber FROM Professor WHERE pName=‘MM’); Inner Select (R) Predicates may include any of (OP above) : s in R  True if tuple s appears in R s not in R  True if tuple s does not appear in R s = R  R must return a single value (otherwise invalid op) Exists R  True if R is not empty Not Exists R  True if R is empty

Example 1: Subqueries Returning Relations Student Professor sNumber sName address pNum 1 Dave 320FL 2 Greg 3 Matt 4 Sam 30IN pNumber pName address 1 MM 141FL 2 ER 201FL 3 XY 30WA Select students of professors with address like ‘%FL’ SELECT sNumber, sName FROM Student WHERE pNum IN (SELECT pNumber FROM Professor WHERE address Like ‘%FL’); sNumber sName 1 Dave 2 Greg 3 Matt CS3431

Example 2: Subqueries Returning Relations Student Professor sNumber sName address pNum 1 Dave 320FL 2 Greg 3 Matt 4 Sam 30IN pNumber pName address 1 MM 141FL 2 ER 201FL 3 XY 30WA SELECT sNumber, sName FROM Student WHERE Exists (SELECT pNumber FROM Professor WHERE address Like ‘%FL’); sNumber sName 1 Dave 2 Greg 3 Matt 4 Sam Always true because it is not empty CS3431

Example 3: Subqueries Registration  R Step 1 Q: Find the student ID taking the largest number of courses Registration  R sID courseID … 1 CS101 2 CS300 CS202 4 CS500 CS203 CS303 SELECT sID, count(courseID) as CNT FROM R Group By sID; Step 1 sID CNT 1 3 2 4 CS3431

Example 3: Subqueries Registration  R Step 2 Q: Find the student ID taking the largest number of courses Registration  R sID CNT 1 3 sID courseID … 1 CS101 2 CS300 CS202 4 CS500 CS203 CS303 Step 2 SELECT sID, count(courseID) as CNT FROM R Group By sID Having count(courseID) = ( SELECT Max(CNT) As Max From (SELECT count(courseID) As CNT Group By sID) q ) Max 3 sID CNT 1 3 2 4 CNT 3 1 2 CS3431

Comparison Using ALL and ANY Outer Select (S) SELECT sNumber, sName FROM Student WHERE pNum OP (SELECT pNumber FROM Professor WHERE pName=‘MM’); Inner Select (R) We took: Exists, IN, NOT IN s > ALL R  True if s > all values in R s > ANY R  True if s > any value in R ‘>’ can be any of the other comparison operators, e.g., <, <=, >=, =, <> R must be relation with single column

Example Student Professor SELECT sNumber, sName FROM Student address pNum 1 Dave 320FL 2 Greg 3 Matt 4 Sam 30IN pNumber pName address 1 MM 141FL 2 ER 201FL 3 XY 30WA SELECT sNumber, sName FROM Student WHERE pNum >= ALL (SELECT pNumber FROM Professor WHERE address Like ‘%FL’); sNumber sName 3 Matt 4 Sam This inner select returns 1 , 2