1. COMP 430 Intro. to Database Systems Grouping & Aggregation Slides use ideas from Chris Ré and Chris Jermaine. Get clickers today!

2. One form of aggregation – Column totals Candy candy_nameprice Reese’s Cup0.50 5 th Avenue1.20 Almond Joy0.75 Jelly Babies2.39 SELECT Sum(price) FROM Candy; 4.84

3. Other aggregations Count Avg Max Min … What else depends on SQL version.

4. Aggregation + DISTINCT Candy candy_nameprice Reese’s Cup0.50 5 th Avenue1.20 Almond Joy0.75 Jelly Babies2.39 Chocolate Orange 2.39 Jelly Nougats0.50 SELECT Count(price), Count (DISTINCT price) FROM Candy; 64

5. Aggregation & NULL Candy candy_nameprice Reese’s Cup0.50 5 th AvenueNULL Almond Joy0.75 Jelly Babies2.39 Chocolate Orange 2.39 Jelly NougatsNULL SELECT Count(price) FROM Candy; 4 No price since no longer made in original form.

6. Counting rows Candy candy_nameprice Reese’s Cup0.50 5 th AvenueNULL Almond Joy0.75 Jelly Babies2.39 Chocolate Orange 2.39 Jelly NougatsNULL SELECT Count(*) FROM Candy; 6

7. Aggregation + other previous features Purchase productdatepricequantity bagel10/21120 banana10/030.5010 banana10/10110 bagel10/251.5020 SELECT Sum(price * quantity) FROM Purchase WHERE product = ‘bagel’; 50

8. Grouping + Aggregation – Column subtotals Purchase productdatepricequantity bagel10/21120 banana10/030.5010 banana10/10110 bagel10/251.5020 SELECT product, Sum(price * quantity) AS subtotal FROM Purchase GROUP BY product; productdatepricequantity bagel 10/21120 10/251.5020 banana 10/030.5010 10/10110 productsubtotal bagel50 banana15

9. Grouping + Aggregation Purchase productdatepricequantity bagel10/21120 banana10/030.5010 banana10/10110 bagel10/251.5020 SELECT product, Count(*) AS purchases, Sum(quantity) AS items, Sum(price * quantity) AS subtotal FROM Purchase GROUP BY product; Can only SELECT fields that you GROUP BY. productpurchasesitemssubtotal bagel24050 banana22015

10. Conditions on aggregates SELECT product, Sum(price * quantity) AS subtotal FROM Purchase WHERE date > ‘10/05’ GROUP BY product HAVING SUM(quantity) >= 10; Condition on individual rows. Condition on aggregates. Purchase productdatepricequantity bagel10/21120 banana10/030.5010 banana10/10110 bagel10/251.5020 apple10/1015 productdatepricequantity bagel 10/21120 10/251.5020 banana10/10110 apple10/1015 productsubtotal bagel50 banana10

11. Query: Products with at least two purchases. A.Count(product) B.Count(*) C.Sum(product) D.product SELECT product FROM Purchase GROUP BY product HAVING ??? >= 2 ; Purchase productdatepricequantity bagel10/21120 banana10/030.5010 banana10/10110 bagel10/251.5020 apple10/1015

12. Sorting on aggregates Purchase productdatepricequantity bagel10/21120 banana10/030.5010 banana10/10110 bagel10/251.5020 SELECT product, Sum(price * quantity) FROM Purchase GROUP BY product ORDER BY Sum(price * quantity); productdatepricequantity bagel 10/21120 10/251.5020 banana 10/030.5010 10/10110 productsubtotal banana15 bagel50

13. Sorting on aggregates Purchase productdatepricequantity bagel10/21120 banana10/030.5010 banana10/10110 bagel10/251.5020 SELECT product, Sum(price * quantity) AS subtotal FROM Purchase GROUP BY product ORDER BY subtotal; productdatepricequantity bagel 10/21120 10/251.5020 banana 10/030.5010 10/10110 productsubtotal banana15 bagel50

14. Syntax summary SELECT [DISTINCT] Computations FROM Tables WHERE Condition 1 GROUP BY a 1, …, a k HAVING Condition 2 ORDER BY attributes LIMIT n; Uses attributes a 1, …, a k or aggregates. Condition on attributes. Condition on aggregates. Some SQLs: SELECT TOP(n) Computations Can also use computation results.

15. Semantics summary SELECT [DISTINCT] Computations FROM Tables WHERE Condition 1 GROUP BY a 1, …, a k HAVING Condition 2 ORDER BY attributes LIMIT n; 5.Computations, [eliminate duplicates] 8.Projections 1.Cross-product 2.Apply Condition 1 3.Group by attributes 4.Apply Condition 2 to each group 6.Sort 7.Use first n rows What we’re they thinking?!?

16. (Potentially confusing) details

17. DISTINCT Sum() vs. Sum(DISTINCT) SELECT DISTINCT Sum(DISTINCT value) FROM Data GROUP BY tag; Data itemtagvalue 1a2 2a2 3b1 4b3 5c4 ?

18. HAVING without grouping SELECT value, COUNT(*) AS count FROM Data HAVING count > 1; Data itemtagvalue 1a2 2a2 3b1 4b3 5c4 ? Fails in SQLite. Without GROUP BY, the entire results form one group.

19. Grouping without aggregation Purchase productdatepricequantity bagel10/21120 banana10/030.510 banana10/10110 bagel10/251.5020 SELECT product FROM Purchase GROUP BY product; product bagel banana

20. GROUP BY vs. DISTINCT SELECT product FROM Purchase GROUP BY product; SELECT DISTINCT product FROM Purchase; But only GROUP BY works well with aggregation. Purchase productdatepricequantity bagel10/21120 banana10/030.510 banana10/10110 bagel10/251.5020 product bagel banana

21. More complicated examples

22. Activity – will break activity into segments 04a-aggregation.ipynb Students (id and name) taking more than 5 courses Courses and their average ratings Highest average rating of a course Course with highest average rating

23. Students with >5 courses – Two solutions SELECT s_id, first_name, last_name FROM Student WHERE (SELECT Count(*) FROM Enrollment WHERE Student.s_id = Enrollment.s_id ) > 5; SELECT s.s_id, s.first_name, s.last_name FROM Student s, Enrollment e WHERE s.s_id = e.s_id GROUP BY Student.s_id HAVING COUNT(crn) > 5;

24. Efficiency comparison SELECT s_id, first_name, last_name FROM Student WHERE (SELECT Count(*) FROM Enrollment WHERE Student.s_id = Enrollment.s_id ) > 5; SELECT s.s_id, s.first_name, s.last_name FROM Student s, Enrollment e WHERE s.s_id = e.s_id GROUP BY Student.s_id HAVING COUNT(crn) > 5; Assuming no major optimizations: 1.How many times do we SELECT … FROM Enrollment? 2.How many joins?

25. Courses and their average ratings SELECT crn, Avg(rating) FROM Enrollment GROUP BY crn;

26. Highest average rating of a course SELECT Max(avg_rating) FROM (SELECT Avg(rating) as avg_rating FROM Enrollment GROUP BY crn ); SELECT crn, Max(Avg(rating)) FROM Enrollment GROUP BY crn; Nesting aggregate functions is not syntactically allowed.

27. Highest avg rating – Solution 1 SELECT crn FROM (SELECT crn, Avg(rating) AS avg_rating1 FROM Enrollment GROUP BY crn ) WHERE avg_rating1 = (SELECT Max(avg_rating2) FROM (SELECT Avg(rating) as avg_rating2 FROM Enrollment GROUP BY crn ) ); 1.Calculate all average ratings. 2.Calculate the maximum. 3.Find which courses have this value as their average.

28. What about repeated subquery? SELECT crn FROM (SELECT crn, Avg(rating) AS avg_rating1 FROM Enrollment GROUP BY crn ) WHERE avg_rating1 = (SELECT Max(avg_rating2) FROM (SELECT Avg(rating) AS avg_rating2 FROM Enrollment GROUP BY crn ) ); CREATE VIEW Average AS SELECT crn, Avg(rating) AS avg_rating FROM Enrollment GROUP BY crn; SELECT crn FROM Average WHERE avg_rating = (SELECT Max(avg_rating) FROM Average );

29. Highest avg rating – Solution 2 CREATE VIEW Average AS SELECT crn, Avg(rating) AS avg_rating FROM Enrollment GROUP BY crn; SELECT crn FROM Average ORDER BY avg_rating DESC LIMIT 1; Doesn’t work for ties.

30. Efficiency comparison CREATE VIEW Averages AS SELECT crn, Avg(rating) AS avg_rating FROM Enrollment GROUP BY crn; SELECT crn FROM Averages ORDER BY avg_rating LIMIT 1; SELECT crn FROM Averages WHERE avg_rating = (SELECT Max(avg_rating) FROM Averages ); Assuming no major optimizations: Cost estimate of each?

31. Aside: Code Style & Indentation Review previous examples.

32. Where we’re headed So far: Have seen a powerful subset of SQL queries. Raised questions about how example schemas are structured. Next: Switch to focusing on database design. Later: Return to more SQL.