1. 1 SQL Structured Query Language

2. Query Language SQL is a query language Used to examine data in the database SQL queries do not change the contents of the database (no side-effects!) The result of an SQL query is printed to the screen, not stored in the database 2

3. 3 Basic SQL query structure SELECT Attributes FROM relations WHERE condition SELECT sid,sname FROM students WHERE sid=1122 For example:

4. 4 Query Components A query can contain the following clauses –select –from –where –group by –having –order by Only select and from are obligatory Order of clauses is always as above

5. 5 Very Basic SQL Query SELECT [Distinct] Attributes FROM relation Attributes: The attributes or values which will appear in the query result (For example: id, name). DISTINCT: Optional keyword to delete duplicates Relation: Relation to perform the query on. Example: Select studentID, studentName From students

6. 6 StudentIDStudentDept.StudentNameStudentAge 1123MathMoshe25 2245ComputersMickey26 55611MathMenahem29 Select studentID, studentName From students Result: StudentIDStudentName 1123Moshe 2245Mickey 55611Menahem

7. 7 Basic SQL Query SELECT [Distinct] Attributes FROM relation WHERE condition condition: A Boolean condition (For example: Eid>21, or Ename=‘Yuval’ ). Only tuples which return ‘true’ for this condition will appear in the result

8. 8 StudentIDStudentDept.StudentNameStudentAge 1123MathMoshe25 2245ComputersMickey26 55611MathMenahem29 Select studentID, studentName From students Where StudentDept=‘Math’ Result: StudentIDStudentName 1123Moshe 55611Menahem

9. 9 SQL and relational algebra SELECT Distinct A 1,…,A n FROM R 1,…,R m WHERE C;  A 1,…,A n (  C (R 1 x…x R m ))

10. 10 Basic SQL Query SELECT [Distinct] attributes FROM relations WHERE condition; Important! The evaluation order, conceptually, is: 1.Compute the cross product of the tables in relations. 2.Delete all rows that do not satisfy condition. 3.Delete all columns that do not appear in attributes. 4.If Distinct is specified eliminate duplicate rows.

11. 11 Example Tables Used Reserves sidbidday 22 58 101 103 10/10/96 11/12/96 Sailors sidsnameratingage 22 31 58 Dustin Lubber Rusty 7 8 10 45.0 55.5 35.0 Boats bidbnamecolor 101 103 Nancy Gloria red green

12. 12 What does this compute? Select sname from sailors, reserves Where sailors.sid=reserves.sid Reserves sidbidday 22 58 101 103 10/10/96 11/12/96 Sailors sidsnameratingage 22 31 58 Dustin Lubber Rusty 7 8 10 45.0 55.5 35.0 All sailors who have reserved a boat

13. 13 SailorsReserves sidsnameratingagesidbidday 22Dustin745.02210110/10/96 22Dustin745.05810311/12/96 31Lubber855.52210110/10/96 31Lubber855.55810311/12/96 58Rusty1035.02210110/10/96 58Rusty1035.05810311/12/96 Stage 1: Sailors x Reserves

14. 14 SailorsReserves sidsnameratingagesidbidday 22Dustin745.02210110/10/96 22Dustin745.05810311/12/96 31Lubber855.52210110/10/96 31Lubber855.55810311/12/96 58Rusty1035.02210110/10/96 58Rusty1035.05810311/12/96 Stage 2: “where sailors.sid=reserves.sid”

15. 15 SailorsReserves sidsnameratingagesidbidday 22Dustin745.02210110/10/96 58Rusty1035.05810311/12/96 Stage 2: “where sailors.sid=reserves.sid”

16. 16 SailorsReserves sidsnameratingagesidbidday 22Dustin745.02210110/10/96 58Rusty1035.05810311/12/96 Stage 3: “select sname”

17. 17 sname Dustin Rusty Stage 3: “select sname” Final answer

18. 18 Example Query SELECT sname, age FROM Sailors WHERE rating>7; Q: What does this compute?

19. 19 Example Query SELECT DISTINCT sname FROM Sailors, Reserves WHERE Sailors.sid = Reserves.sid and bid = 103; Q: What does this compute?

20. 20 SailorsReserves sidsnameratingagesidbidday 22Dustin745.02210110/10/96 22Dustin745.05810311/12/96 31Lubber855.52210110/10/96 31Lubber855.55810311/12/96 58Rusty1035.02210110/10/96 58Rusty1035.05810311/12/96 WHERE Sailors.sid = Reserves.sid and bid = 103;

21. 21 SailorsReserves sidsnameratingagesidbidday 22Dustin745.02210110/10/96 22Dustin745.05810311/12/96 31Lubber855.52210110/10/96 31Lubber855.55810311/12/96 58Rusty1035.02210110/10/96 58Rusty1035.05810311/12/96 Select sname

22. 22 A Few SELECT Options Select all columns: SELECT * FROM Sailors; Rename selected columns: SELECT S.sname AS Sailors_Name FROM Sailors S; Applying functions (e.g., Mathematical manipulations) SELECT (age-5)*2 FROM Sailors S;

23. 23 The WHERE Clause Numerical and string comparison: !=,<>,=,, >=, <=, between(val1 AND val2) Logical components: AND, OR Null verification: IS NULL, IS NOT NULL Checking against a list with IN, NOT IN.

24. 24 Examples SELECT sname FROM Sailors WHERE age>=40 AND rating IS NOT NULL ; SELECT sid, sname FROM sailors WHERE sid IN (1223, 2334, 3344) or sname between(‘George’ and ‘Paul’);

25. 25 The LIKE Operator A pattern matching operator (regular expression) Basic format: colname LIKE pattern –Example: _ is a single character % is 0 or more characters SELECT sid FROM Sailors WHERE sname LIKE ‘R_%y’;

26. 26 Relation naming SELECT S.sname FROM Sailors S, Reserves R WHERE S.sid = R.sid and R.bid = 103; Naming relations is good style It is necessary if the same relation appears twice in the FROM clause

27. 27 Example Query SELECT S.sname FROM Sailors S, Reserves R WHERE S.sid = R.sid and R.bid != 103; Q: Does this return the names of sailors who did not reserve boat 103? A: No! it returns the names of sailors who reserved a boat other than boat 103 Explanation in the next slides

28. 28 Reserves sidbidday 22 31 101 103 104 10/10/07 11/12/07 12/2/07 Sailors sidsnameratingage 22 31 Dustin Lubber 7878 45.0 55.5 SELECT S.sname FROM Sailors S, Reserves R WHERE S.sid = R.sid and R.bid != 103;

29. 29 SailorsReserves sidsnameratingagesidbidday 22Dustin745.02210110/10/07 22Dustin745.02210311/12/07 22Dustin745.03110412/2/07 31Lubber855.52210110/10/07 31Lubber855.52210311/12/07 31Lubber855.53110412/2/07

30. 30 SailorsReserves sidsnameratingagesidbidday 22Dustin745.02210110/10/07 22Dustin745.02210311/12/07 22Dustin745.03110412/2/07 31Lubber855.52210110/10/07 31Lubber855.52210311/12/07 31Lubber855.53110412/2/07

31. 31 SailorsReserves sidsnameratingagesidbidday 22Dustin745.02210110/10/07 31Lubber855.53110412/2/07 sname Dustin Lubber But Dustin did order boat 103!

32. 32 SQL query SELECT S.sid FROM Sailors S, Reserves R WHERE S.sid = R.sid; When would adding DISTINCT give a different result? When there is a sailor who reserved more than a single boat

33. 33 Are any of these the same? SELECT S.sid FROM Sailors S, Reserves R WHERE S.sid = R.sid; SELECT DISTINCT R.sid FROM Sailors S, Reserves R WHERE S.sid = R.sid; SELECT R.sid FROM Reserves R Sailors sidsnameratingage Reserves sidbidday

34. 34 Example Query SELECT S.sname FROM Sailors S, Reserves R1, Reserves R2 WHERE S.sid = R1.sid and R1.sid=R2.sid and R1.bid!=R2.bid; How would you find sailors who have reserved more than one boat?

35. 35 SQL query SELECT S.sname FROM Sailors S, Reserves R, Boats B WHERE S.sid = R.sid and R.bid = B.bid and B.color = 'red' Q: What does this return?

36. 36 SQL query SELECT distinct B.color FROM Sailors S, Reserves R, Boats B WHERE S.sname = ‘Bob’ and S.sid = R.sid and R.bid = B.bid Q: How would you find the colors of boats reserved by Bob? A:

37. 37 Order Of the Result The ORDER BY clause can be used to sort results by one or more columns The default sorting, when ORDER BY is used, is in ascending order Can specify ASC or DESC SELECT sname, rating, age FROM Sailors S WHERE age > 50 ORDER BY rating ASC, age DESC

38. 38 What does this return? SELECT DISTINCT S.sname FROM Sailors S, Reserves R, Boats B WHERE S.sid = R.sid and R.bid = B.bid and (B.color = 'red' or B.color='green') What would happen if we replaced or by and ? We would get no results! Then how can we find sailors who have reserved both a green and a red boat?

39. 39 Sailors who’ve reserved red and green boats SELECT S.sname FROM Sailors S, Reserves R1, Reserves R2 Boats B1, Boats B2 WHERE S.sid = R1.sid and R1.bid = B1.bid and B1.color = ‘red’ and S.sid = R2.sid and R2.bid = B2.bid and B2.color = ‘green’;

40. 40 Other Relational Algebra Operators So far, we have seen selection, projection and Cartesian product How do we do operators UNION and MINUS?

41. 41 Three SET Operators [Query] UNION [Query] [Query] EXCEPT [Query] [Query] INTERSECT [QUERY] Note: The operators remove duplicates by default!

42. 42 Sailors who’ve reserved red or green boat SELECT S.sname FROM Sailors S, Boats B, Reserves R WHERE S.sid = R.sid and R.bid = B.bid and B.color = ‘red’ UNION SELECT S.sname FROM Sailors S, Boats B, Reserves R WHERE S.sid = R.sid and R.bid = B.bid and B.color = ‘green’; Would INTERSECT give us sailors who reserved both red and green boats? Almost, but not quite because sname is not unique…

43. 43 Multiset (Bag) Operators Union without removing duplicates: UNION ALL SELECT DISTINCT sname FROM Sailors S UNION ALL SELECT DISTINCT sname FROM Sailors S

44. 44 Nested Queries

45. A query is nested if one of its clauses contains a query Queries can be nested in the following clauses: –Select –From –Where –Having 45

46. Nested Queries (cont) A sub-query of a nested query is correlated if it refers to relations appearing in the outer portion of the query We start by discussing subqueries in the WHERE clause –Common operators used to correlate are: IN, ANY/ALL, EXISTS 46

47. Remember! The WHERE clause is evaluated for each tuple in the Cartesian Product formed by the FROM clause, and a Boolean answer is returned –The subquery is used to define the Boolean answer! 47

48. 48 Nested queries in WHERE SELECT S.sname FROM Sailors S WHERE S.sid IN (SELECT R.sid FROM Reserves R WHERE R.bid = 103); Subqueries with multiple results: What would happen if we wrote NOT IN?

49. In/Not In Format The format of these subqueries is always: –Attribute or value –In / Not in –Subquery that returns a single column Returns true if the attribute value is in / not in the result of the subquery 49

50. 50 What does this produce? SELECT S.sname FROM Sailors S WHERE S.sid NOT IN (SELECT R.sid FROM Reserves R WHERE R.bid IN (SELECT B.bid FROM Boats B WHERE B.color='red')) Names of sailors who did not reserve a red boat

51. Any/All Format The format of these subqueries is always: –Attribute or value –Arithmetic comparison operator –Any / All –Subquery that returns a single column Returns true if the attribute value satisfies the arithmetic operator with respect to any/all of the query results 51

52. 52 Set-Comparison Queries SELECT * FROM Sailors S1 WHERE S1.age > ANY (SELECT S2.age FROM Sailors S2); We can also use op ALL (op is >, =, ).

53. Exists/Not Exists Format The format of these subqueries is always: –Exists / Not Exists –Subquery that returns any number of columns Returns true if the subquery returns a non- empty (resp. empty) result 53

54. 54 Correlated Nested Queries SELECT S.sid FROM Sailors S WHERE EXISTS (SELECT * FROM Reserves R WHERE R.bid = 103 and S.sid = R.sid); Sid of sailors who reserved boat 103 Q: What if we wrote NOT EXISTS? A: We would get sid of sailors who did not reserve boat 103 S not in subquery, refers to outer loop

55. 55 Exists and Not Exists Differs from In and Not In Exists: For every tuple in the outer loop, the inner loop is tested. If the inner loop produces a result, the outer tuple is added to the result.

56. 56 How would you find the names of sailors who have reserved a red boat but not a green boat? SELECT SS.sname from sailors SS where SS.sid in ( SELECT R1.sid FROM Reserves R1, Boats B1 WHERE R1.bid=B1.bid and B1.color=‘red’ EXCEPT SELECT R2.sid FROM Reserves R2, Boats B2 WHERE R2.bid=B2.bid and B2.color=‘green’ );

57. 57 Rewrite using not in SELECT SS.sname from sailors SS where SS.sid in ( SELECT R1.sid FROM Reserves R1, Boats B1 WHERE R1.bid=B1.bid and B1.color=‘red’ and R1.sid not in ( SELECT R2.sid FROM Reserves R2, Boats B2 WHERE R2.bid=B2.bid and B2.color=‘green’ ));

58. 58 How would you find the sailors who have reserved all boats?

59. 59 Remember: Algebraic Operator of Division Consider: A(X,Y) and B(Y). Then A  B = In general, we require that the set of fields in B be contained in those of A.

60. 60 Suppliers from A who supply All Parts from B (1) snopno S1 S2 S3 S4 P1 P2 P3 P4 P1 P2 P4 A P2 pno B1 sno  =

61. 61 Suppliers from A who supply All Parts from B (2) snopno S1 S2 S3 S4 P1 P2 P3 P4 P1 P2 P4 A sno  = P2 P4 pno B2

62. 62 Suppliers from A who supply All Parts from B (3) snopno S1 S2 S3 S4 P1 P2 P3 P4 P1 P2 P4 A sno  = P1 P2 P4 pno B3

63. 63 Sailors who Reserved all Boats Reserves(sid,bid)  Boats(bid) Sailor S whose "set of boats reserved" contains the "set of all boats" So what is the result of this?

64. 64 What is the strategy for finding sailors who have reserved all boats? The sailors for which there does not exist a boat which they have not reserved

65. 65 Sailors who reserved all boats (Division 1) SELECT sid FROM Sailors S WHERE NOT EXISTS (SELECT B.bid FROM Boats B WHERE B.bid NOT IN (SELECT R.bid FROM Reserves R WHERE R.sid = S.sid)); Sailors for which there does not exist a boat that they did not reserve

66. 66 Sailors who reserved all boats (Division 2) SELECT S.sid FROM Sailors S WHERE NOT EXISTS( SELECT B.bid FROM Boats B WHERE NOT EXISTS( SELECT R.bid FROM Reserves R WHERE R.bid=B.bid and R.sid=S.sid)) Sailors for which there does not exist a boat that they did not reserve

67. 67 Sailors who reserved all boats (Division 3) SELECT S.sid FROM Sailors S WHERE NOT EXISTS((SELECT B.bid FROM Boats B) EXCEPT (SELECT R.bid FROM Reserves R WHERE R.sid = S.sid)); Sailors for which there does not exist a boat for which there is no reservation in Reserves

68. 68 Aggregation

69. 69 Aggregate Operators The aggregate operators available in SQL are: –COUNT(*) –COUNT([DISTINCT] A) –SUM([DISTINCT] A) –AVG([DISTINCT] A) –MAX(A) –MIN(A)

70. 70 Some Examples SELECT COUNT(*) FROM Sailors S SELECT AVG(S.age) FROM Sailors S WHERE S.rating=10 SELECT COUNT(distinct color) FROM Boats SELECT COUNT(sid) FROM Sailors S 

71. 71 Find Average Age for each Rating So far, aggregation has been applied to all tuples that passed the WHERE clause test. How can we apply aggregation to groups of tuples?

72. 72 Find Average Age for each Rating SELECT AVG(age) FROM Sailors GROUP BY rating;

73. 73 Basic SQL Query SELECT [Distinct] attributes FROM relation-list WHERE condition GROUP BY grouping-attributes HAVING group-condition; attributes: must appear in grouping-attributes or aggregation operators group-condition: Constrains groups. Can only constrain attributes appearing in grouping-attributes or in aggregation operators

74. 74 Evaluation- important! 1.Compute cross product of relation-list 2.Tuples failing condition are thrown away 3.Tuples are partitioned into groups by values of grouping- attributes 4.The group-condition is applied to eliminate groups 5.One answer in generated for each group! SELECT [Distinct] attributes FROM relation-list WHERE condition GROUP BY grouping-attributes HAVING group-condition;

75. 75 Sailors sidSnameratingage 22 31 58 63 78 84 Dustin Lubber Rusty Fluffy Morley Popeye 7 8 10 7 10 45.0 55.5 35.0 44.0 31.0 33.0 Sailors sidsnameratingage 22 63 78 31 58 84 Dustin Fluffy Morley Lubber Rusty Popeye 7 8 10 45.0 44.0 31.0 55.5 35.0 33.0 40 55.534 SELECT AVG(age) FROM Sailors GROUP BY rating;

76. 76 SELECT AVG(age) FROM Sailors Where age<50 GROUP BY rating Having count(*)>2; SidSnameratingage 22 31 58 63 78 84 Dustin Lubber Rusty Fluffy Morley Popeye 7 8 10 7 10 45.0 55.5 35.0 44.0 31.0 33.0 Step 1 SidSnameratingage 22 58 63 78 84 Dustin Rusty Fluffy Morley Popeye 7 10 7 10 45.0 35.0 44.0 31.0 33.0

77. 77 SELECT AVG(age) FROM Sailors Where age<50 GROUP BY rating Having count(*)>2; Step 2 Sidsnameratingage 22 63 78 58 84 Dustin Fluffy Morley Rusty Popeye 7 10 45.0 44.0 31.0 35.0 33.0

78. 78 SELECT AVG(age) FROM Sailors Where age<50 GROUP BY rating Having count(*)>2; Step 3 Sidsnameratingage 22 63 78 58 84 Dustin Fluffy Morley Rusty Popeye 7 10 45.0 44.0 31.0 35.0 33.0 Sidsnameratingage 22 63 78 Dustin Fluffy Morley 777777 45.0 44.0 31.0

79. 79 SELECT AVG(age) FROM Sailors Where age<50 GROUP BY rating Having count(*)>2; Step 4 Sidsnameratingage 22 63 78 Dustin Fluffy Morley 777777 45.0 44.0 31.0 40 Final Answer:

80. 80 Find name and age of oldest Sailor..? SELECT S.sname, MAX(S.age) FROM Sailors S Wrong! Why? SELECT S.sname, MAX(S.age) FROM Sailors S GROUP BY S.sname Wrong! Why?

81. 81 Find name and age of oldest Sailor SELECT S.sname, S.age FROM Sailors S WHERE S.age = (SELECT MAX(S2.age) FROM Sailors S2) Right!! How else can this be done? HINT: Use ALL

82. 82 What does this return? SELECT B.bid, COUNT(*) FROM Boats B, Reserves R WHERE R.bid=B.bid and B.color=‘red’ GROUP BY B.bid What would happen if we put the condition about the color in the HAVING clause?

83. 83 What would happen if we put the condition about the color in the HAVING clause? SELECT B.bid, COUNT(*) FROM Boats B, Reserves R WHERE R.bid=B.bid GROUP BY B.bid, B.color HAVING B.color=‘red’

84. 84 What does this return? Can we move the condition in the HAVING to the WHERE? SELECT bname FROM Boats B, Reserves R WHERE R.bid=B.bid GROUP BY bid, bname HAVING count(DISTINCT day) <= 5 No! Aggregate functions are not allowed in WHERE Names of Boats that were not Reserved on more than 5 days

85. 85 The Color for which there are the most boats..? SELECT color FROM Boats B GROUP BY color HAVING max(count(bid)) What is wrong with this? How would you fix it?

86. 86 The Color for which there are the most boats SELECT color FROM Boats B GROUP BY color HAVING count(bid) >= ALL (SELECT count(bid) FROM Boats GROUP BY Color)

87. 87 Aggregation Instead of Exists Aggregation can take the place of exists. What does this return? SELECT color FROM Boats B1 WHERE NOT EXISTS( SELECT * FROM Boats B2 WHERE B1.bid <> B2.bid AND B1.color=B2.color) The color of boats which have a unique color (no other boats with the same color)

88. 88 Aggregation Instead of Exists SELECT color FROM Boats B1 GROUP BY color HAVING count(bid) = 1 Somewhat simpler…

89. 89 Subqueries in the FROM and in the SELECT clauses

90. 90 A Complex Query We would like to create a table containing 3 columns: –Sailor id –Sailor age –Age of the oldest Sailor (same value in all rows) How can this be done?

91. What We Want: 91 Sailors sidsnameratingage 22 31 58 Dustin Lubber Rusty 7 8 10 45.0 55.5 35.0 Result sidage Max- age 22 31 58 45.0 55.5 35.0 55.5

92. 92 Attempt 1 SELECT S.sid, S.age, MAX(S.age) FROM Sailors S; Why is this wrong?

93. 93 Attempt 2 SELECT S.sid, S.age, MAX(S.age) FROM Sailors S GROUP BY S.id, S.age; Why is this wrong?

94. 94 Solution 1: Subquery in FROM SELECT S.sid, S.age, M.mx FROM Sailors S,(SELECT MAX(S2.age) as mx FROM Sailors S2) M; We can put a query in the FROM clause instead of a table The query in the FROM clause must be renamed with a range variable (M in this case).

95. 95 Solution 2: Subquery in SELECT SELECT S.sid, S.age, (SELECT MAX(S2.age) FROM Sailors S2) FROM Sailors S; A query in the SELECT clause must return at most one value for each row returned by the outer query.

96. 96 Another Example of a Sub-query in SELECT SELECT S.sid, S.age, (SELECT MAX(S2.age) FROM Sailors S2 WHERE S2.age<S.age) FROM Sailors S; What does this query return? Note the use of S (defined in the outer query) within the inner query.

97. Result: 97 Sailors sidsnameratingage 22 31 58 Dustin Lubber Rusty 7 8 10 45.0 55.5 35.0 Result sidage (Select …) 22 31 58 45.0 55.5 35.0 45.0 null

98. 98 Another Example of a Sub-query in FROM?? SELECT S.sid, S.age, M.mx FROM Sailors S, (SELECT MAX(S2.age) as mx FROM Sailors S2 WHERE S2.age<S.age); Why is this wrong?

99. Translating RA to SQL 99

100. RA is strictly less expressive than SQL Every query in relational algebra can be equivalently written in SQL There are SQL queries that cannot be expressed using relational algebra –Examples? We now present a procedure for translating RA to SQL –Note: This is not the most efficient translation 100

101. Assumptions To make the presentation simpler, assume we are translating a relational algebra expression E into SQL where: –E does not use the same relation twice –No two relations have any attributes with the same names Easy to overcome these assumptions using RA renaming and SQL aliasing. 101

102. Translation By Induction on Structure of E Induction on the number of relational algebra operators appearing in E. Base case: 0 operators. –E is simply a relation R 102 SELECT DISTINCT * FROM R

103. Translation By Induction on Structure of E Induction Step: Assume that for all E with less than k operators, there is an SQL expression equivalent to E. We show for k Must consider several cases, depending on the “last operator” using in E: , , , , - 103

104. Last Operator is  E =  C (E 1 ), where C is a Boolean condition Let S be an SQL expression equivalent to E 1 (there is one by the induction hypothesis) E is equivalent to 104 SELECT DISTINCT * FROM S WHERE C Sub-query in the FROM Clause!

105. Last Operator is  E =  A1,..,Ak (E 1 ), where A1,…,Ak are attributes Let S be an SQL expression equivalent to E 1 (there is one by the induction hypothesis) E is equivalent to 105 SELECT DISTINCT A1,…,Ak FROM S Sub-query in the FROM Clause!

106. Last Operator is  E = E 1  E 2 Let S 1,S 2 be SQL expressions equivalent to E 1 and E 2 E is equivalent to 106 S1 UNION S2

107. Last Operator is  E = E 1  E 2 Let S 1,S 2 be SQL expressions equivalent to E 1 and E 2 E is equivalent to 107 SELECT * FROM S1, S2 Sub-queries in the FROM Clause!

108. Last Operator is - E = E 1 - E 2 Let S 1,S 2 be SQL expressions equivalent to E 1 and E 2 E is equivalent to 108 S1 EXCEPT S2

109. Example Translate –  sname,color (  rating<10 (Sailors  Boats)) 109

110. 110 Null Values

111. What does NULL Mean? There are different interpretations to a value of NULL 1.Value Unknown: I know that there is a value that belongs here, but I don’t know what it is. –Example: Birthday attribute 2.Value Inapplicable: There is no value that makes sense here –Example: Spouse attribute for unmarried person 111

112. What does NULL Mean? (cont) 3.Value Withheld: We are not entitled to know this value –Example: phone number attribute 112

113. 113 Null Values in Expressions Two important rules: –When we operate on NULL and any other value, (including another NULL), using an arithmetic operator like * or +, the result is always NULL –When we compare NULL and any other value (including another NULL), using a comparison operator like = or >, the result is UNKNOWN. The correct way to determine if an attribute x has value NULL is using x IS NULL or x IS NOT NULL, which will return true or false

114. 3 Valued Logic: True, False, Unknown ABA and BA or B True False True Unknown True FalseTrueFalseTrue False UnknownFalseUnknown TrueUnknownTrue UnknownFalse Unknown 114 ANot A TrueFalse True Unknown Only tuples for which the WHERE clause has value TRUE are used to create tuples in the result

115. What does this return? SELECT * FROM Sailors WHERE sname = sname 115

116. What does this return? SELECT * FROM Sailors WHERE rating > 5 or rating <= 5 116

117. What do these return? SELECT sname, rating * 0 FROM Sailors 117 SELECT sname, rating - rating FROM Sailors

118. 118 Nulls in Aggregation Functions count(*): counts all rows (even rows that are all null values) count(A): counts non-null A-s. returns 0 if all As are null sum(A), avg(A), min(A), max(A) –ignore null values of A –if A only contains null value, the result is null

119. 119 Distinct and Group By Rows are considered identical, for group by and distinct, if they have all the same non- null values and both have null values in the same columns Distinct removes duplicates of such rows Such rows form a single group when using GROUP BY

120. 120 Example SELECT count(*), count(c), min(c), sum(c) FROM (SELECT c FROM R WHERE c IS NULL or c <> NULL GROUP BY c) BC 1null 2 34 35 R

121. Join Operators in the FROM Clause “Syntactic Sugar” 121

122. Shorthand for Conditional Join 122 SELECT S1.sname, S2.sname FROM Sailors S1, Sailors S2 WHERE S1.sid != S2.sid and S1.sname = ‘Rusty’ SELECT S1.sname, S2.sname FROM Sailors S1 JOIN Sailors S2 on S1.sid != S2.sid WHERE S1.sname = ‘Rusty’

123. Shorthand for Equi-Join 123 SELECT S.sname, FROM Sailors S, Reserves R WHERE S.sid = R.sid and S.age > 20 SELECT S.sname, FROM Sailors S JOIN Reserves R USING (sid) WHERE S.age > 20

124. Shorthand for Natural Join 124 SELECT S.sname, FROM Sailors S, Reserves R WHERE S.sid = R.sid and S.age > 20 SELECT S.sname, FROM Sailors S NATURAL JOIN Reserves R WHERE S.age > 20 Requires equality on all common fields

125. 125 Outer Join

126. 126 Left Outer Join The left outer join of R and S contains: –all the tuples in the join of R and S –all the tuples in R that did not join with tuples from S, padded with null values SELECT Sailors.sid, Reserves.bid FROM Sailors NATURAL LEFT OUTER JOIN Reserves

127. 127 Result Reserves sidbidday 22 58 101 103 10/10/96 11/12/96 Sailors sidsnameratingage 22 31 58 Dustin Lubber Rusty 7 8 10 45.0 55.5 35.0 Result sidbid 31 22 58 null 101 103

128. 128 Right Outer Join, Full Outer Join The right outer join of R and S contains: –all the tuples in the join of R and S –all the tuples in S that did not join with tuples from R, padded with null values The full outer join of R and S contains: –all the tuples in the left outer join of R and S –all the tuples in the right outer join of R and S

129. 129 Express the Left Outer Join in SQL, without the Left Outer Join Operator Suppose we have R(A,B) and S(B,C). Can you write a query that returns the left outer join of R and S, that does not use the left outer join operator?

130. ALL and ANY: Special Cases 130

131. Query 1: What does this return? SELECT * FROM Sailors WHERE age > ANY (SELECT age FROM Sailors WHERE sname=‘Joe’) 131

132. Query 2: What does this return? SELECT * FROM Sailors WHERE age > ALL (SELECT age FROM Sailors WHERE sname=‘Joe’) 132

133. Query Containment We say that a query Q1 contains query Q2, if for all databases D, the result of applying Q1 to D contains the result of applying Q2 to D. Does Query 2 contain Query 1? Does Query 1 contain Query 2? (the answer to both questions is no – but why?) 133

134. Query Q3: What does this return? SELECT * FROM Sailors WHERE age = ANY (SELECT age FROM Sailors WHERE sname=‘Joe’) 134

135. Query Q4: What does this return? SELECT * FROM Sailors WHERE age IN (SELECT age FROM Sailors WHERE sname=‘Joe’) 135 Equivalent to Q3

136. Query Q5: What does this return? SELECT * FROM Sailors WHERE age <> ANY (SELECT age FROM Sailors WHERE sname=‘Joe’) 136

137. Query Q6: What does this return? SELECT * FROM Sailors WHERE age NOT IN (SELECT age FROM Sailors WHERE sname=‘Joe’) 137 Not equivalent to Q5 – why?

138. 138 Views

139. 139 What is a View? A view is a virtual table A view is defined by a query The result of the query is the contents of the virtual table –always update with respect to the database –does not exist, is computed every time referenced Changing a table (insert/update/delete) automatically changes the view

140. Defining a View CREATE VIEW AS ; –Where view-def is an SQL query Example: CREATE VIEW GreatSailors AS SELECT sid, sname FROM Sailors WHERE rating>=9 140

141. Defining a View Another example: CREATE VIEW SailorsDates AS SELECT sid, date FROM Sailors S, Reservations R WHERE S.sid = R.sid 141

142. Querying a View Once you have defined a view, you can use it in a query (in the same way that you use a relation) SELECT sid FROM GreatSailors WHERE sname = ‘Joe’ 142

143. Querying a View You can use a view and a regular relation together in a query SELECT bname FROM GreatSailors G, Reservations R, Boats B WHERE G.sid = R.sid and R.bid = B.bid 143

144. Understanding Queries using Views When writing a query with a view it is as if the expression defining the view is a sub- query is the FROM clause 144 SELECT bname FROM GreatSailors G, Reservations R, Boats B WHERE G.sid = R.sid and R.bid = B.bid SELECT bname FROM (SELECT sid FROM Sailors WHERE rating >=9) G, Reservations R, Boats B WHERE G.sid = R.sid and R.bid = B.bid

145. 145 What are views good for? (1) Simplifying complex queries: Here is an example allows the user to "pretend" that there is a single table in the database –CREATE VIEW SRB as SELECT S.sid, sname, rating, age, R.bid, day, bname, color FROM Sailors S, Boats B, Reserves R WHERE S.sid = R.sid and R.bid = B.bid

146. 146 What are views good for? (1) Now: Find snames of Sailors who reserved red boats on 1/11/09 using SRB SELECT sname FROM SRB WHERE color = ‘red’ and day = ‘1/11/09’

147. 147 What are views good for? (2) Security issues – preventing unauthorized access. Example: hiding the rating value CREATE VIEW SailorInfo SELECT sname, sid, age FROM Sailors grant SELECT on SailorInfo to shimon;

148. Modifying Views Sometimes it is possible to insert into, delete from, or update, a view !!! Actually, the user request is translated into a modification of the base tables (the tables used in the view definition) Modifications are possible only when the view is updatable 148

149. Updatable Views There are complex rules determining when a view is updatable Basically, updates are possible when the view is defined by selecting (SELECT, not SELECT DISTINCT) from a single relation R such that: 1.The WHERE clause does not involve R in a subquery 2.The FROM clause contains only the one relation R 3.The list in the SELECT clause must include enough attributes such that for every tuple inserted through the view, we can fill the other attributes with NULL or a default value 149

150. Inserting Example CREATE VIEW GreatSailors AS SELECT sid, sname FROM Sailors WHERE rating>=9 150 INSERT INTO GreatSailors VALUES(113, ‘Sam’) INSERT INTO Sailors(sid, sname) VALUES(113, ‘Sam’) Interestingly, we won’t see this tuple when we query GreatSailors

151. IMPORTANT NOTE There is no relation GreatSailors Insertion actually affects the table over which GreatSailors is defined, i.e., Sailors Similarly, deletion and updates will affect the underlying tables… 151

152. Deleting Example CREATE VIEW GreatSailors AS SELECT sid, sname FROM Sailors WHERE rating>=9 152 DELETE FROM GreatSailors WHERE sname = ‘John’ DELETE FROM Sailors WHERE sname = ‘John’ and rating>=9 We add the where condition from the view definition to make sure that only tuples appearing in the view are deleted

153. Updating Example CREATE VIEW GreatSailors AS SELECT sid, sname FROM Sailors WHERE rating>=9 153 Update GreatSailors SET sname = ‘Abraham’ WHERE sname = ‘John’ Update Sailors SET sname = ‘Abraham’ WHERE sname = ‘John’and rating>=9

154. Postgres Support Postgres does not support updatable views Can achieve the same effect using triggers… 154

155. Recursion 155

156. Flights Flight(airline, from, to) 156 AirlineFrmTo El AlTel AvivNew York ContinentalNew YorkLos Angeles Air CanadaLos AngelesToronto Air CanadaLos AngelesMontreal

157. Can you find? How can you find all places that you can get to by a direct flight from Tel Aviv? 157 SELECT to FROM Flights WHERE frm = ‘Tel Aviv’

158. Can you find? How can you find all places that you can get to by a flight with one stop-over from Tel Aviv? 158 SELECT F2.to FROM Flights F1, Flights F2 WHERE F1.frm = ‘Tel Aviv’ and F1.to = F2.frm

159. Can you find? How can you find all places that you can get to by a flight with zero or one stop-over from Tel Aviv? 159 SELECT to FROM Flights WHERE frm = ‘Tel Aviv’ UNION SELECT F2.to FROM Flights F1, Flights F2 WHERE F1.frm = ‘Tel Aviv’ and F1.to = F2.frm

160. Why can’t you find? How can you find all places that you can get to by a flight any number of stop-overs from Tel Aviv? Problem: How many times should Flights appear in the FROM clause? 160

161. Recursion in SQL The SQL-99 standard allows us to define temporary relations which can be recursive WITH R AS Or more generally: WITH [RECURSIVE] R1 AS, …, [RECURSIVE] Rn AS 161

162. Example WITH RECURSIVE Reaches(frm,to) AS (SELECT frm, to FROM Flights) UNION (SELECT R1.frm, R2.to FROM Reaches R1, Reaches R2 WHERE R1.to = R2.frm) SELECT to FROM Reaches WHERE frm=‘Tel Aviv’ 162

163. Fix-Point Semantics The value of Reaches is derived by repeatedly evaluating its definition until no changes are made –Before starting evaluation, Reaches is empty –Then, its definition is repeatedly evaluated, and the result defines Reaches –This continues until no more changes appear 163

164. 164 WITH RECURSIVE Reaches(frm,to) AS (SELECT frm, to FROM Flights) UNION (SELECT R1.frm, R2.to FROM Reaches R1, Reaches R2 WHERE R1.to = R2.frm) SELECT to FROM Reaches WHERE frm=‘Tel Aviv’ AirlineFrmTo El AlTel AvivNew York ContinentalNew YorkLos Angeles Air CanadaLos AngelesToronto Air CanadaLos AngelesMontreal Flights FrmTo Tel AvivNew York Los Angeles Toronto Los AngelesMontreal Reaches: Step 1

165. 165 AirlineFrmTo El AlTel AvivNew York ContinentalNew YorkLos Angeles Air CanadaLos AngelesToronto Air CanadaLos AngelesMontreal Flights FrmTo Tel AvivNew York Los Angeles Toronto Los AngelesMontreal Reaches: Step 1 FrmTo Tel AvivNew York Los Angeles Toronto Los AngelesMontreal Tel AvivLos Angeles New YorkToronto New YorkMontreal Reaches: Step 2

166. 166 AirlineFrmTo El AlTel AvivNew York ContinentalNew YorkLos Angeles Air CanadaLos AngelesToronto Air CanadaLos AngelesMontreal Flights FrmTo Tel AvivNew York Los Angeles Toronto Los AngelesMontreal Reaches: Step 1 FrmTo Tel AvivNew York Los Angeles Toronto Los AngelesMontreal Tel AvivLos Angeles New YorkToronto New YorkMontreal Tel AvivToronto Tel AvivMontreal Reaches: Step 3

167. 167 AirlineFrmTo El AlTel AvivNew York ContinentalNew YorkLos Angeles Air CanadaLos AngelesToronto Air CanadaLos AngelesMontreal Flights FrmTo Tel AvivNew York Los Angeles Toronto Los AngelesMontreal Reaches: Step 1 FrmTo Tel AvivNew York Los Angeles Toronto Los AngelesMontreal Tel AvivLos Angeles New YorkToronto New YorkMontreal Tel AvivToronto Tel AvivMontreal Reaches: Final Value Query will return values in Red

168. Mutually Recursive Relations We can define several recursive queries, which can use one another in their definitions. A dependency graph has a node for each relation defined, and an edge from one node to another if the first uses the second in its definition –In particular, in the previous example, there would be an edge from Reaches to itself R and S are mutually recursive, if there is a cycle in the graph involving nodes R and S 168

169. Example WITH RECURSIVE P(x) AS (SELECT * FROM R) EXCEPT (SELECT * FROM Q) RECURSIVE Q(x) AS (SELECT * FROM R) EXCEPT (SELECT * FROM P) SELECT * FROM P 169 R P Q P and Q are mutually recursive

170. Problematic Recursion Complicated recursions are allowed However, sometimes the result may not be well defined –These cases are not allowed by SQL Before defining exactly what is not allowed, we consider an example 170

171. Is there a Fix Point? Recall that the result of a defined relation is derived by simply evaluating it again and again until it no longer changes. However, what happens if this process never terminates? 171

172. Example WITH RECURSIVE P(x) AS (SELECT * FROM R) EXCEPT (SELECT * FROM Q) RECURSIVE Q(x) AS (SELECT * FROM R) EXCEPT (SELECT * FROM P) SELECT * FROM P 172 What is in P and Q if R has the single tuple (0)?

173. Monotonicity Requirement R can be defined using a mutually recursive relation S only if R is monotone in S, i.e., –Adding an arbitrary tuple to S might add tuples to R, or might leave R unchanged, but can never cause a tuple to be deleted from R 173

174. Monotonicity Requirement In the previous example, R uses the mutually recursive relation Q, but R is not monotonic in Q (adding tuples to Q can cause tuples to be removed from R) Therefore, this type of recursion is not allowed in SQL 174

175. Another Non-monotonic (Bad) Example WITH RECURSIVE P(x) AS (SELECT * FROM R) UNION (SELECT * FROM Q) RECURSIVE Q(x) AS SELECT SUM(x) FROM P SELECT * FROM P 175 Suppose that R starts out with the tuples (1) and (2) What will be in P? What will be in Q?