M.P. Johnson, DBMS, Stern/NYU, Spring C : Database Management Systems Lecture #7 M.P. Johnson Stern School of Business, NYU Spring, 2008

M.P. Johnson, DBMS, Stern/NYU, Spring Agenda Basic SQL Joins Hw1 probably soon ( from Blackboard)…

M.P. Johnson, DBMS, Stern/NYU, Spring Current topic: SQL Standard language for querying and manipulating data Structured Query Language Many standards: ANSI SQL, SQL92/SQL2, SQL3/SQL99 Originally: Structured English Query Language (SEQUEL) Vendors support various subsets/extensions We’ll do Oracle/MySQL/generic  “No one ever got fired for buying Oracle.” Basic form (many more bells and whistles in addition): SELECT attributes FROM relations (possibly multiple, joined) WHERE conditions (selections) SELECT attributes FROM relations (possibly multiple, joined) WHERE conditions (selections)

M.P. Johnson, DBMS, Stern/NYU, Spring Next (parallel) topic: relational algebra Projection Selection Cartesian Product Joins: natural joins, theta joins Set operations: union, intersection, difference Combining operations to form queries Dependent and independent operations

M.P. Johnson, DBMS, Stern/NYU, Spring What is relational algebra? An algebra for relations “High-school” algebra: an algebra for numbers Algebra = formalism for constructing expressions  Operations  Operands: Variables, Constants, expressions Expressions:  Vars & constants  Operators applied to expressions  They evaluate to values AlgebraVars/constsOperatorsEval to High-schoolNumbers+ * - / etc.Numbers RelationalRelations (=sets of tupes) union, intersection, join, etc. Relations

M.P. Johnson, DBMS, Stern/NYU, Spring Why do we care about relational algebra? 1. The exprs are the form that questions about the data take  The relations these exprs cash out to are the answers to our questions 2. RA ~ more succinct rep. of many SQL queries 3. DBMS parse SQL into something like RA First proofs of concept for RDBMS/RA:  System R at IBM  Ingress at Berkeley “Modern” implementation of RA: SQL  Both state of the art, mid-70s

M.P. Johnson, DBMS, Stern/NYU, Spring Relation operators Basic operators:  Selection:   Projection:   Cartesian Product:  Other set-theoretic ops:  Union:   Intersection:  Difference: - Additional operators:  Joins (natural, equijoin, theta join, semijoin)  Renaming:   Grouping…

M.P. Johnson, DBMS, Stern/NYU, Spring Selection op Selects all tuples satisfying a condition Notation:  c (R) Examples   salary > (Employee)   name = “Smith” (Employee) The condition c can have  comparison ops:=,, , <>  boolean ops: and, or

M.P. Johnson, DBMS, Stern/NYU, Spring Selection example Select the movies at Angelica:   Theater=“Sunshine” (Showings) Masc. Fem.VillageFilm Forum Village N’hood Bad Edu. Annie Hall Title Sunshine Theater Village N’hood Bad Edu. Annie Hall Title Sunshine Theater

M.P. Johnson, DBMS, Stern/NYU, Spring Projection op Keep only certain columns Projection: op we used for decomposition  Eliminates other columns, then removes duplicates Notation:  A1,…,An (R)

M.P. Johnson, DBMS, Stern/NYU, Spring Join op Corresponds to SQL query doing cross & equality test Specifically: R 1 R 2 =  every att once (  shared atts = (R 1  R 2 ))  I.e., first compute the cross product R 1 x R 2  Next, select the rows in which shared fields agree  Finally, project onto the union of R 1 and R 2 ’s fields (remove duplicates)

M.P. Johnson, DBMS, Stern/NYU, Spring Rename op Changes the schema, not the instance Notation:  B1,…,Bn (R)  is spelled “rho”, pronounced “row” Example:  Employee(ssn,name)    social, name) (Employee)  Or just:   (Employee)

M.P. Johnson, DBMS, Stern/NYU, Spring RA  SQL SQL SELECT  RA Projection  SQL WHERE  RA Selection  SQL FROM  RA Join/cross  Comma-separated list… SQL renaming  RA rho  More ops later Keep RA in the back of your mind…

M.P. Johnson, DBMS, Stern/NYU, Spring Next: Joins in SQL Connect two or more tables: PNamePriceCategoryManufacturer Gizmo$19.99GadgetsGizmoWorks Powergizmo$29.99GadgetsGizmoWorks SingleTouch$149.99PhotographyCanon MultiTouch$203.99HouseholdHitachi Product Company CNameStockPriceCountry GizmoWorks25USA Canon65Japan Hitachi15Japan What is the connection between them?

M.P. Johnson, DBMS, Stern/NYU, Spring Joins in SQL Product (pname, price, category, manufacturer) Company (cname, stockPrice, country) Find all products under $200 manufactured in Japan; return their names and prices. SELECT PName, Price FROM Product, Company WHERE Manufacturer=CName AND Country='Japan' AND Price <= 200 Join between Product and Company

M.P. Johnson, DBMS, Stern/NYU, Spring Joins in SQL PNamePriceCategoryManufacturer Gizmo$19.99GadgetsGizmoWorks Powergizmo$29.99GadgetsGizmoWorks SingleTouch$149.99PhotographyCanon MultiTouch$203.99HouseholdHitachi Product Company CnameStockPriceCountry GizmoWorks25USA Canon65Japan Hitachi15Japan PNamePrice SingleTouch$ SELECT PName, Price FROM Product, Company WHERE Manufacturer=CName AND Country='Japan' AND Price <= 200

M.P. Johnson, DBMS, Stern/NYU, Spring Joins in SQL Product (pname, price, category, manufacturer) Company (cname, stockPrice, country) Find all countries that manufacture some product in the ‘Gadgets’ category. SELECT Country FROM Product, Company WHERE Manufacturer=CName AND Category='Gadgets'

M.P. Johnson, DBMS, Stern/NYU, Spring Joins in SQL NamePriceCategoryManufacturer Gizmo$19.99GadgetsGizmoWorks Powergizmo$29.99GadgetsGizmoWorks SingleTouch$149.99PhotographyCanon MultiTouch$203.99HouseholdHitachi Product Company CnameStockPriceCountry GizmoWorks25USA Canon65Japan Hitachi15Japan Country ?? What is the problem? What’s the solution? SELECT Country FROM Product, Company WHERE Manufacturer=CName AND Category='Gadgets'

M.P. Johnson, DBMS, Stern/NYU, Spring Joins Product (pname, price, category, manufacturer) Purchase (buyer, seller, store, product) Person(name, phone, city) Find names of Seattleites who bought Gadgets, and the names of the stores they bought such product from. SELECT DISTINCT name, store FROM Person, Purchase, Product WHERE persname=buyer AND product = pname AND city='Seattle' AND category='Gadgets'

M.P. Johnson, DBMS, Stern/NYU, Spring SQL Query Semantics Parallel assignment – all tuples Doesn’t impose any order Answer = {} for all assignments x1 in R1, …, xn in Rn do if Conditions then Answer = Answer  {(a1,…,ak)} return Answer Answer = {} for all assignments x1 in R1, …, xn in Rn do if Conditions then Answer = Answer  {(a1,…,ak)} return Answer SELECT a1, a2, …, ak FROM R1 AS x1, R2 AS x2, …, Rn AS xn WHERE Conditions SELECT a1, a2, …, ak FROM R1 AS x1, R2 AS x2, …, Rn AS xn WHERE Conditions

M.P. Johnson, DBMS, Stern/NYU, Spring SQL Query Semantics Nested loops: Answer = {} for x1 in R1 do for x2 in R2 do ….. for xn in Rn do if Conditions then Answer = Answer  {(a1,…,ak)} return Answer Answer = {} for x1 in R1 do for x2 in R2 do ….. for xn in Rn do if Conditions then Answer = Answer  {(a1,…,ak)} return Answer SELECT a1, a2, …, ak FROM R1 AS x1, R2 AS x2, …, Rn AS xn WHERE Conditions SELECT a1, a2, …, ak FROM R1 AS x1, R2 AS x2, …, Rn AS xn WHERE Conditions

M.P. Johnson, DBMS, Stern/NYU, Spring Multiple join syntaxes Old-style syntax simply lists tables separated by commas New-style makes the join explicit: Functionally equivalent to old-style, but perhaps more elegant Introduced in Oracle 8i, MySQL 3.x/4.x Older versions / other DBMSs may not support this SELECT * FROM A,B WHERE …; SELECT * FROM A,B WHERE …; SELECT * FROM A JOIN B ON … WHERE …; SELECT * FROM A JOIN B ON … WHERE …;

M.P. Johnson, DBMS, Stern/NYU, Spring New-style join types Cross joins (simplest):  FROM A CROSS JOIN B Inner joins (regular joins):  FROM A [INNER] JOIN B ON … Natural join:  FROM A NATURAL JOIN B;  Joins on common fields and merges Outer joins (later)  No dangling rows

M.P. Johnson, DBMS, Stern/NYU, Spring CROSS JOIN e.g. NameAddressGenderBirthdate Hanks123 Palm RdM01/01/60 Taylor456 Maple AvF02/02/40 Lucas789 Oak StM03/03/55 NameAddressNetworth Spielberg246 Palm Rd10M Taylor456 Maple Av20M Lucas789 Oak St30M MovieStar MovieExec

M.P. Johnson, DBMS, Stern/NYU, Spring CROSS JOIN e.g. MovieS tar.nam e MovieStar.add ress MovieSta r. Gender MovieStar.Birthdate MovieEx ec. Name MovieExec.Ad dress MovieEx ec. Networth Hanks123 Palm RdM01/01/60Spielberg246 Palm Rd10M Hanks123 Palm RdM01/01/60Taylor456 Maple Av20M Hanks123 Palm RdM01/01/60Lucas789 Oak St30M Taylor456 Maple AvF02/02/40Spielberg246 Palm Rd10M Taylor456 Maple AvF02/02/40Taylor456 Maple Av20M Taylor456 Maple AvF02/02/40Lucas789 Oak St30M Lucas789 Oak StM03/03/55Spielberg246 Palm Rd10M Lucas789 Oak StM03/03/55Taylor456 Maple Av20M Lucas789 Oak StM03/03/55Lucas789 Oak St30M Row SELECT * FROM MovieStar CROSS JOIN MovieExec SELECT * FROM MovieStar CROSS JOIN MovieExec

M.P. Johnson, DBMS, Stern/NYU, Spring JOIN … ON e.g MovieSt ar.name MovieStar.addr ess MovieS tar. Gender MovieStar. Birthdate MovieExec. Name MovieExec.Add ress MovieEx ec. Networth Hanks123 Palm RdM01/01/60Spielberg246 Palm Rd10M Hanks123 Palm RdM01/01/60Taylor456 Maple Av20M Hanks123 Palm RdM01/01/60Lucas789 Oak St30M Taylor456 Maple AvF02/02/40Spielberg246 Palm Rd10M Taylor456 Maple AvF02/02/40Taylor456 Maple Av20M Taylor456 Maple AvF02/02/40Lucas789 Oak St30M Lucas789 Oak StM03/03/55Spielberg246 Palm Rd10M Lucas789 Oak StM03/03/55Taylor456 Maple Av20M Lucas789 Oak StM03/03/55Lucas789 Oak St30M Row SELECT * FROM MovieStar JOIN MovieExec ON MovieStar.Name <> MovieExec.Name SELECT * FROM MovieStar JOIN MovieExec ON MovieStar.Name <> MovieExec.Name

M.P. Johnson, DBMS, Stern/NYU, Spring NATURAL JOIN MovieStar(name, address, gender, birthdate) MovieExec(name, address, networth) Natural Join syntax:  FROM MovieStar NATURAL JOIN MovieExec Results: list of movie stars who are also execs:  (Name, address, gender, birthdate, networth)

M.P. Johnson, DBMS, Stern/NYU, Spring NATURAL JOIN e.g. NameAddressGenderBirthdate Hanks123 Palm RdM01/01/60 Taylor456 Maple AvF02/02/40 Lucas789 Oak StM03/03/55 NameAddressNetworth Spielberg246 Palm Rd10M Taylor456 Maple Av20M Lucas789 Oak St30M MovieStar MovieExec NameAddressGenderBirthdateNetworth Taylor456 Maple AvF02/02/4020M Lucas789 Oak StM03/03/5530M SELECT * FROM MovieStar NATURAL JOIN MovieExec

M.P. Johnson, DBMS, Stern/NYU, Spring Disambiguating Attributes Sometimes two relations have the same attr: Person(pname, address, worksfor) Company(cname, address) SELECT DISTINCT pname, address FROM Person, Company WHERE worksfor = cname SELECT DISTINCT Person.pname, Company.address FROM Person, Company WHERE Person.worksfor = Company.cname Which address?

M.P. Johnson, DBMS, Stern/NYU, Spring Tuple Var e.g. SELECT DISTINCT x.store FROM Purchase AS x, Purchase AS y WHERE x.product = y.product AND y.store = 'BestBuy' SELECT DISTINCT x.store FROM Purchase AS x, Purchase AS y WHERE x.product = y.product AND y.store = 'BestBuy' Find all stores that sold at least one product that the store BestBuy also sold: Result: (store) Product (pname, price, category, manufacturer) Purchase (buyer, seller, store, product) Person(persname, phoneNumber, city)

M.P. Johnson, DBMS, Stern/NYU, Spring Details: Disambiguation in SQL Every selected field must be unambiguous For R(A,B),  Select A from R, R   Select R1.A from R R1, R R2 Consider: Prob:* is shorthand for all fields  each must be unambiguous Soln: Select * from R R1, R R2 SQL> Select * from R, R; Select * from R, R * ERROR at line 1: ORA-00918: column ambiguously defined SQL> Select * from R, R; Select * from R, R * ERROR at line 1: ORA-00918: column ambiguously defined

M.P. Johnson, DBMS, Stern/NYU, Spring Details: Disambiguation in Oracle SQL Depending on DBMS, can rename fields by:  Select name as n …  Select name n …  Select name=n…(not in Oracle) Can rename relations only by  … from tab t1, tab t2

M.P. Johnson, DBMS, Stern/NYU, Spring SQL e.g. with tuple vars Reps(ssn, name, etc.) Clients(ssn, name, rssn) Q: Who are George’s clients, in SQL? Conceptually:   Clients.name (  Reps.name=“George” and Reps.ssn=rssn (Reps x Clients))

M.P. Johnson, DBMS, Stern/NYU, Spring Ordering the Results Ordering is ascending, unless you specify the DESC keyword per attribute: SELECT pname, price, manufacturer FROM Product WHERE category='gizmo' AND price > 50 ORDER BY price, pname SELECT pname, price, manufacturer FROM Product WHERE category='gizmo' AND price > 50 ORDER BY price, pname SELECT pname, price, manufacturer FROM Product WHERE category='gizmo' AND price > 50 ORDER BY price DESC, pname ASC SELECT pname, price, manufacturer FROM Product WHERE category='gizmo' AND price > 50 ORDER BY price DESC, pname ASC

M.P. Johnson, DBMS, Stern/NYU, Spring Ordering the Results SELECT Category FROM Product ORDER BY PName SELECT Category FROM Product ORDER BY PName PNamePriceCategoryManufacturer Gizmo$19.99GadgetsGizmoWorks Powergizmo$29.99GadgetsGizmoWorks SingleTouch$149.99PhotographyCanon MultiTouch$203.99HouseholdHitachi ?

M.P. Johnson, DBMS, Stern/NYU, Spring Details: Case-sensitivity In Oracle, compares are case-sensitive by default If want case-insensitive, some options: 1. Create a function index  Maybe later… 2. Manually convert vals to upper/lower case  SQL> select * from emp where upper(ename) = upper(‘Blake'); 3. Modify the nls_sort setting:  SQL> alter session set nls_sort=binary_ci;  SQL> alter session set nls_comp=ansi;  The other values: binary, binary_ai

M.P. Johnson, DBMS, Stern/NYU, Spring The LIKE operator s LIKE p: pattern matching on strings p may contain two special symbols:  _ = any single character  % = zero or more chars Product(Name, Price, Category, Manufacturer) Find all products whose name contains 'gizmo': SELECT * FROM Products WHERE Name LIKE '%gizmo%'

M.P. Johnson, DBMS, Stern/NYU, Spring The LIKE operator Q: How to search the actual '%' char?  The usual meta-char issue PName LIKE '%%' won’t work Instead, must use escape chars  In C/C++/J, prepend \  In SQL, prepend an arbitrary escape char: PName LIKE '%x%' ESCAPE 'x'

M.P. Johnson, DBMS, Stern/NYU, Spring Details: more on escape chars SQL: no official default escape char In Oracle’s SQL*Plus: default escape char = '\'  Can set with SQL> set escape x  Other tools, DBMSs: your mileage may vary SQL string literals put in ' ':  'mystring' Single-quote literals escaped with single-quotes:  'George''s string' No distinction between strings and single chars

M.P. Johnson, DBMS, Stern/NYU, Spring Details: more on escape chars Q: Can an escape char be an escape string? A: No. SQL> select * from newtable where a like '%\%' escape '\'; A B h%i there SQL> select * from newtable where a like '%\%' escape '\\'; select * from newtable where a like '%\%' escape '\\' * ERROR at line 1: ORA-01425: escape character must be character string of length 1 SQL> select * from newtable where a like '%\%' escape '\'; A B h%i there SQL> select * from newtable where a like '%\%' escape '\\'; select * from newtable where a like '%\%' escape '\\' * ERROR at line 1: ORA-01425: escape character must be character string of length 1

M.P. Johnson, DBMS, Stern/NYU, Spring Details: more on single-quotes Dates with DATE:  DATE ' ' Timestamps with TIMESTAMP:  TIMESTAMP ' :00:00' Details may vary by DBMS…

M.P. Johnson, DBMS, Stern/NYU, Spring Details: more on quotes Q: What about double quotes? A: Can’t be used in place of single quotes  But are used… But can be used when Oracle would otherwise misparse your command, e.g.: 1. Names with spaces:  create table bad table name (a int, b int); 2. Reserved words as names:  create table badfieldname(from int, b int);

M.P. Johnson, DBMS, Stern/NYU, Spring Another complex example People(ssn, name, street, city, state, state) Q: Who lives on George’s street? A: First, generate pairs of (renamed) people:   p1 (People) x  p2 (People) Then pick out pairs with George:   p1.name='George' (  p1 (People) x  p2 (People)) And refine to rows with George and someone else:   p1.name='George‘ AND p1.name<>p2.name (  p1 (People) x  p2 (People)) Finally, project out the names:   p2.name (  p1.name='George‘ AND p1.name<>p2.name (  p1 (People) x  p2 (People))

M.P. Johnson, DBMS, Stern/NYU, Spring Live examples Q: produce a list of employees and their bosses  What if no boss? Or no subordinate? Joins on emp, emp man:  Comma-based  Inner  Natural  Cross  Outer – left, right, full

M.P. Johnson, DBMS, Stern/NYU, Spring More live examples Inner joins require an ON clause  Like a where clause  Arbitrary boolean expression  If always true (1=1), reduces to cross join New compar op: BETWEEN  a between 5 and 10  a >= 5 and a <= 10 Q: produce a list of employees with their salary grades  emp, salgrade

M.P. Johnson, DBMS, Stern/NYU, Spring Review Examples from sqlzoo.netsqlzoo.net SELECT L FROM R 1, …, R n WHERE C SELECT L FROM R 1, …, R n WHERE C  L (  C (R 1 x … R n )