Relational Algebra Chpt 4a Xintao Wu Raghu Ramakrishnan The slides for this text are organized into several modules. Each lecture contains about enough material for a 1.25 hour class period. (The time estimate is very approximate--it will vary with the instructor, and lectures also differ in length; so use this as a rough guideline.) This covers Lecture 1 (of 6) in Module (3). Module (1): Introduction (DBMS, Relational Model) Module (2): Storage and File Organizations (Disks, Buffering, Indexes) Module (3): Database Concepts (Relational Queries, DDL/ICs, Views and Security) Module (4): Relational Implementation (Query Evaluation, Optimization) Module (5): Database Design (ER Model, Normalization, Physical Design, Tuning) Module (6): Transaction Processing (Concurrency Control, Recovery) Module (7): Advanced Topics Raghu Ramakrishnan 1

Relational Query Languages Query languages: Allow manipulation and retrieval of data from a database. Relational model supports simple, powerful QLs: Strong formal foundation based on logic. Allows for much optimization. Query Languages != programming languages! QLs not expected to be “Turing complete”. QLs not intended to be used for complex calculations. QLs support easy, efficient access to large data sets. Raghu Ramakrishnan 2

Formal Relational Query Languages Two mathematical Query Languages form the basis for “real” languages (e.g. SQL), and for implementation: Relational Algebra: More operational, very useful for representing execution plans. Relational Calculus: Lets users describe what they want, rather than how to compute it. (Non-operational, declarative.) Understanding Algebra is key to understanding SQL, query processing! Raghu Ramakrishnan

Preliminaries A query is applied to relation instances, and the result of a query is also a relation instance. Schemas of input relations for a query are fixed (but query will run regardless of instance!) The schema for the result of a given query is also fixed! Determined by definition of query language constructs. Positional vs. named-field notation: Positional notation easier for formal definitions, named-field notation more readable. Both used in SQL Raghu Ramakrishnan 4

Example Instances R1 “Sailors” and “Reserves” relations for our examples. We’ll use positional or named field notation, assume that names of fields in query results are `inherited’ from names of fields in query input relations. S1 S2 Raghu Ramakrishnan 5

Relational Algebra Basic operations: Additional operations: Selection ( ) Selects a subset of rows from relation. Projection ( ) Deletes unwanted columns from relation. Cross-product ( ) Allows us to combine two relations. Set-difference ( ) Tuples in reln. 1, but not in reln. 2. Union ( ) Tuples in reln. 1 and in reln. 2. Additional operations: Intersection, join, division, renaming: Not essential, but (very!) useful. Raghu Ramakrishnan 6

Projection Deletes attributes that are not in projection list. Schema of result contains exactly the fields in the projection list, with the same names that they had in the (only) input relation. Projection operator has to eliminate duplicates! Note: real systems typically don’t do duplicate elimination unless the user explicitly asks for it. Raghu Ramakrishnan 7

Selection Selects rows that satisfy selection condition. No duplicates in result! Schema of result identical to schema of (only) input relation. Result relation can be the input for another relational algebra operation! (Operator composition.) Raghu Ramakrishnan 8

Union, Intersection, Set-Difference All of these operations take two input relations, which must be union-compatible: Same number of fields. `Corresponding’ fields have the same type. What is the schema of result? Occur in either S1 or S2 or both Occur in S1 but not in S2 Occur in both S1 and S2 Raghu Ramakrishnan 9

Cross-Product Each row of S1 is paired with each row of R1. Result schema has one field per field of S1 and R1, with field names `inherited’ if possible. Conflict: Both S1 and R1 have a field called sid. Renaming operator: Raghu Ramakrishnan 10

Joins Condition Join: Result schema same as that of cross-product. Fewer tuples than cross-product, might be able to compute more efficiently Sometimes called a theta-join. Raghu Ramakrishnan 11

Joins Equi-Join: A special case of condition join where the condition c contains only equalities. Result schema similar to cross-product, but only one copy of fields for which equality is specified. Natural Join: Equijoin on all common fields. Raghu Ramakrishnan 12

Division Not supported as a primitive operator, but useful for expressing queries like: Find sailors who have reserved all boats. Let A have 2 fields, x and y; B have only field y: A/B contains all x tuples (sailors) such that for every y tuple (boat) in B, there is an xy tuple in A. Or: If the set of y values (boats) associated with an x value (sailor) in A contains all y values in B, the x value is in A/B. In general, x and y can be any lists of fields; y is the list of fields in B, and x y is the list of fields of A. Raghu Ramakrishnan 13

Examples of Division A/B Raghu Ramakrishnan 14

Expressing A/B Using Basic Operators Division is not essential op; just a useful shorthand. (Also true of joins, but joins are so common that systems implement joins specially.) Idea: For A/B, compute all x values that are not `disqualified’ by some y value in B. x value is disqualified if by attaching y value from B, we obtain an xy tuple that is not in A. Disqualified x values: A/B: all disqualified tuples Raghu Ramakrishnan 15

Division Find all sailors who have booked all boats Idea A is instance of Reserve(sid, bid, date) B is instance of Boat(bid,bname,bcolor) Idea Find all sailors who have booked at least one boat Assume all the above sailors book all the boats Compare with the original A to find those false reservation Get the sailors who are involved in the false reservation, those sailors are disqualified. Remove those disqualified from sailors, we get the qualified sailors who have booked all the boats. Raghu Ramakrishnan

Expressing A/B Using Basic Operators (continued) sno pno s1 p2 p4 s2 s3 s4 sno pno s2 p4 s3 A/B2 sno s1 s2 s3 s4 p sno A X B2 ) - A ( sno s2 s3 p sno A p sno A X B2 ) - A ( ) p sno A X B2 sno p sno A X B2 ) - A ( )) = s1 p sno A - ( s4 Raghu Ramakrishnan 15

Expressing A/B Using Basic Operators (continued) Work A/B3 on board Raghu Ramakrishnan 15

Expressing A/B Using Basic Operators (continued) sno pno s2 p4 sno pno A/B3 s3 p1 s1 p1 sno s1 s2 s3 s4 s3 p4 s1 p2 s1 p4 s4 p1 s2 p1 p sno A X B3 ) - A ( s2 p2 s2 p4 sno s3 p1 s3 p2 s2 p sno A s3 p4 s3 s4 p1 s4 s4 p2 p sno A X B3 p sno A X B3 ) - A ( ) s4 p4 p sno A X B3 ) - A ( )) = - ( sno s1 p sno A Raghu Ramakrishnan 15

Example Instances bid bname bcolor 101 Gippy red green 103 Fullsail R1 Raghu Ramakrishnan 5

Find names of sailors who’ve reserved boat #103 Solution 1: decompose: Solution 2: Raghu Ramakrishnan 16

Find names of sailors who’ve reserved a red boat Information about boat color only available in Boats; so need an extra join: A more efficient solution: A query optimizer can find this given the first solution! Raghu Ramakrishnan 17

Find sailors who’ve reserved a red or a green boat Can identify all red or green boats, then find sailors who’ve reserved one of these boats: Can also define Tempboats using union! Select boats color=red and union with color=green What happens if is replaced by in this query? Nothing selected because color can’t be red & green Raghu Ramakrishnan 18

Find sailors names who’ve reserved a red and a green boat Previous approach won’t work! Must identify sailors who’ve reserved red boats, sailors who’ve reserved green boats, then find the intersection (note that sid is a key for Sailors): Raghu Ramakrishnan 19

Find the names of sailors who’ve reserved all boats Uses division; schemas of the input relations to / must be carefully chosen: To find sailors who’ve reserved all ‘Interlake’ boats: ..... Raghu Ramakrishnan 20

Summary The relational model has rigorously defined query languages that are simple and powerful. Relational algebra is more operational; useful as internal representation for query evaluation plans. Several ways of expressing a given query; a query optimizer should choose the most efficient version. Raghu Ramakrishnan

Overview of Query Optimization Plan: Tree of R.A. ops, with choice of alg for each op. Each operator typically implemented using a `pull’ interface: when an operator is `pulled’ for the next output tuples, it `pulls’ on its inputs and computes them. Two main issues: For a given query, what plans are considered? Algorithm to search plan space for cheapest (estimated) plan. How is the cost of a plan estimated? Ideally: Want to find best plan. Practically: Avoid worst plans! We will study the System R approach. 2

Schema for Examples Similar to old schema; rname added for variations. Sailors (sid: integer, sname: string, rating: integer, age: real) Reserves (sid: integer, bid: integer, day: dates, rname: string) Similar to old schema; rname added for variations. Reserves: Each tuple is 40 bytes long, 100 tuples per page, 1000 pages. Sailors: Each tuple is 50 bytes long, 80 tuples per page, 500 pages. 3

Motivating Example RA Tree: R.bid=100 AND S.rating>5 Reserves Sailors sid=sid bid=100 rating > 5 sname Motivating Example SELECT S.sname FROM Reserves R, Sailors S WHERE R.sid=S.sid AND R.bid=100 AND S.rating>5 Cost: 1000+500*1000 I/Os By no means the worst plan! Misses several opportunities: selections could have been `pushed’ earlier, no use is made of any available indexes, etc. Goal of optimization: To find more efficient plans that compute the same answer. Reserves Sailors sid=sid bid=100 rating > 5 sname (Simple Nested Loops) (On-the-fly) Plan: 4

Alternative Plans 1 (No Indexes) Reserves Sailors sid=sid bid=100 sname (On-the-fly) rating > 5 (Scan; write to temp T1) temp T2) (Sort-Merge Join) Alternative Plans 1 (No Indexes) Main difference: push selects. With 5 buffers, cost of plan: Scan Reserves (1000) + write temp T1 (10 pages, if we have 100 boats, uniform distribution). Scan Sailors (500) + write temp T2 (250 pages, if we have 10 ratings). Sort T1 (2*2*10), sort T2 (2*4*250), merge (10+250) Total: 4060 page I/Os. If we used BNL join, join cost = 10+4*250, total cost = 2770. If we `push’ projections, T1 has only sid, T2 only sid and sname: T1 fits in 3 pages, cost of BNL drops to under 250 pages, total < 2000. 5

Alternative Plans 2 With Indexes (On-the-fly) sname Alternative Plans 2 With Indexes (On-the-fly) rating > 5 With clustered index on bid of Reserves, we get 100,000/100 = 1000 tuples on 1000/100 = 10 pages. INL with pipelining (outer is not materialized). (Index Nested Loops, sid=sid with pipelining ) (Use hash index; do bid=100 Sailors not write result to temp) Reserves Projecting out unnecessary fields from outer doesn’t help. Join column sid is a key for Sailors. At most one matching tuple, unclustered index on sid OK. Decision not to push rating>5 before the join is based on availability of sid index on Sailors. Cost: Selection of Reserves tuples (10 I/Os); for each, must get matching Sailors tuple (1000*1.2); total 1210 I/Os. 6