Query Processing SQL Queries in a high level language such as SQL are processed by Horizontal DBMSs in the following steps: 1. SCAN and PARSE (SCANNER-PARSER):

Query Processing SQL Queries in a high level language such as SQL are processed by Horizontal DBMSs in the following steps: 1. SCAN and PARSE (SCANNER-PARSER): The Scanner identifies the tokens or language elements. The Parser check for syntax or grammar validity. 2. VALIDATED: The Validator checks for valid names and semantic correctness. 3. CONVERTER converts to an internal representation (usually a QUERY TREE) |4. QUERY OPTIMIZED: Query Optimzier devises a stategy for executing query (chooses among alternative Query trees). 5. CODE GENERATION: generates code to implement each operator in the selected query plan (the optimizer-selected the query tree). 6. RUNTIME DATABASE PROCESSORING: run plan code

The CONVERTER converts to an internal representation (usually a QUERY TREE). E.g., given the database: _S______________ _C___________ _E______ |S#|SNAME |LCODE | |C#|CNAME|SITE||S#|C#|GR| |25|CLAY |NJ5101| |8 |DSDE |ND | |32|8 |89| |32|THAISZ|NJ5102| |7 |CUS |ND | |32|7 |91| |38|GOOD |FL6321| |6 |3UA |NJ | |25|7 |68| |17|BAID |NY2091| |5 |3UA |ND | |25|6 |76| |57|BROWN |NY2092| |32|6 |62| The SQL request: SELECT S.SNAME, C.CNAME, E.GR FROM S,C,E WHERE S.LCODE=NJ5101 and C.SITE="ND" and E.GR=68 and S.S#=E.S# and C.C#=E.C#; gets SCANNED, PARSED, VALIDATED, then may get CONVERTED to query tree following the sequencing of the WHERE-clause.

CONVERTER _S______________ _C___________ _E______ |S#|SNAME |LCODE | |C#|CNAME|SITE||S#|C#|GR| |25|CLAY |NJ5101| |8 |DSDE |ND | |32|8 |89| |32|THAISZ|NJ5102| |7 |CUS |ND | |32|7 |91| |38|GOOD |FL6321| |6 |3UA |NJ | |25|7 |68| |17|BAID |NY2091| |5 |3UA |ND | |25|6 |76| |57|BROWN |NY2092| |32|6 |62| M=PROJ(L)[SNAME,CNAME,GR] | L=SELECT(K.GR=68) | K=SELECT(H.SITE="ND") | H=SELECT(G.LCODE="NJ5101") | G=JOIN(F.C#=C.C#) /\ JOIN(S.S#=E.S#)=F C /\ S E This is simplest CONVERTER (uses the ordering in WHERE clause) WHERE S.LCODE=NJ5101 and C.SITE="ND" and E.GR=68 and S.S#=E.S# and C.C#=E.C#;

CONVERTER S#|SNAME |LCODE 25|CLAY |NJ5101 32|THAISZ|NJ5102 38|GOOD |FL6321 17|BAID |NY2091 57|BROWN |NY2092 M=PROJ(L)[SNAME,CNAME,GR] | | L=SELECT(K.GR=68) | | K=SELECT(H.SITE="ND") | | H=SELECT(G.LCODE="NJ5101") | | G=JOIN(F.C#=C.C#) /\ JOIN(S.S#=E.S#)=F /\ S E Let's see the results at each step. C#|CNAME|SITE 8 |DSDE | ND 7 |CUS | ND 6 |3UA | NJ 5 |3UA | ND S#|C#|GR 32|8 |89 32|7 |91 25|7 |68 25|6 |76 32|6 |62 S#|SNAME |LCODE |C#|GR 25|CLAY |NJ5101|7 |68 25|CLAY |NJ5101|6 |76 32|THAISZ|NJ5102|8 |89 32|THAISZ|NJ5102|7 |91 32|THAISZ|NJ5102|6 |62 C S#|SNAME |LCODE |C#|GR|CNAME|SITE 25|CLAY |NJ5101|7 |68|CUS |ND 25|CLAY |NJ5101|6 |76|3AU |NJ 32|THAISZ|NJ5102|8 |89|DSDE |ND 32|THAISZ|NJ5102|7 |91|CUS |ND 32|THAISZ|NJ5102|6 |62|3UA |NJ S#|SNAME |LCODE |C#|GR|CNAME|SITE 25|CLAY |NJ5101|7 |68|CUS |ND 25|CLAY |NJ5101|6 |76|3AU |NJ S#|SNAME |LCODE |C#|GR|CNAME|SITE 25|CLAY |NJ5101|7 |68|CUS |ND S#|SNAME |LCODE |C#|GR|CNAME|SITE 25|CLAY |NJ5101|7 |68|CUS |ND SNAME |CNAME|GR CLAY |CUS |68

The OPTIMIZER devises a stategy for executing the query (chooses among alternative Query trees). Is the query tree optimal? Is this tree better? M=PROJ(L)[SNAME,CNAME,GR] | | G=JOIN(F.C#=K.C#) /\ JOIN(H.S#=L.S#)=F \ /\ \ SEL(S.LCODE=NJ5101)=H L=SEL(E.GR=68) K=SEL(C.SITE=ND) S#|SNAME |LCODE 25|CLAY |NJ5101 32|THAISZ|NJ5102 38|GOOD |FL6321 17|BAID |NY2091 57|BROWN |NY2092 C#|CNAME|SITE 8 |DSDE | ND 7 |CUS | ND 6 |3UA | NJ 5 |3UA | ND S#|C#|GR 32|8 |89 32|7 |91 25|7 |68 25|6 |76 32|6 |62 CE S C#|CNAME|SITE 8 |DSDE | ND 7 |CUS | ND 5 |3UA | ND S#|C#|GR 25|7 |68 S#|SNAME |LCODE 25|CLAY |NJ5101 S#|SNAME |LCODE |C#|GR 25|CLAY |NJ5101|7 |68 S#|SNAME |LCODE |C#|GR|CNAME|SITE 25|CLAY |NJ5101|7 |68|CUS |ND SNAME |CNAME|GR CLAY |CUS |68 YES! This tree is better since the intermediate files created are much smaller!!

Notes on the examples Note that the following could be done: i. The SITE attribute can be projected from K (doesn't require elimination of duplicates because it is not part of the key). ii. The LCODE attrib can be projected off of H (doesn't require elimination of duplicates because it is not part of the key). iii. S# could be projected off of F (it is part of the key but duplicate elimination could be deferred until M since it will have to be done again there anyway - thus this projection can be a "non duplicate- eliminating" projection also (which we will denote by [[ ]]). [[ ]]- projections take no time, whereas duplicate eliminating projections take a lot of time). iv. C# can be (non-duplicate-eliminating) projected off of G (note: this projection is reordering attrs and eliminating duplicates, if any)

M=PROJ(L)[SNAME,CNAME,GR] | | G=JOIN(F.C#=K.C#) /\ JOIN(H.S#=L.S#)=F \ /\ \ H=SEL(S.LCODE=NJ5101)[[S#,SNAME]] L=SEL(E.GR=68) K=SEL(C.SITE=ND)[[C#,CNAME]] S#|SNAME |LCODE 25|CLAY |NJ5101 32|THAISZ|NJ5102 38|GOOD |FL6321 17|BAID |NY2091 57|BROWN |NY2092 C#|CNAME|SITE 8 |DSDE | ND 7 |CUS | ND 6 |3UA | NJ 5 |3UA | ND S#|C#|GR 32|8 |89 32|7 |91 25|7 |68 25|6 |76 32|6 |62 CE S C#|CNAME 8 |DSDE 7 |CUS 5 |3UA S#|C#|GR 25|7 |68 S#|SNAME 25|CLAY S#|SNAME |C#|GR 25|CLAY |7 |68 SNAME |GR|CNAME CLAY |68|CUS SNAME |CNAME|GR CLAY |CUS |68 Even better! The intermediate files created are even smaller!!

What have we learned about QP? GOOD RULES? a. Do SELECTS first (push to the bottom of the tree). b. Do attribute elimination part of PROJECT as soon as possible (push down). c. Only do duplicate elimination once (at top-most PROJECT only or in conjunction with a latter join step). QUERY OPTIMIZATION, then, is finding an efficient strategy to implement query requests (Automatically, Heuristically, not necessarily optimally) Note: In lower level languages, the user does the query optimization by writing the procedural code to specify all steps and order those steps. (of course there are optimizing compilers that will automatically alter your "procedures", but still you are mostly responsible for ordering). Relational queries are issued at a high level (SQL or ODBC), so that system has maximal oportunity to optimize them. HEURISTIC RULES are used to re-order query tree. (e.g., RULES a. b. c. above). Some rules depend upon size and complexity estimates. ESTIMATION estimates the cost of different strategies and chooses the best. Challenge: Get acceptable performance (took 10 years to optimize join process acceptably so that the first viable Relational DBMSs could be successfully sold!).

Some SELECT implementations : ( Each of S2 - S6 requires a special access path.) S1. Linear search: sequentially search every record. S2. Binary search: (for selections on a clustered or ordered attribute) S3. Using indexes (or hash structures) for an equality comparison S4. Using primary index for an inequality comparison on a key (clustered). S5. Using a clustering index for "=" comparison S6. Using a secondary B+-tree index for "=", use the index set. SELECTION methods with a WHERE conjunction (AND): S7. Of the many conjunctive attributes, select 1 attribute (usually involving an "=") S8. Intersection of Rrecord Pointers: Intersect RRN-sets then retrieve records S9. If there are Bitmapped Indexes, AND bitmaps CASE-1: SELECT is on an attribute with few distinct values. CASE-2: SELECT is on an attribute with uniqueness (key) or near uniqueness. S10. If there is a composite index on the attributes involved in condition, use it. S11. If there is a composite hash function, use it. SELECTION methods when there is a WHERE disjuntion (OR): S12. If there is no access path (indexes or hash functions), use S1 (brute force). S13. If there are access paths, use them and UNION the results. S14. If there are BitMaps, take the OR of the bitmaps. CODE GENERATION implements the operators above (e.g., SELECT, PROJECT, JOIN...)

S1. Linear search: sequentially search every record. Required for selections from an unordered relation with no index or access path. SELECT C#, GRFROM ENROLL WHERE S# = 32; S#|C#|GR 32|8 |89 32|7 |91 25|7 |68 25|6 |76 32|6 |62 38|6 |98 17|5 |96 ENROLL 32|89 S2. Binary search: For selections on a clustered (ordered) attribute (in this case, S#): SELECT C#, GR FROM ENROLL WHERE S# = 38; 32|91  Go half way (to RRN=3), since S# < 38, go half way down what's left (to RRN= 5). RRN|S#|C#|GR 0 |17|5 |96 1 |25|7 |68 2 |25|6 |76 3 |32|8 |89 4 |32|7 |91 5 |34|6 |62 6 |38|6 |98 ENROLL  Since S# < 38, go half way down what's left (to RRN= 6).  Match! Output. Scan aheadand output until no match or EoF. 32|91

S3. Using Indexes: (or hash structures) for an equality comparison. SELECT C#, NAMEFROM STUDENT WHERE S# = 32 S4. Using primary index for an inequality comparison on a key (clustered). (Find starting point with "=", then retrieve all records beyond that point). SELECT S#,NAME FROM STUDENT WHERE S#  32 RRN|S#|SNAME | LCODE 0 |25|CLAY |NJ5101 1 |32|THAISZ|NJ5102 2 |38|GOOD |FL6321 3 |17|BAID |NY2091 4 |57|BROWN |NY2092 STUDENT RRN| S# Index on S# 0 | 25 4 | 57 3 | 17 1 | 32 2 | 38 Index always clustered on the key (here S#) for binary key search. 32| THAISZ RID| S# nondense Primary Index on S# 3,0| 57 1,0| 17 2,0| 32 Find starting point (first S#  32) then scan ahead taking all until End 32| THAISZ 38| GOOD 57| BROWN RID|S#|SNAME | LCODE 1,0|17|BAID |NY2091 1,1|25|CLAY |NJ5101 2,0|32|THAISZ|NJ5102 2,1|38|GOOD |FL6321 3,0|57|BROWN |NY2092 STUDENT

S5. Using a Clustered Index: for = comparison. SELECT C#, GRFROM ENROLL WHERE S# = 32 RRN|S#|C#|GR 0 |17|5 |96 1 |25|7 |68 2 |25|6 |76 3 |32|8 |89 4 |32|7 |91 5 |32|6 |62 6 |38|6 |98 ENROLL=E RRN| S# Clustering Index on S# 1 | 25 0 | 17 3 | 32 6 | 38 Find first S#=32, then scan E ahead for others. 32| 89 32| 91 32| 62 RRN|S#|SNAME |CITY |ST 0 |57|BROWN |NY |NY 1 |32|THAISZ|KNOB |NJ 2 |17|BAID |NY |NY 3 |38|GOOD |GATER|FL 4 |25|CLAY |OUTBK|NJ 5 |20|JOB |MRHD |MN 6 |56|BURGUM|FARGO|ND 7 |35|BOYD |FLAX |NE STUDENT S6. Using a secondary B+-tree index: For "=", use the index set (assuming a B+tree index) SELECT NAME,CITY FROM STUDENT WHERE S# = 25 *32*38* *20* n n32* n*56* n |17  20|25  32|35  38|  56|57 | 2 5| 4 1| 7 3| 6| 0 CLAY|OUTBK

RRN|S#|SNAME |CITY |ST 0 |57|BROWN |NY |NY 1 |32|THAISZ|KNOB |NJ 2 |17|BAID |NY |NY 3 |38|GOOD |GATER|FL 4 |25|CLAY |OUTBK|NJ 5 |20|JOB |MRHD |MN 6 |56|BURGUM|FARGO|ND 7 |35|BOYD |FLAX |NE STUDENT S6. Using a secondary B+-tree index: For  use the index set, then use sequence set (of B+) SELECT NAME,CITY FROM STUDENT WHERE S#  38 *32*38* *20* n n32* n*56* n |17  20|25  32|35  38|  56|57 | 2 5| 4 1| 7 3| 6| 0 GOOD |GATER BURGUM|FARGO BROWN |NY

S7. Of the many conjunctive attributes, select on 1 attribute (usually 1 involving an "=") then check the other condition(s) for each retrieved record. SELECT NAME, CITYFROM STUDENT WHERE S#>25 and ST=NE RRN| ST Secondary Index on ST 5 | MN 3 | FL 7 | NE 6 | ND 1,4| NJ 0,2| NY *32*38* *20* n n32* n*56* n |17  20|25  32|35  38|  56|57 | 2 5| 4 1| 7 3| 6| 0 RRN|S#|SNAME |CITY |ST 0 |57|BROWN |NY |NY 1 |32|THAISZ|KNOB |NJ 2 |17|BAID |NY |NY 3 |38|GOOD |GATER|FL 4 |25|CLAY |OUTBK|NJ 5 |20|JOB |MRHD |MN 6 |56|BURGUM|FARGO|ND 7 |35|BOYD |FLAX |NE STUDENT BOYD |FLAX

S7. Of the many conjunctive attributes, select on 1 attribute (neither involve =! taking S#) then check the other condition(s) for each retrieved record. SELECT NAME, CITYFROM STUDENT WHERE S#>38 and ST  NE RRN| ST Secondary Index on ST 5 | MN 3 | FL 7 | NE 6 | ND 1,4| NJ 0,2| NY *32*38* *20* n n32* n*56* n |17  20|25  32|35  38|  56|57 | 2 5| 4 1| 7 3| 6| 0 RRN|S#|SNAME |CITY |ST 0 |57|BROWN |NY |NY 1 |32|THAISZ|KNOB |NJ 2 |17|BAID |NY |NY 3 |38|GOOD |GATER|FL 4 |25|CLAY |OUTBK|NJ 5 |20|JOB |MRHD |MN 6 |56|BURGUM|FARGO|ND 7 |35|BOYD |FLAX |NE STUDENT GOOD |GATER BURGUM|FARGO BROWN |NY true

S#-RRN-list ST-RRN-list intersection 1,7,3,6,0 0,2,7 0,7 RRN|S#|SNAME |CITY |ST 0 |57|BROWN |NY |NY 1 |32|THAISZ|KNOB |NJ 2 |17|BAID |NY |NY 3 |38|GOOD |GATER|FL 4 |25|CLAY |OUTBK|NJ 5 |20|JOB |MRHD |MN 6 |56|BURGUM|FARGO|ND 7 |35|BOYD |FLAX |NE STUDENT S8. INTERSECTION OF RECORD POINTERS: Intersect RRN-sets then retrieve records. SELECT NAME,CITY FROM STUDENT WHERE S#>25 and (ST=NE or ST=NY); (This can be done in conjunction with any of the above methods. If the RRN-sets are stored ahead of time for particular selection criterial, then they can greatly speed up the execution. The question is, which should be generated and stored?). S#|bit-filter 17| 00100000 20| 00000100 25| 00001000 32| 01000000 OR here to end (S#>25) result:11010011 35| 00000001 OR NE, NY bitfilters: 10100001 38| 00010000 AND two for result:10000001 56| 00000010 57| 10000000 S9. If Bitmap Indexes BMI on ST ST|bit-filter FL| 00010000 MN| 00001000 NE| 00000001 ND| 00000010 NJ| 01001000 NY| 10100000

S8. INTERSECTION OF RECORD POINTERS: ANDing bitmaps, then retrieve records. SELECT NAME,CITY FROM STUDENT WHERE S#>25 and (ST=NE or ST=NY); BitMapped Indexes (BMIs) are used only for "low cardinality" attributes in DataWarehouses. (those with a small domain - ie, only a few possible values. The reason is that for low-cordinality domains (eg, MONTH, STATE, GENDER, etc.), BMI has few entries (rows) and each bitmap is quite dense (many 1-bits To see why this is so, consider two extremes. CASE-1: For a GENDER attribute in a relation with 80,000 tuples. The BMI looks like: GENDER| bit-filter Female| 0111001010100...1 Male | 1000110101011...0 Eaach bitfilter is 80,000 bits or 10KB so the index is ~20KB with only two distinct values (Note the Male entry is unnecessary since it can be calculated from Female bitfilter as the bit-compliment. Thus, the index is only ~10KB in size altogether. If a regular index were used: GENDER|RID-list Female|RID-F 1, RID-F 2,..., RID-F n Male |RID-M 1, RID-M 2,..., RID-M n Each RID takes 8 bytes (maybe more?) The size is ~640KB. Thus BMI size could be as low as ~10KB and the regular index size ~640KB.

S8. INTERSECTION OF RECORD POINTERS: ANDing bitmaps, then retrieve records. SELECT NAME,CITY FROM STUDENT WHERE S#>25 and (ST=NE or ST=NY); BitMapped Indexes (BMIs) CASE-2: SSN attr of employee file for large company (say, with 80,000 employees) BMI: SSN |bit-filter 324-66-9870|1000000000000...0... 687-99-2536|0000000000000...1  Extant Domain (only those SSN's of existing employees) Each bitfilter 80Kb (10KB) so the index is 80,000 * ~10KB or ~800MB in size. If a regular index were used: SSN |RID 324-66-9870 |RID 1... 687-99-2536 |RID 80000 If RIDs take 8 bytes and SSN+separators take another 12 bytes, the size is ~20*80,000 bits = ~200KB Thus the BMI size could be as low as ~800,000KB and the regular index size would be ~200KB

S10. If there is a composite index on the attrs involved in condition, use it. If there is a composite hash function, use it Selection implementation is matter of choosing among these alternatives (possibly others?). SELECTION methods when there is a WHERE disjuntion (OR) in the condition If there is no access path (indexes or hash fctns), use S1 (brute force). If there are access paths, use them and UNION the results, or UNION the RID-sets, then get the records (rather than interesection as in the case of AND condition). If there are BitMaps, take the OR of BitMaps, then get records

J1. NESTED LOOP R JOIN S on R.A=S.B ( R  R.A=S.B S ) For each record, t in R, (the outer loop over records from the outer or driver relation), retrieve every record, s from S, (the inner loops over the inner relation), test join condition, if it's true, concatenate the tuples (project off unwanted columns) and output, else go to next inner-relation record. R=SELECT SNAME,C#,GRADE FROM S=STUDENT,E=ENROLL WHERE S.S#=E.S#; CODE GENERATION implements the operator, JOIN (equi-join, we will use  for it.) S#|C#|GR 32|8 |89 32|7 |91 25|7 |68 25|6 |76 32|6 |62 E S#|SNAME |LCODE 25|CLAY |NJ5101 32|THAISZ|NJ5102 38|GOOD |FL6321 17|BAID |NY2091 57|BROWN |NY2092 S S.S#=E.S# FALSE S.S#=E.S# FALSE S.S#=E.S# TRUE SNAME |C# |GRADE R CLAY | 7 | 68 S.S#=E.S# TRUE CLAY | 6 | 76 S.S#=E.S# FALSE

J1. NESTED LOOP R JOIN S on R.A=S.B ( R  R.A=S.B S ) Second inner loop pass: R=SELECT SNAME,C#,GRADE FROM S=STUDENT,E=ENROLL WHERE S.S#=E.S#; CODE GENERATION implements the operator, JOIN (equi-join, we will use  for it.) S#|C#|GR 32|8 |89 32|7 |91 25|7 |68 25|6 |76 32|6 |62 E S#|SNAME |LCODE 25|CLAY |NJ5101 32|THAISZ|NJ5102 38|GOOD |FL6321 17|BAID |NY2091 57|BROWN |NY2092 S S.S#=E.S TRUE S.S#=E.S# FALSE S.S#=E.S# FALSE SNAME |C# |GRADE R CLAY | 7 | 68 S.S#=E.S# TRUE CLAY | 6 | 76 THAISZ| 7 | 91 THAISZ| 6 | 62 S.S#=E.S TRUE THAISZ| 8 | 89

J1. NESTED LOOP R JOIN S on R.A=S.B ( R  R.A=S.B S ) Third inner loop pass: R=SELECT SNAME,C#,GRADE FROM S=STUDENT,E=ENROLL WHERE S.S#=E.S#; CODE GENERATION implements the operator, JOIN (equi-join, we will use  for it.) S#|C#|GR 32|8 |89 32|7 |91 25|7 |68 25|6 |76 32|6 |62 E S#|SNAME |LCODE 25|CLAY |NJ5101 32|THAISZ|NJ5102 38|GOOD |FL6321 17|BAID |NY2091 57|BROWN |NY2092 S S.S#=E.S FALSE S.S#=E.S# FALSE S.S#=E.S# FALSE SNAME |C# |GRADE R CLAY | 7 | 68 CLAY | 6 | 76 THAISZ| 7 | 91 THAISZ| 6 | 62 THAISZ| 8 | 89 S.S#=E.S FALSE S.S#=E.S FALSE

J1. NESTED LOOP R JOIN S on R.A=S.B ( R  R.A=S.B S ) 4 th inner loop pass: R=SELECT SNAME,C#,GRADE FROM S=STUDENT,E=ENROLL WHERE S.S#=E.S#; CODE GENERATION implements the operator, JOIN (equi-join, we will use  for it.) S#|C#|GR 32|8 |89 32|7 |91 25|7 |68 25|6 |76 32|6 |62 E S#|SNAME |LCODE 25|CLAY |NJ5101 32|THAISZ|NJ5102 38|GOOD |FL6321 17|BAID |NY2091 57|BROWN |NY2092 S SNAME |C# |GRADE R CLAY | 7 | 68 CLAY | 6 | 76 THAISZ| 7 | 91 THAISZ| 6 | 62 THAISZ| 8 | 89 S.S#=E.S FALSE S.S#=E.S# FALSE S.S#=E.S FALSE S.S#=E.S FALSE S.S#=E.S FALSE

J1. NESTED LOOP R JOIN S on R.A=S.B ( R  R.A=S.B S ) 5 th and last inner loop pass: R=SELECT SNAME,C#,GRADE FROM S=STUDENT,E=ENROLL WHERE S.S#=E.S#; CODE GENERATION implements the operator, JOIN (equi-join, we will use  for it.) S#|C#|GR 32|8 |89 32|7 |91 25|7 |68 25|6 |76 32|6 |62 E S#|SNAME |LCODE 25|CLAY |NJ5101 32|THAISZ|NJ5102 38|GOOD |FL6321 17|BAID |NY2091 57|BROWN |NY2092 S SNAME |C# |GRADE R CLAY | 7 | 68 CLAY | 6 | 76 THAISZ| 7 | 91 THAISZ| 6 | 62 THAISZ| 8 | 89 S.S#=E.S FALSE S.S#=E.S# FALSE S.S#=E.S FALSE S.S#=E.S FALSE S.S#=E.S FALSE

J2. When there is an Index on one join attribute the join can be done in one pass (called Indexed Nested Loop. If there is an index on E.S#, get r in S, get matching E-tuples using the index (need not scan entire inner relation, E, each time as was necessary with J1). R=SELECT SNAME,C#,GRADE FROM S=STUDENT,E=ENROLL WHERE S.S#=E.S#; CODE GENERATION implements the operator, JOIN (equi-join, we will use  for it.) S#|SNAME |LCODE 25|CLAY |NJ5101 32|THAISZ|NJ5102 38|GOOD |FL6321 17|BAID |NY2091 57|BROWN |NY2092 RRN|S#|C#|GR 0 |32|8 |89 1 |32|7 |91 2 |25|7 |68 3 |25|6 |76 4 |32|6 |62 E S RRN |S# 2,3 |25 0,1,4|32 Dense Index on E.S# SNAME |C# |GRADE R CLAY | 7 | 68 CLAY | 6 | 76 THAISZ| 7 | 91 THAISZ| 6 | 62 THAISZ| 8 | 89

J3. MERGE JOIN: If both S.S# and E.S# are clustered, then scan both S and E once in order, keeping in mind that S.S# is the primary key (uniqueness property), but E.S# is not. R=SELECT SNAME,C#,GRADE FROM S=STUDENT,E=ENROLL WHERE S.S#=E.S#; CODE GENERATION implements the operator, JOIN (equi-join, we will use  for it.) S#|SNAME |LCODE 17|BAID |NY2091 25|CLAY |NJ5101 32|THAISZ|NJ5102 38|GOOD |FL6321 57|BROWN |NY2092 S#|C#|GR 25|7 |68 25|6 |76 32|6 |62 32|8 |89 32|7 |91 E S S.S#=E.S# FALSE SNAME |C# |GRADE R S.S#=E.S# TRUE CLAY | 7 | 68 S.S#=E.S# TRUE CLAY | 6 | 76 S.S#=E.S# FALSE S.S#=E.S# TRUE THAISZ| 6 | 62 S.S#=E.S# TRUE THAISZ| 8 | 89 S.S#=E.S# TRUE THAISZ| 7 | 91 S.S#=E.S# FALSE S.S#=E.S# FALSE J3'. SORT-MERGE JOIN: If R.A and S.B are not ordered, sort them first (into R' clustered on A and S' clustered on B), then apply MERGE (J2 above).

J4. HASH-JOIN: RIDs hashed to buckets (pages). Corresponding buckets retrieved and scanned GRACE JOIN: (first example of a hash-join technique): Allocate M pages of memory to the join process. Partition the M pages as follows: One page for putting new pages as they are read from disk (called INPUT), B+1 for hash buckets R0,..,RB ( therefore B = M-2 ). Use hash function, h with range, {0,...,B} (e.g., MOD(B) if A is numeric). Partial Sort Phase: Partial-Sort-R: Read each R-page to IN, hash each record using h(A) to R0,...RB. Upon collision in any of the buckets (pages), R0..RB, flush it to temporary disk file (also called R0,...RB). Partial-Sort-S: Read each S-page into IN, hash each record with h(A) to R0..RB Upon collision in any bucket, R0..RB, flush its' contents to temp disk file, S0..SB. Build Phase: With each pair of temporary files, R0 & S0, R1 & R2, R2 & S2,... in turn, do as follows: Re-partition memory into IN, OUT and one large hash area. For Ri, BUILD internal a hash table in the hash area using another hash function, k(a) FOR Si, PROBE hash table using k(a) for matches, output join of matches to OUT. CODE GENERATION implements the operator, JOIN

2,0|25|6 |76|NY2091 J4. HASH-JOIN: RIDs hashed to buckets (pages). Matching buckets are retrieved and scanned. R=SELECT SNAME,C#,GRADE FROM S=STUDENT,E=ENROLL WHERE S.S#=E.S#; GRACE JOIN: (first example of a hash-join technique): Allocate M=4 pages of memory to the join process. Partition the M pages as follows: - One page for INPUT (putting new pages as they are read from disk), M-1=3 pages for buckets R 0,..,R 2. Build Phase. Use hash function, h=MOD 3 (assuming join key is numeric, if not, map to numeric first.) CODE GENERATION implements the operator, JOIN S RID|S#|SNAME | LCODE 1,0|17|BAID |NY2091 3,0|38|GOOD |FL6321 1,1|25|CLAY |NJ5101 2,0|32|THAISZ|NJ5102 2,1|57|BROWN |NY2092 E RID|S#|C#|GR| LCODE 1,0|17|5 |96|NJ5101 1,1|25|7 |68|ND4456 2,1|32|8 |89|NY2091 3,0|32|7 |91|FL6320 3,1|34|6 |62|ND4456 R0R0 R1R1 R2R2 IN Partial-Sort-S: Read each S-page to IN, hash each record using h(S#) to R 0,R 1,R 2. Upon collision in any of the buckets, flush to temporary disk file, called S 0,S 1,S 2. S0S0 S1S1 S2S2 1,0|17|BAID |NY2091 1,1|25|CLAY |NJ5101 2,0|32|THAISZ|NJ5102 2,1|57|BROWN |NY2092 3,0|38|GOOD |FL6321 Collision! Dump R 2 Then flush all. 17|BAID |NY2091 25|CLAY |NJ5101 32|THAISZ|NJ5102 57|BROWN |NY2092 38|GOOD |FL6321 Do the same with E. E0E0 E1E1 E2E2 25|6 |76|NY2091 17|5 |96|NJ5101 25|7 |68|ND4456 32|8 |89|NY2091 32|7 |91|FL6320 34|6 |62|ND4456

CODE GENERATION implements the operator, GRACE JOIN S0S0 S1S1 S2S2 Build Phase: With each pair of temporary files, S 0 & E 0, S 1 & E 2, S 2 & E 2,... in turn, do as follows: Re-partition memory into IN, OUT and one large hash area. Probe Phase: For S i, BUILD internal a hash table in the hash area using another hash function, k(S#)=MOD 4 (open addr for collisions) FOR E i, PROBE hash table using k for matches, output join of matches to OUT. Start with S 0 and E 0. But E 0 empty (no output will be produced) so skip. PROBE S 1 and E 1 : 1. Read S 1 page-1 to IN. 2. Hash IN to HASH. 17|BAID |NY2091 25|CLAY |NJ5101 32|THAISZ|NJ5102 57|BROWN |NY2092 38|GOOD |FL6321 E0E0 E1E1 E2E2 25|6 |76|NY2091 17|5 |96|NJ5101 25|7 |68|ND4456 32|8 |89|NY2091 32|7 |91|FL6320 34|6 |62|ND4456 25|CLAY |NJ5101 0 OUT IN 1 2 3 Hash

25|7 |68|ND4456 CODE GENERATION implements GRACE JOIN S0S0 S1S1 S2S2 PROBE S 1, E 1 : 1. Read S 1 page-1 to IN. 2. Hash IN to HASH. 3. Read E 1 to IN 4. Concatenate to OUT. 17|BAID |NY2091 25|CLAY |NJ5101 32|THAISZ|NJ5102 57|BROWN |NY2092 38|GOOD |FL6321 E0E0 E1E1 E2E2 25|6 |76|NY2091 17|5 |96|NJ5101 25|7 |68|ND4456 32|8 |89|NY2091 32|7 |91|FL6320 34|6 |62|ND4456 25|CLAY |NJ5101 CLAY|7 |68 25|6 |76|NY2091 CLAY|6 |76 0 OUT IN 1 2 3 Hash

CODE GENERATION implements GRACE JOIN S0S0 S1S1 S2S2 PROBE S 1, E 1 : 1. Read S 1 pg1 to IN. 2. Hash IN to HASH. 3. Read E 1 to IN 4. Concatenate to OUT. 5. Read (next page of) E 1 to IN. 6. Hash to HASH. 7. If match, Concatenate to OUT (Since OUT is full, flush OUT first.). But no match! 8. Flush HASH and IN when done with S 1, E 1 Probe (before starting Probe S 2, E 2 ). 17|BAID |NY2091 25|CLAY |NJ5101 32|THAISZ|NJ5102 57|BROWN |NY2092 38|GOOD |FL6321 E0E0 E1E1 E2E2 25|6 |76|NY2091 17|5 |96|NJ5101 25|7 |68|ND4456 32|8 |89|NY2091 32|7 |91|FL6320 34|6 |62|ND4456 25|CLAY |NJ5101 CLAY|7 |68 CLAY|6 |76 0 OUT IN 1 2 3 Hash 34|6 |62|ND4456

CODE GENERATION implements GRACE JOIN S0S0 S1S1 S2S2 PROBE S 1, E 1 : 1. Read S 1 pg1 to IN. 2. Hash IN to HASH. 3. Read E 1 to IN 4. Concatenate to OUT. 5. Read (next page of) E 1 to IN. 6. Hash to HASH. 7. If match, Concatenate to OUT... 8. Flush HASH and IN when done Probing E 1. 17|BAID |NY2091 25|CLAY |NJ5101 32|THAISZ|NJ5102 57|BROWN |NY2092 38|GOOD |FL6321 E0E0 E1E1 E2E2 25|6 |76|NY2091 17|5 |96|NJ5101 25|7 |68|ND4456 32|8 |89|NY2091 32|7 |91|FL6320 34|6 |62|ND4456 CLAY|7 |68 CLAY|6 |76 0 OUT IN 1 2 3 Hash PROBE S 2, E 2 : 1. Read S 2 pg1 to IN 2. MOD 3 hash IN to HASH (open addressing for collisions). 17|BAID |NY2091 32|THAISZ|NJ5102

CODE GENERATION implements GRACE JOIN S0S0 S1S1 S2S2 PROBE S 1, E 1 : 1. Read S 1 pg1 to IN. 2. Hash IN to HASH. 3. Read E 1 to IN 4. Concatenate to OUT. 5. Read (next page of) E 1 to IN. 6. Hash to HASH. 7. If match, Concatenate to OUT... 8. Flush HASH and IN when done Probing E 1. 17|BAID |NY2091 25|CLAY |NJ5101 32|THAISZ|NJ5102 57|BROWN |NY2092 38|GOOD |FL6321 E0E0 E1E1 E2E2 25|6 |76|NY2091 17|5 |96|NJ5101 25|7 |68|ND4456 32|8 |89|NY2091 32|7 |91|FL6320 34|6 |62|ND4456 CLAY|7 |68 CLAY|6 |76 0 OUT IN 1 2 3 Hash PROBE S 2, E 2 : 1. Read S 2 pg1 to IN 2. MOD 3 hash IN to HASH (open addressing for collisions). 3. Read S2 pg2 to IN 4. MOD3 hash IN to HASH (open addressing for collisions). 17|BAID |NY2091 32|THAISZ|NJ5102 38|GOOD |FL6321

CODE GENERATION implements GRACE JOIN S0S0 S1S1 S2S2 PROBE S 1, E 1 : 1. Read S 1 pg1 to IN. 2. Hash IN to HASH. 3. Read E 1 to IN 4. Concatenate to OUT. 5. Read (next page of) E 1 to IN. 6. Hash to HASH. 7. If match, Concatenate to OUT... 8. Flush HASH and IN when done Probing E 1. 17|BAID |NY2091 25|CLAY |NJ5101 32|THAISZ|NJ5102 57|BROWN |NY2092 38|GOOD |FL6321 E0E0 E1E1 E2E2 25|6 |76|NY2091 17|5 |96|NJ5101 25|7 |68|ND4456 32|8 |89|NY2091 32|7 |91|FL6320 34|6 |62|ND4456 CLAY|7 |68 CLAY|6 |76 0 OUT IN 1 2 3 Hash PROBE S 2, E 2 : 1. Read S 2 pg1 to IN 2. MOD 3 hash IN to HASH (open addressing for collisions). 3. Read S 2 pg2 to IN 4. MOD 3 hash IN to HASH (open addressing for collisions). 5.Read E 2 to IN. 6.Hash to HASH. 7. If match, Concatenate to OUT... 17|BAID |NY2091 32|THAISZ|NJ5102 38|GOOD |FL6321 17|5 |96|NJ5101 32|8 |89|NY2091 BAID |5|96

CODE GENERATION implements GRACE JOIN S0S0 S1S1 S2S2 PROBE S 1, E 1 : 1. Read S 1 pg1 to IN. 2. Hash IN to HASH. 3. Read E 1 to IN 4. Concatenate to OUT. 5. Read (next page of) E 1 to IN. 6. Hash to HASH. 7. If match, Concatenate to OUT... 8. Flush HASH and IN when done Probing E 1. 17|BAID |NY2091 25|CLAY |NJ5101 32|THAISZ|NJ5102 57|BROWN |NY2092 38|GOOD |FL6321 E0E0 E1E1 E2E2 25|6 |76|NY2091 17|5 |96|NJ5101 25|7 |68|ND4456 32|8 |89|NY2091 32|7 |91|FL6320 34|6 |62|ND4456 CLAY|7 |68 CLAY|6 |76 0 OUT IN 1 2 3 Hash PROBE S 2, E 2 : 1. Read S 2 pg1 to IN 2. MOD 3 hash IN to HASH (open addressing for collisions). 3. Read S 2 pg2 to IN 4. MOD 3 hash IN to HASH (open addressing for collisions). 5.Read E 2 to IN. 6.Hash to HASH. 7. If match, Concatenate to OUT... 17|BAID |NY2091 32|THAISZ|NJ5102 38|GOOD |FL6321 32|8 |89|NY2091 BAID |5|96 THAISZ|8|89

CODE GENERATION implements GRACE JOIN S0S0 S1S1 S2S2 PROBE S 1, E 1 : 1. Read S 1 pg1 to IN. 2. Hash IN to HASH. 3. Read E 1 to IN 4. Concatenate to OUT. 5. Read (next page of) E 1 to IN. 6. Hash to HASH. 7. If match, Concatenate to OUT... 8. Flush HASH and IN when done Probing E 1. 17|BAID |NY2091 25|CLAY |NJ5101 32|THAISZ|NJ5102 57|BROWN |NY2092 38|GOOD |FL6321 E0E0 E1E1 E2E2 25|6 |76|NY2091 17|5 |96|NJ5101 25|7 |68|ND4456 32|8 |89|NY2091 32|7 |91|FL6320 34|6 |62|ND4456 CLAY |7|68 CLAY |6|76 0 OUT IN 1 2 3 Hash PROBE S 2, E 2 : 1. Read S 2 pg1 to IN 2. MOD 3 hash IN to HASH (open addressing for collisions). 3. Read S 2 pg2 to IN 4. MOD 3 hash IN to HASH (open addressing for collisions). 5.Read E 2 to IN. 6.Hash to HASH. 7. If match, Concatenate to OUT...(repeat until E2 empty) 8. Flush HASH and IN when done Probing E 1. 17|BAID |NY2091 32|THAISZ|NJ5102 38|GOOD |FL6321 32|7 |91|FL6320 BAID |5|96 THAISZ|8|89 THAISZ|7|91

J4. HASH-JOIN: (a better way than GRACE JOIN): HYBRID HASH JOIN of S  S# E (developed by former chair of NDSU CS, Dr. L. Shapiro): Partition the M pages of main memory allocated to the join process as: One page for the INPUT buffer, One page for the OUTPUT buffer, B pages for hash buckets, R 1..R B, Leave the rest for a large hash bucket, R 0. BUILD PHASE, BUILD S: Read each S page to IN, hash each record using h=MOD B to R 0..R B. If a record hashes to R 0, apply an internal hash function, k (which hashes the record to a slot in R 0. Use open addressing for k-collisions), copy the record to the k(S#) slot of R 0. When a h-collision occurs in any page R i i=1..B, flush that page to a disk file, called S i i=1..B. BUILD E: Read each E page into IN, hash each record with h to R 0..R B If record hashes to R 0, apply internal hash function k, and concatenate record with all matches found to OUT. If Collision occurs in any page, R i i=1..B, flush to temporary disk file, E i i=1..B. PROBE PHASE for pairs, S i and E i i=1..B, is the same as in Grace Join. Hybrid Hash Join can be done with Bit Filtering to eliminate non-participating tuples early and avoid wasted processing of non-participating tuples Much more detail and example walkthroughs can be found in the HTML version of these notes (also available from "Other Materials" http://www.cs.ndsu.nodak.edu/~perrizo/classes/765/09query.htmlhttp://www.cs.ndsu.nodak.edu/~perrizo/classes/765/09query.html CODE GENERATION implements the operator, HYBRID HASH JOIN

2,0|25|6 |76|NY2091 h=MOD 3 k=MOD 4 SELECT SNAME,C#,GRADE FROM S,E WHERE S.S#=E.S#; CODE GENERATION implements the operator, HYBRID HASH JOIN S RID|S#|SNAME | LCODE 1,0|17|BAID |NY2091 3,0|38|GOOD |FL6321 1,1|25|CLAY |NJ5101 2,0|32|THAISZ|NJ5102 2,1|57|BROWN |NY2092 E RID|S#|C#|GR| LCODE 1,0|17|5 |96|NJ5101 1,1|25|7 |68|ND4456 2,1|32|8 |89|NY2091 3,0|32|7 |91|FL6320 3,1|34|6 |62|ND4456 R1R1 R2R2 IN BUILD S: Read each S-page to IN, hash each record using h(S#)=MOD 3 (S#) to R 0,R 1,R 2. If h(S#)=0, k(S#) determines R 0 slot (open addressing for collisions in R 0 ). h collisions in R 1, R 2 flush to file, S 1, S 2 resp. BUILD E: similarly (note all h hashes goes to R 1 or R 2, So all E records flush to E 1 and E 2 and R 0 is flushed too. It would have been more efficient to choose. e.g., R 2 as the internal hash bucket! (done later!) S0S0 S1S1 S2S2 1,0|17|BAID |NY2091 1,1|25|CLAY |NJ5101 2,0|32|THAISZ|NJ5102 2,1|57|BROWN |NY2092 3,0|38|GOOD |FL6321 17|BAID |NY2091 25|CLAY |NJ5101 32|THAISZ|NJ5102 57|BROWN |NY2092 38|GOOD |FL6321 E0E0 E1E1 E2E2 25|6 |76|NY2091 17|5 |96|NJ5101 25|7 |68|ND4456 32|8 |89|NY2091 32|7 |91|FL6320 34|6 |62|ND4456 R0R0 0 1 2 3 OUT

25|7 |68|ND4456 CODE GENERATION implements HYBRID JOIN S0S0 S1S1 S2S2 PROBE S 1, E 1 : 1. Read S 1 page-1 to IN. 2. Hash IN to HASH. 3. Read E 1 to IN 4. Concatenate to OUT. 17|BAID |NY2091 25|CLAY |NJ5101 32|THAISZ|NJ5102 38|GOOD |FL6321 E0E0 E1E1 E2E2 25|6 |76|NY2091 17|5 |96|NJ5101 25|7 |68|ND4456 32|8 |89|NY2091 32|7 |91|FL6320 34|6 |62|ND4456 25|CLAY |NJ5101 CLAY|7 |68 25|6 |76|NY2091 CLAY|6 |76 0 OUT IN 1 2 3 Hash

CODE GENERATION implements HYBRID JOIN S0S0 S1S1 S2S2 PROBE S 1, E 1 : 1. Read S 1 pg1 to IN. 2. Hash IN to HASH. 3. Read E 1 to IN 4. Concatenate to OUT. 5. Read (next page of) E 1 to IN. 6. Hash to HASH. 7. If match, Concatenate to OUT (Since OUT is full, flush OUT first.). But no match! 8. Flush HASH and IN when done with S 1, E 1 Probe (before starting Probe S 2, E 2 ). 17|BAID |NY2091 25|CLAY |NJ5101 32|THAISZ|NJ5102 38|GOOD |FL6321 E0E0 E1E1 E2E2 25|6 |76|NY2091 17|5 |96|NJ5101 25|7 |68|ND4456 32|8 |89|NY2091 32|7 |91|FL6320 34|6 |62|ND4456 25|CLAY |NJ5101 CLAY|7 |68 CLAY|6 |76 0 OUT IN 1 2 3 Hash 34|6 |62|ND4456

CODE GENERATION implements HYBRID JOIN S0S0 S1S1 S2S2 PROBE S 1, E 1 : 1. Read S 1 pg1 to IN. 2. Hash IN to HASH. 3. Read E 1 to IN 4. Concatenate to OUT. 5. Read (next page of) E 1 to IN. 6. Hash to HASH. 7. If match, Concatenate to OUT... 8. Flush HASH and IN when done Probing E 1. 17|BAID |NY2091 25|CLAY |NJ5101 32|THAISZ|NJ5102 38|GOOD |FL6321 E0E0 E1E1 E2E2 25|6 |76|NY2091 17|5 |96|NJ5101 25|7 |68|ND4456 32|8 |89|NY2091 32|7 |91|FL6320 34|6 |62|ND4456 CLAY|7 |68 CLAY|6 |76 0 OUT IN 1 2 3 Hash PROBE S 2, E 2 : 1. Read S 2 pg1 to IN 2. MOD 3 hash IN to HASH (open addressing for collisions). 17|BAID |NY2091 32|THAISZ|NJ5102

CODE GENERATION implements HYBRID JOIN S0S0 S1S1 S2S2 PROBE S 1, E 1 : 1. Read S 1 pg1 to IN. 2. Hash IN to HASH. 3. Read E 1 to IN 4. Concatenate to OUT. 5. Read (next page of) E 1 to IN. 6. Hash to HASH. 7. If match, Concatenate to OUT... 8. Flush HASH and IN when done Probing E 1. 17|BAID |NY2091 25|CLAY |NJ5101 32|THAISZ|NJ5102 38|GOOD |FL6321 E0E0 E1E1 E2E2 25|6 |76|NY2091 17|5 |96|NJ5101 25|7 |68|ND4456 32|8 |89|NY2091 32|7 |91|FL6320 34|6 |62|ND4456 CLAY|7 |68 CLAY|6 |76 0 OUT IN 1 2 3 Hash PROBE S 2, E 2 : 1. Read S 2 pg1 to IN 2. MOD 3 hash IN to HASH (open addressing for collisions). 3. Read S2 pg2 to IN 4. MOD3 hash IN to HASH (open addressing for collisions). 17|BAID |NY2091 32|THAISZ|NJ5102 38|GOOD |FL6321

CODE GENERATION implements HYBRID JOIN S0S0 S1S1 S2S2 PROBE S 1, E 1 : 1. Read S 1 pg1 to IN. 2. Hash IN to HASH. 3. Read E 1 to IN 4. Concatenate to OUT. 5. Read (next page of) E 1 to IN. 6. Hash to HASH. 7. If match, Concatenate to OUT... 8. Flush HASH and IN when done Probing E 1. 17|BAID |NY2091 25|CLAY |NJ5101 32|THAISZ|NJ5102 38|GOOD |FL6321 E0E0 E1E1 E2E2 25|6 |76|NY2091 17|5 |96|NJ5101 25|7 |68|ND4456 32|8 |89|NY2091 32|7 |91|FL6320 34|6 |62|ND4456 CLAY|7 |68 CLAY|6 |76 0 OUT IN 1 2 3 Hash PROBE S 2, E 2 : 1. Read S 2 pg1 to IN 2. MOD 3 hash IN to HASH (open addressing for collisions). 3. Read S 2 pg2 to IN 4. MOD 3 hash IN to HASH (open addressing for collisions). 5.Read E 2 to IN. 6.Hash to HASH. 7. If match, Concatenate to OUT... 17|BAID |NY2091 32|THAISZ|NJ5102 38|GOOD |FL6321 17|5 |96|NJ5101 32|8 |89|NY2091 BAID |5|96

CODE GENERATION implements HYBRID JOIN S0S0 S1S1 S2S2 PROBE S 1, E 1 : 1. Read S 1 pg1 to IN. 2. Hash IN to HASH. 3. Read E 1 to IN 4. Concatenate to OUT. 5. Read (next page of) E 1 to IN. 6. Hash to HASH. 7. If match, Concatenate to OUT... 8. Flush HASH and IN when done Probing E 1. 17|BAID |NY2091 25|CLAY |NJ5101 32|THAISZ|NJ5102 38|GOOD |FL6321 E0E0 E1E1 E2E2 25|6 |76|NY2091 17|5 |96|NJ5101 25|7 |68|ND4456 32|8 |89|NY2091 32|7 |91|FL6320 34|6 |62|ND4456 CLAY|7 |68 CLAY|6 |76 0 OUT IN 1 2 3 Hash PROBE S 2, E 2 : 1. Read S 2 pg1 to IN 2. MOD 3 hash IN to HASH (open addressing for collisions). 3. Read S 2 pg2 to IN 4. MOD 3 hash IN to HASH (open addressing for collisions). 5.Read E 2 to IN. 6.Hash to HASH. 7. If match, Concatenate to OUT... 17|BAID |NY2091 32|THAISZ|NJ5102 38|GOOD |FL6321 32|8 |89|NY2091 BAID |5|96 THAISZ|8|89

CODE GENERATION implements HYBRID JOIN S0S0 S1S1 S2S2 PROBE S 1, E 1 : 1. Read S 1 pg1 to IN. 2. Hash IN to HASH. 3. Read E 1 to IN 4. Concatenate to OUT. 5. Read (next page of) E 1 to IN. 6. Hash to HASH. 7. If match, Concatenate to OUT... 8. Flush HASH and IN when done Probing E 1. 17|BAID |NY2091 25|CLAY |NJ5101 32|THAISZ|NJ5102 38|GOOD |FL6321 E0E0 E1E1 E2E2 25|6 |76|NY2091 17|5 |96|NJ5101 25|7 |68|ND4456 32|8 |89|NY2091 32|7 |91|FL6320 34|6 |62|ND4456 CLAY |7|68 CLAY |6|76 0 OUT IN 1 2 3 Hash PROBE S 2, E 2 : 1. Read S 2 pg1 to IN 2. MOD 3 hash IN to HASH (open addressing for collisions). 3. Read S 2 pg2 to IN 4. MOD 3 hash IN to HASH (open addressing for collisions). 5.Read E 2 to IN. 6.Hash to HASH. 7. If match, Concatenate to OUT...(repeat until E2 empty) 8. Flush HASH and IN when done Probing E 1. 17|BAID |NY2091 32|THAISZ|NJ5102 38|GOOD |FL6321 32|7 |91|FL6320 BAID |5|96 THAISZ|8|89 THAISZ|7|91

2,0|32|THAISZ|NJ5102 1,0|17|BAID |NY2091 2,1|57|BROWN |NY2092 1,1|25|CLAY |NJ5101 3,0|38|GOOD |FL6321 2,0|25|6 |76|NY2091 h=MOD 3 k=MOD 4 SELECT SNAME,C#,GRADE FROM S,E WHERE S.S#=E.S#; CODE generation of HYBRID HASH JOIN using R 2 as internal hash bucket! S RID|S#|SNAME | LCODE 1,0|17|BAID |NY2091 3,0|38|GOOD |FL6321 1,1|25|CLAY |NJ5101 2,0|32|THAISZ|NJ5102 2,1|57|BROWN |NY2092 E RID|S#|C#|GR| LCODE 1,0|17|5 |96|NJ5101 1,1|25|7 |68|ND4456 2,1|32|8 |89|NY2091 3,0|32|7 |91|FL6320 3,1|34|6 |62|ND4456 IN BUILD S: Read each S-page to IN, hash each record using h(S#)=MOD 3 (S#) to R 0,R 1,R 2. If h(S#)=2, k(S#) determines R 2 slot (open addressing for R 2 ). h collisions in R 1, R 0 flush to file, S 1, S 0 resp. S0S0 25|CLAY |NJ5101 57|BROWN |NY2092 E0E0 R1R1 R0R0 R2R2 0 1 2 3 OUT E1E1 S1S1

17|BAID |NY2091 32|THAISZ|NJ510 2 38|GOOD |FL6321 2,0|25|6 |76|NY2091 h=MOD 3 k=MOD 4 SELECT SNAME,C#,GRADE FROM S,E WHERE S.S#=E.S#; CODE generation implements HYBRID HASH JOIN using R 2 as internal hash bucket! S RID|S#|SNAME | LCODE 1,0|17|BAID |NY2091 3,0|38|GOOD |FL6321 1,1|25|CLAY |NJ5101 2,0|32|THAISZ|NJ5102 2,1|57|BROWN |NY2092 E RID|S#|C#|GR| LCODE 1,0|17|5 |96|NJ5101 1,1|25|7 |68|ND4456 2,1|32|8 |89|NY2091 3,0|32|7 |91|FL6320 3,1|34|6 |62|ND4456 IN BUILD E: Read first E-page to IN, hash each record using h(S#)=MOD 3 (S#) to R 0,R 1,R 2. If h(S#)=2, k(S#) determines R 2 slot (open addressing for R 2 ). If match, concatenate to OUT. h collisions in R 1, R 0 flush to file, E 1, E 0 respectively. S0S0 S1S1 S2S2 25|CLAY |NJ5101 57|BROWN |NY2092 E0E0 R1R1 R0R0 R2R2 0 1 2 3 OUT E1E1 S1S1 1,0|17|5 |96|NJ5101 1,1|25|7 |68|ND4456 BAID |5 |96

17|BAID |NY2091 32|THAISZ|NJ510 2 38|GOOD |FL6321 2,0|25|6 |76|NY2091 h=MOD 3 k=MOD 4 SELECT SNAME,C#,GRADE FROM S,E WHERE S.S#=E.S#; S RID|S#|SNAME | LCODE 1,0|17|BAID |NY2091 3,0|38|GOOD |FL6321 1,1|25|CLAY |NJ5101 2,0|32|THAISZ|NJ5102 2,1|57|BROWN |NY2092 E RID|S#|C#|GR| LCODE 1,0|17|5 |96|NJ5101 1,1|25|7 |68|ND4456 2,1|32|8 |89|NY2091 3,0|32|7 |91|FL6320 3,1|34|6 |62|ND4456 IN BUILD E: Read second E-page to IN, hash each record using h(S#)=MOD 3 (S#) to R 0,R 1,R 2. If h(S#)=2, k(S#) determines R 2 slot (open addressing for R 2 ). If match, concatenate to OUT. h collisions in R 1, R 0 flush to file, E 1, E 0 respectively. S0S0 S1S1 S2S2 25|CLAY |NJ5101 57|BROWN |NY2092 E0E0 R1R1 R0R0 R2R2 0 1 2 3 OUT E1E1 S1S1 25|7 |68|ND4456 BAID |5 |96 2,0|25|6 |76|NY2091 2,1|32|8 |89|NY2091 THAISZ|8 |89 CODE GEN implements HYBRID HASH JOIN using R 2 as internal hash bucket!

17|BAID |NY2091 32|THAISZ|NJ510 2 38|GOOD |FL6321 2,0|25|6 |76|NY2091 h=MOD 3 k=MOD 4 SELECT SNAME,C#,GRADE FROM S,E WHERE S.S#=E.S#; S RID|S#|SNAME | LCODE 1,0|17|BAID |NY2091 3,0|38|GOOD |FL6321 1,1|25|CLAY |NJ5101 2,0|32|THAISZ|NJ5102 2,1|57|BROWN |NY2092 E RID|S#|C#|GR| LCODE 1,0|17|5 |96|NJ5101 1,1|25|7 |68|ND4456 2,1|32|8 |89|NY2091 3,0|32|7 |91|FL6320 3,1|34|6 |62|ND4456 IN BUILD E: Read third E-page to IN, hash each record using h(S#)=MOD 3 (S#) to R 0,R 1,R 2. If h(S#)=2, k(S#) determines R 2 slot (open addressing for R 2 ). If match, concatenate to OUT. h collisions in R 1, R 0 flush to file, E 1, E 0 respectively. When done building E, flush R 2. S0S0 S1S1 S2S2 25|CLAY |NJ5101 57|BROWN |NY2092 E0E0 R1R1 R0R0 R2R2 0 1 2 3 OUT E1E1 S1S1 25|7 |68|ND4456 BAID |5 |96 25|6 |76|NY2091 THAISZ|8 |89 3,0|32|7 |91|FL6320 3,1|34|6 |62|ND4456 THAISZ|7 |91 25|7 |68|ND4456 25|6 |76|NY2091 34|6 |62|ND4456 CODE GEN implements HYBRID HASH JOIN using R 2 as internal hash bucket!

Note: If memory allocation is static, use all Ri pages for internal hash function! So k=MOD 8 CODE GEN implements HYBRID HASH JOIN using R2 as internal hash bucket probe: PROBE S: S0S0 S1S1 S2S2 25|CLAY |NJ5101 57|BROWN |NY2092 E0E0 E1E1 S1S1 THAISZ|7 |91 25|7 |68|ND4456 25|6 |76|NY2091 34|6 |62|ND4456 IN R2R2 0 1 2 3 OUT 4 5 6 7 25|CLAY |NJ5101 25|7 |68|ND4456 25|6 |76|NY2091 CLAY |7 |68 BAID |5 |96 THAISZ|8 |89 THAISZ|6 |76 h=MOD 3 k=MOD 4 SELECT SNAME,C#,GRADE FROM S,E WHERE S.S#=E.S#; Notice how much more efficient the probe phase of HH JOIN is than Grace JOIN when the internal Hash table is chosen to apply to the right bucket! (And how it may not be faster, if that decision is badly made!)

Projection is removal of certian specified attributes (columns) from a relation. Given a relation, R(A,B,C,D,E), the projection of R onto A,B,D written, PROJ R [A,B,D], is done by removing columns C and E and then eliminating any duplicated tuples from the result. {A,B,D} is called the projection attribute-list. Note: many system provide a more flexible projection (in which duplicates are not removed). Strictly speaking, this is not a relational operator since the result is not a relation (the result is what mathematicians term, a "bag"). If the attribute-list contains a key, then there are no duplicates to be removed (why not?). In this case, the projection implementation code gets each tuple in turn, trims off the unspecified attributes and outputs the result. If list does not contain a key, sort (or hash) and then get a record, trim off the non-attribute-list attributes and eliminate duplicates. Note: this can be expensive (about as expensive as a join). Projection codes (with duplicate elimination) are similar to join codes. e.g., methods include: Nested loop: For each tuple, scan the projection for duplicates. Since there is a physical order to the tuples (even though it may not be any particular logical ordering) consider the tuples in that order and then scan from that tuple only (not from the beginning of the file). Indexed nested loop: For each tuple, consult index for duplicates and remove them. Sort-remove (like sort-merge), Sort result, scan once for duplicates (now situated adjacent to each other) and remove them. Hash methods. Partially sort projection by hashing (similar to grace join??) then use reduced nested loop on each subset (one at a time) to discard duplicates. CODE GENERATION implementing the operator, PROJECTION

METHODS for fast SPJ processing: MATERIALIZED VIEWS (MV), DOMAIN VECTORS (DV) JOIN INDICES (JI), others... MATERIALIZED VIEW method is just a matter of precomputing the query result and storing it for the next request of that query (so that it does not have to be recomputed each time). - this may work well if the result is not too large and if the underlying base relations from which the view is generated are quite "static" (changed very seldom). CODE GENERATION implements the operator, SELECT-PROJECT-JOIN (SPJ)

METHODS for fast SPJ processing: MATERIALIZED VIEWS (MV), DOMAIN VECTORS (DV) JOIN INDICES (JI), others... MATERIALIZED VIEW method is just a matter of precomputing the query result and storing it for the next request of that query (so that it does not have to be recomputed each time). - this may work well if the result is not too large and if the underlying base relations from which the view is generated are quite "static" (changed very seldom). See "Other Materials" http://www.cs.ndsu.nodak.edu/~perrizo/classes/765/09query.htmlhttp://www.cs.ndsu.nodak.edu/~perrizo/classes/765/09query.html for details. See the next 15 slides for a treatment of SPJ query operator implementations on vertical databases (e.g., Ptree databases). CODE GENERATION implements the operator, SELECT-PROJECT-JOIN (SPJ)

Query Optimization: Relational Queries to Data Mining Most people have Data from which they want information. So, most people need DBMSs whether they know it or not. A major component of any DBMS is the query processor. Queries can range from structure to unstructured: SELECT FROM WHERE Complex queries (nested, EXISTS.. ) FUZZY queries (e.g., BLAST searches,.. OLAP (rollup, drilldown, slice/dice.. Machine LearningData Mining Relational querying Simple Searching and aggregating Supervised - Classification Regression Unsupervised- Clustering Association Rule Mining Although we looked fairly closely at the structured end of this spectrum, much research is yet to be done on that end to solve the problem of delivering standard workload answers with low response times and high throughput (D. DeWitt, ACM SIGMOD’02 plenary symposium). On the Data Mining end, we have barely scratched the surface. (But those scratches have made the difference between becoming the world’s biggest corporation and filing for bankruptcy – Walmart vs. KMart)

Some Vertical DBMS approaches BSM: A Bit Level Decomposition Storage Model A model of query optimization of all types Vertical partitioning has been studied within the context of both centralized database system as well as distributed ones. It is a good strategy when small numbers of columns are retrieved by most queries. The decomposition of a relation also permits a number of transactions to execute concurrently. Copeland et al presented an attribute level decomposition storage model (DSM) [CK85] storing each column of a relational table into a separate binary table. The DSM showed great comparability in performance. Beyond attribute level decomposition, Wong et al further took the advantage of encoding attribute values using a small number of bits to reduce the storage space [WLO+85]. In this paper, we will decompose attributes of relational tables into bit position level, utilize SPJ query optimization strategy on them, store the query results in one relational table, finally data mine using our very good P-tree methods. Our method offers these advantages: –(1) By vertical partitioning, we only need to read everything we need. This method makes hardware caching work really well and greatly increases the effectiveness of the I/O device. –(2) We encode attribute values into bit vector format, which makes compression easy to do. –(3) SPJ queries can be formulated as Boolean expressions, which facilitates fast implementation on hardware. –(4) Our model is fit not only for query processing but for data mining as well. [CK85] G.Copeland, S. Khoshafian. A Decomposition Storage Model. Proc. ACM Int. Conf. on Management of Data (SIGMOD’85), pp.268-279, Austin, TX, May 1985. [WLO + 85] H. K. T. Wong, H.-F. Liu, F. Olken, D. Rotem, and L. Wong. Bit Transposed Files. Proc. Int. Conf. on Very Large Data Bases (VLDB’85), pp.448-457, Stockholm, Sweden, 1985.

SPJ Query Optimization Strategies - One-table Selections There are two categories of queries in one-table selections: Equality Queries and Range Queries. Most techniques [WLO+85, OQ97, CI98] used to optimize them employ encoding schemes – equality encoding and range encoding. Chan and Ioannidis [CI99] defined a more general query format called interval query. An interval query on attribute A is a query of the form “x≤A≤y” or “NOT (x≤A≤y)”. It can be an equality query or a range query when x or y satisfies different kinds of conditions. We defined interval P-trees in previous work [DKR+02], which is equivalent to the bit vectors of corresponding intervals. So for each restriction in the form above, we have one corresponding interval P-tree. The ANDing result of all the corresponding interval P-trees represents all the rows satisfy the conjunction of all the restriction in the where clause. [CI98] C.Y. Chan and Y. Ioannidis. Bitmap Index Design and Evaluation. Proc. ACM Intl. Conf. on Management of Data (SIGMOD’98), pp.355-366, Seattle, WA, June 1998. [CI99] C.Y. Chan and Y.E. Ioannidis. An Efficient Bitmap Encoding Scheme for Selection Queries. Proc. ACM Intl. Conf. on Management of Data (SIGMOD’99), pp.216-226, Philadephia, PA, 1999. [DKR + 02] Q. Ding, M. Khan, A. Roy, and W. Perrizo. The P-tree algebra. Proc. ACM Symposium Applied Computing (SAC 2002), pp.426-431, Madrid, Spain, 2002. [OQ97]P. O’Neill and D. Quass. Improved Query Performance with Variant Indexes. Proc. ACM Int. Conf. on Management of Data (SIGMOD’97), pp.38-49, Tucson, AZ, May 1997.

Vertical Select-Project-Join (SPJ) Queries A Select-Project-Join query has joins, selections and projections. Typically there is a central fact relation (e.g., Enrollments or E below) to which several dimension relations are to be joined (e.g., Student(S), Course(C) below). A bit encoding is shown in reduced font italics for certain attributes, e.g., gen=gender, s=Student#, etc. S|s____|name_|gen| C|c____|name|st|term| E|s____|c____|grade | |0 000|CLAY |M 0| |0 000|BI |ND|F 0| |0 000|1 001|B 10| |1 001|THAIS|M 0| |1 001|DB |ND|S 1| |0 000|0 000|A 11| |2 010|GOOD |F 1| |2 010|DM |NJ|S 1| |3 011|1 001|A 11| |3 011|BAID |F 1| |3 011|DS |ND|F 0| |3 011|3 011|D 00| |4 100|PERRY|M 0| |4 100|SE |NJ|S 1| |1 001|3 011|D 00| |5 101|JOAN |F 1| |5 101|AI |ND|F 0| |1 001|0 000|B 10| |2 010|2 010|B 10| |2 010|3 011|A 11| |4 100|4 100|B 10| |5 101|5 101|B 10| Vertical bit sliced (uncompressed P-trees) attributes stored as: S.s 2 S.s 1 S.s 0 S.gC.c 2 C.c 1 C.c 0 C.tE.s 2 E.s 1 E.s 0 E.c 2 E.c 1 E.c 0 E.g 1 E.g 0 0000000000000110 0010001100000011 1000100100101100 1011101000100010 0101010101101111 0111011001101100 01001010 01000111 10010010 10110110 Vertical (un-bit-sliced) attributes are stored: S.name C.name C.st |CLAY | |BI | |ND| |THAIS| |DB | |ND| |GOOD | |DM | |NJ| |BAID | |DS | |ND| |PERRY| |SE | |NJ| |JOAN | |AI | |ND|

O.o 2 0 1 When 1 or more joins are required and there are more than 1 join attributes, e.g., the following SPJ on Student, Course, Offerings, Rooms, Enrollments files (next 5 slides): R:r cap |0 00|30 11| |1 01|20 10| |2 10|30 11| |3 11|10 01| SELECT S.n, C.n FROM S, C, O, R, E WHERE S.s=E.s & C.c=O.c & O.o=E.o & O.r=R.r & S.g=M & C.r=2 & E.g=A & R.c=20; S:s n gen |0 000|A|M| |1 001|T|M| |2 010|S|F| |3 011|B|F| |4 100|C|M| |5 101|J|F| C:c n cred |0 00|B|1 01| |1 01|D|3 11| |2 10|M|3 11| |3 11|S|2 10| E:s o grade |0 000|1 001|2 10| |0 000|0 000|3 11| |3 011|1 001|3 11| |3 011|3 011|0 00| |1 001|3 011|0 00| |1 001|0 000|2 10| |2 010|2 010|2 10| |2 010|7 111|3 11| |4 100|4 100|2 10| |5 101|5 101|2 10| O :o c r |0 000|0 00|0 01| |1 001|0 00|1 01| |2 010|1 01|0 00| |3 011|1 01|1 01| |4 100|2 10|0 00| |5 101|2 10|2 10| |6 110|2 10|3 11| |7 111|3 11|2 10| S.s 2 0 1 0 E.s 2 0 1 C.c 1 0 1 R.r 1 0 1 S.s 1 0 1 S.s 0 0 1 0 1 0 1 S.n A T S B C J S.g M F M F C.c 0 0 1 0 1 C.n B D M S C.r 1 0 1 C.r 0 1 0 R.r 0 0 1 0 1 R.c 1 1 0 R.c 0 1 0 1 O.o 1 0 1 0 1 O.o 0 0 1 0 1 0 1 0 1 O.c 1 0 1 O.c 0 0 1 0 1 O.r 1 0 1 O.r 0 1 0 1 0 1 0 E.s 1 0 1 0 1 0 E.s 0 0 1 0 1 E.o 2 0 1 E.o 1 0 1 0 1 0 E.o 0 1 0 1 0 1 0 1 E.g 1 1 0 1 E.g 0 0 1 0 1 0

For selections, S.g=M C.r=2 E.g=A R.c=20 create selection masks (note that C.r=2 is coded in binary as 10 b S.s 2 0 1 0 S.s 1 0 1 S.s 0 0 1 0 1 0 1 S.n A T S B C J S.g M F M F E.s 2 0 1 E.s 1 0 1 0 1 0 E.s 0 0 1 0 1 E.o 2 0 1 0 1 E.o 1 0 1 0 1 0 E.o 0 1 0 1 0 1 0 1 E.g 1 1 0 1 E.g 0 0 1 0 1 0 C.c 1 0 1 C.c 1 0 1 0 1 C.n B D M S C.r 1 0 1 C.r 2 1 0 O.o 2 0 1 O.o 1 0 1 0 1 O.o 0 0 1 0 1 0 1 0 1 O.c 1 0 1 O.c 0 0 1 0 1 O.r 1 0 1 O.r 0 1 0 1 0 1 0 R.r 1 0 1 R.r 0 0 1 0 1 R.c 1 1 0 R.c 0 1 0 1 SELECT S.n, C.n FROM S, C, O, R, E WHERE S.s=E.s & C.c=O.c & O.o=E.o & O.r=R.r & S.g=M & C.r=2 & E.g=A & R.c=20; SM 1 0 1 0 C.r 1 0 1 C.r’ 2 0 1 Cr2 0 1 E.g 1 1 0 1 E.g 0 0 1 0 1 0 EgA 0 1 0 1 0 R.c 1 1 0 R.c’ 0 0 1 0 Rc20 0 1 0 Apply selection masks (Zero out numeric values, blanked out others). S.s 2 0 S.s 1 0 1 0 S.s 0 0 1 0 S.n A T C E.s 2 0 E.s 1 0 1 0 1 0 E.s 0 0 1 0 E.o 2 0 1 0 E.o 1 0 1 0 E.o 0 0 1 0 1 0 C.c 1 0 1 C.c 0 0 1 C.n S O.o 2 0 1 O.o 1 0 1 0 1 001010101001010101 O.c 1 0 1 O.c 0 0 1 0 1 O.r 1 0 1 O.r 0 1 0 1 0 1 0 R.r 1 0 R.r 0 0 1 0

SELECT S.n, C.n FROM S, C, O, R, E WHERE S.s=E.s & C.c=O.c & O.o=E.o & O.r=R.r & S.g=M & C.r=2 & E.g=A & R.c=20; S.s 2 0 S.s 1 0 1 0 S.s 0 0 1 0 S.n A T C E.s 2 0 E.s 1 0 1 0 1 0 E.s 0 0 1 0 E.o 2 0 1 0 E.o 1 0 1 0 E.o 0 0 1 0 1 0 C.c 1 0 1 C.c 0 0 1 C.n S O.o 2 0 1 O.o 1 0 1 0 1 O.o 0 0 1 0 1 0 1 0 1 O.c 1 0 1 O.c 0 0 1 0 1 O.r 1 0 1 O.r 0 1 0 1 0 1 0 R.r 1 0 R.r 0 0 1 0 For the joins, S.s=E.s C.c=O.c O.o=E.o O.r=R.r, one approach is to follow an indexed nested loop like method (note that the P-trees themselves are self indexing). The join O.r=R.r is simply part of a selection on O (R doesn’t contribute output nor participate in any further operations) Use the Rc20-masked R as the inner relation and O as the r-indexed outer relation) to produce a further selection mask for O. Rc20 0 1 0 Get 1 st R.r value, 01 b Mask the corresponding O tuples, P O.r 1 ^P’ O.r 0 O.r 1 0 1 O’.r 0 0 1 0 1 0 1 OM 0 1 0 1 This is the only R.r value (if there were more, one would do the same for each, then OR those masks to get the final O-mask). Next, we apply the O-mask, OM to O O.o 2 0 1 0 1 O.o 1 0 1 O.o 0 0 1 0 1 O.c 1 0 1 0 1 O.c 0 0 1

SELECT S.n, C.n FROM S, C, O, R, E WHERE S.s=E.s & C.c=O.c & O.o=E.o & O.r=R.r & S.g=M & C.r=2 & E.g=A & R.c=20; S.s 2 0 S.s 1 0 1 0 S.s 0 0 1 0 S.n A T C E.s 2 0 E.s 1 0 1 0 1 0 E.s 0 0 1 0 E.o 2 0 1 0 E.o 1 0 1 0 E.o 0 0 1 0 1 0 C.c 1 0 1 C.c 0 0 1 C.n S For the final 3 joins C.c=O.c O.o=E.o E.s=S.s the same indexed nested loop like method can be used. O.o 2 0 1 0 1 O.o 1 0 1 O.o 0 0 1 0 1 O.c 1 0 1 0 1 O.c 0 0 1 Get 1 st masked C.c value, 11 b Mask corresponding O tuples: P O.c 1 ^P O.c 0 O.c 1 0 1 0 1 O.c 0 0 1 OM 0 1 Get 1 st masked O.o value, 111 b Mask corresponding E tuples: P E.o 2 ^P E.o 1 ^P E.o 0 E.o 1 0 1 0 E.o 0 0 1 0 1 0 Get 1 st masked E.s value, 010 b Mask corresponding S tuples: P’ S.s 2 ^P S.s 1 ^P’ S.s 0 S’.s 2 1 0 1 0 S.s 1 0 1 0 S’.s 0 1 0 1 0 SM 0 1 0 Get S.n-value(s), C, pair it with C.n-value(s), S, output concatenation, C.n S.n There was just one masked tuple at each stage in this example. In general, one would loop through the masked portion of the extant domain at each level (thus, Indexed Horizontal Nested Loop or IHNL) E.o 2 0 1 0 EM 0 1 0 S C

SELECT S.n, C.n FROM S, C, O, R, E WHERE S.s=E.s & C.c=O.c & O.o=E.o & O.r=R.r & S.g=M & C.r=2 & E.g=A & R.c=20; S.s 1 0 S.s 2 0 1 0 S.s 3 0 1 0 S.n A T C E.s 1 0 E.s 2 0 1 0 1 0 E.s 3 0 1 0 E.o 1 0 1 0 E.o 2 0 1 0 E.o 3 0 1 0 1 0 C.c 1 0 1 C.c 1 0 1 C.n S Having done the query tree sequentially (selections first, then joins and projections) it appears that the entire query tree could be done in one combined step by looping through the masked C tuples, for each C.n value, determine if there is an S.n value that should be paired with it by logical operations output those S.n, C.n pair(s), if any, else go to the next masked C.n value. Does this lead to a one-pass vertical query optimizer?!?!?! Can the indexed nested loop like algorithm be modified to loop horizontally? (across bit positions, rather than down tuples?) O.o 1 0 1 0 1 O.o 2 0 1 O.o 3 0 1 0 1 O.c 1 0 1 0 1 O.c 2 0 1

DISTINCT Keyword, GROUP BY Clause, ORDER BY Clause, HAVING Clause and Aggregate Operations Duplicate elimination after a projection (SQL DISTINCT keyword) is one of the most expensive operations in query optimisation. In general, it is as expensive as the join operation. However, in our approach, it can automatically be done while forming the output tuples (since that is done in an order). While forming all output records for a particular value of the ORDER BY attribute, duplicates can be easily eliminated without the need for an expensive algorithm. The ORDER BY and GROUP BY clauses are very commonly used in queries and can require a sorting of the output relation. However, in our approach, if the central relation is chosen to be the one with the sort attribute and the surrogation is according to the attribute order (typically the case – always the case for numeric attributes), then the final output records can be put together and aggregated in the requested order without a separate sort step at no additional cost. Aggregation operators such as COUNT, SUM, AVG, MAX, and MIN can be implemented without additional cost during the output formation step and any HAVING decision can be made as output records are being composed, as well (See Yue Cui’s Master’s thesis in NDSU library for vertical aggregation computations using P-trees.) If the Count aggregate is requested by itself, we note that P-trees automatically provide the full counts for any predicate with just one multiway AND operation.

The following example illustrates these points. SELECTDISTINCT C.c, R.capacity FROM S,C,E,O,R WHERE S.s=E.s AND C.c=O.c AND O.o=E.o AND O.r=R.r AND C.cred>1 AND (E.grade='B' OR E.grade='A') AND R.capacity>10 ORDER BY C.c; S___________ C___________ E_________________ O_______________ R_____________ |s |n|gen| |c |n|cred| |s |o |grade| |o |c |r | |r |capacity| |0 000|A|M 0| |0 00|B|1 01| |0 000|1 001|2 10| |0 000|0 00|0 01| |0 00|30 11| |1 001|T|M 0| |1 01|D|3 11| |0 000|0 000|3 11| |1 001|0 00|1 01| |1 01|20 10| |2 010|S|F 1| |2 10|M|3 11| |3 011|1 001|3 11| |2 010|1 01|0 00| |2 10|30 11| |3 011|B|F 1| |3 11|S|2 10| |3 011|3 011|0 00| |3 011|1 01|1 01| |3 11|10 01| |4 100|C|M 0| |1 001|3 011|0 00| |4 100|2 10|0 00| |5 101|J|F 1| |1 001|0 000|2 10| |5 101|2 10|2 10| Sn |2 010|2 010|2 10| |6 110|2 10|3 11| A |2 010|3 011|3 11| |7 111|3 11|2 10| T |4 100|4 100|2 10| S |5 101|5 101|2 10| Ss1 Ss2 Ss3 Sgen B 0011 0000 0101 0001 C Egrade1 Egrade2 Cn 00 11 01 11 J 1101 0100 Cc1 Cc2 Ccred1 Ccred2 B 1011 1001 00 01 01 11 D Es1 Es2 Es3 Eo1 Eo2 Eo3 11 00 11 01 11 10 M 0000 0000 0011 0000 0010 1010 S 0000 1111 1100 0000 0111 1101 Rr1 Rr2 Rcap1 Rcap2 11 00 01 11 00 01 00 01 11 10 11 01 10 11 Oo1 Oo2 Oo3 Oc1 Oc2 Or1 Or2 0011 0000 0101 0011 0000 0001 1100 0011 1111 0101 0011 1101 0011 0110 Apply selection masks: mE =Egrade1 mR =Rcap1 mC =Ccred1 1101 11 01 1011 10 11 11

results in, Es1 Es2 Es3 Eo1 Eo2 Eo3 Rr1 Rr2 Cc1 Cc2 00 0 00 0 00 1 00 0 00 0 10 0 00 01 0 1 0 00 1 11 1 00 0 00 0 11 1 01 1 0 11 01 11 00 01 11 00 01 Semijoin (toward center), E  O(on o=0,1,2,3,4,5), R  O(on r=0,1,2), C  O(on c=1,2,3), reduces Oo1 Oo2 Oo3 Oc1 Oc2 Or1 Or2 0011 0000 0101 0011 0000 0001 1100 0011 1111 0101 0011 1101 0011 0110 to Oo1 Oo2 Oo3 Oc1 Oc2 Or1 Or2 11 00 01 11 00 01 00 00 11 01 00 11 00 01 Thus, the participants are c=1,2; r=0,1,2; o=2,3,4,5. Semijoining back again produces the following. Cc1 Cc2 Rr1 Rr2 Es1 Es2 Es3 Eo1 Eo2 Eo3 0 1 00 01 00 11 00 00 11 01 1 0 1 0 11 00 01 11 00 01 Thus, s partic are s=2,4,5. Ss1 Ss2 Ss3 11 00 01 0 1 0 Output tuples are determined from participating O.c P-trees. RC(P O.c (2)) = RC(Oc 1 ^Oc 2 ’)=2, since Oc1 ^ Oc2’ 11 11 = 11 00 00 00 Since the 1-bits are in positions 4 and 5, the two O-tuples have O.o surrogate values 4 and 5. The r-values at positions 4 and 5 of O.r are 0 and 2. Thus, we retrieve the R.capacity values at offsets 0 and 2. However, both of these R.capacity values are 30. Thus, this duplication is discovered without sorting or additional processing. The only output is (2,30). Similarly, RCntP O.c (1) = RCntOc 1 ’^Oc 2 =2, Oc1’ ^ Oc2 00 00 = 00 11 11 11 Finally note, if ORDER BY clause is over an attribute which is not in the relation O (e.g., over student number, s) then we center the query tree (or wheel) on a fact file that contains the ORDER BY attribute (e.g., on E in this case). If the ORDER BY attribute is not in any fact file (in a dimension file only) then the final query tree can be re-arranged to center on the dimension file containing that attribute. Since output ordering and duplicate elimination are traditionally very expensive sub-operations of SPJ query processing, the fact that our BDM model and the P-tree data structure provide a fast and efficient way to accomplish these operations is a very favorable aspect of the approach.

Combining Data Mining and Query Processing Many data mining request involve pre-selection, pre-join, and pre-projection on a database to isolate the specific data subset to which the data mining algorithm is to be applied. For example, in the above database, one might be interested in all Association Rules of a given support threshold and confidence threshold but only on the result relations of the complex SPJ query shown. The brute force way to do this is to first join all relations into one universal relation and then to mine that gigantic relation. This is not a feasible solution in most cases due to the size of the resulting universal relation. Furthermore, often some selection on that universal relation is desirable prior to the mining step. Our approach accommodates combinations of querying and data mining without necessitation the creation of a massive universal relation as an intermediate step. Essentially, the full vertical partitioning and P-trees provide a selection and join path which can be combined with the data mining algorithm to produce the desired solution without extensive processing and massive space requirements. The collection of P-trees and BSQ files constitute a lossless, compressed version of the universal relation. Therefore the above techniques, when combined with the required data mining algorithm can produce the combination result very efficiently and directly.

O.o 2 0 1 R:r cap |0 00|30 11| |1 01|20 10| |2 10|20 10| |3 11|10 01| S:s n gen |0 000|A|M| |1 001|T|M| |2 010|S|F| |3 011|B|F| |4 100|C|M| |5 101|J|F| C:c n cred |0 00|B|1 01| |1 01|D|3 11| |2 10|M|3 11| |3 11|S|2 10| E:s o grade |0 000|1 001|2 10| |0 000|0 000|3 11| |3 011|1 001|3 11| |3 011|3 011|0 00| |1 001|3 011|0 00| |1 001|0 000|2 10| |2 010|2 010|2 10| |2 010|7 111|3 11| |4 100|4 100|2 10| |5 101|5 101|2 10| O :o c r |0 000|0 00|0 01| |1 001|0 00|1 01| |2 010|1 01|0 00| |3 011|1 01|1 01| |4 100|2 10|0 00| |5 101|2 10|2 10| |6 110|2 10|3 11| |7 111|3 11|2 10| S.s 2 0 1 0 E.s 2 0 1 C.c 1 0 1 R.r 1 0 1 S.s 1 0 1 S.s 0 0 1 0 1 0 1 S.n A T S B C J S.g 0 1 0 1 C.c 0 0 1 0 1 C.n B D M S C.r 1 0 1 C.r 0 1 0 R.r 0 0 1 0 1 R.c 1 1 0 R.c 0 1 0 1 O.o 1 0 1 0 1 O.o 0 0 1 0 1 0 1 0 1 O.c 1 0 1 O.c 0 0 1 0 1 O.r 1 0 1 O.r 0 1 0 1 0 1 0 E.s 1 0 1 0 1 0 E.s 0 0 1 0 1 E.o 2 0 1 E.o 1 0 1 0 1 0 E.o 0 1 0 1 0 1 0 1 E.g 1 1 0 1 E.g 0 0 1 0 1 0 Horizontal Indexed Nested Loop Join??? SELECT * FROM S,E WHERE S.s=E.s 1 st if 0<rc(S.s 2 ) thenif rc(S.s 2 )<|S| thenif 0<rc(S.s 1 )^ thenif rc(S.s 1 )<|S| thenif 0<rc(S.s 0 ) thenif rc(S.s 0 )<|S| then… So depth-first traversal down the bitslice tree for S.s, skipping all values that are not present, and for each S.s value that is present, one and gives that value P-tree in E (index into E) so optimal retrieval can be done. If the Ptrees are organized according to physical boundaries as below, then is there a P-tree based Hybrid Hash join that allows us to avoid excessive rereads of extents? It seems clear that compressing bit vectors into P-trees based, not on 1/2 d boundaries, but on page and extent boundaries is important. Use the Dr. Md Masum Serazi approach, but with the following levels (possibly collapsing levels 0 and 1 together) The level-0 fanout is the bfr of the page blocks. The level-1 fanout is the extent size (# of blocks per extent). The level-2 fanout is the (maximum) number of extents per file. The level-3 fanout is the number of files in the DB The real advantage of this approach may to apply it to join algorithms where the location of join Attribute values is known ( see V. Goli’s thesis) since we know the location of all values Through ANDs.

E.s 2 1 0 1 0 1 0 1 E.s 1 0 1 0 1 0 1 0 E.s 0 0 1 0 1 0 1 0 S.s 2 0 1 S.s 1 0 1 0 1 S.s 0 0 1 0 1 0 1 0 1 S.a 1 0 1 0 C.c 1 0 1 C.c 0 0 1 0 1 C.n 1 0 1 E.g 0 1 0 1 0 1 0 E.c 1 0 1 0 1 0 1 0 1 0 E.c 0 1 0 1 0 1 0 1 0 1 0 1 0 Graph G=(N,E) is (T,I)-bipartite iff N=T  !I and  e={e 1,e 2 }  E, if e 1  T [I] then e 2  I [T]. WOLOG write e={e T,e I } (E is directed from T to I e=(e T,e I ) ) E={ {e k,T,e k,I } | k=1..|E|} or the edge relationship can be expressed as tIset, E T = { (t,Iset(t) | t  T and Iset(t)={i|{t,i}  E} iTset, E I = { (i,Iset(i) | where Iset(i)={t | {t,i}  E} tImap, E Tb ={ (t,b 1,...,b |I| ) | where b k =1 iff e k,T =t} iTmap, E Ib ={ (i,b 1,...,b |T| ) | where b k =1 iff e k,I =t} Given a star schema with fact, E and dimensions, S, C. E is a ER-relationship between entities, S and C and is therefore a bipartite graph, G=(N,E) where N is the disjoint union of S and C. Given a join S.s with E.s, JoinIndex (JI) is a relationship between S and E, giving a bipartite graph, G=(S  !E,JI). The sEmap of this relationship is the association matrix of Qiang Ding's thesis.

Desirable Features of a Distributed DBMS: LOCATION TRANSPARENCY is achieved if a user can access needed data without having to know which site has that data. -simplifies logic of programs -allows data movement as usage patterns change A data object (typically a file) is fragmented if it is divided into multiple pieces for storage and/or placement purposes at different sites. e.g., accounts files: Fargo customer accounts can be stored in Fargo, Grand Forks customer accounts can be stored in Grand Forks...) FRAGMENTATION TRANSPARENCY is achieved if users can access needed data without having to know whether it is fragmented. a data object (typically a record or file) is REPLICATED if it has ≥ 1 physical copy -distributed replication advantages include availability -disadvantages include increased update overhead. REPLICATION TRANSPARENCY is achieved if users can access needed data without knowing whether or not it is replicated. Additional desirable DDBMS features include: LOCAL AUTONOMY is achieved if the system is distributed consistent with the logical and physical distribution of the enterprise. It allows local control over local data, It allows local accountability and less dependency on remote Data Processing Support for INCREMENTAL GROWTH AVAILABILITY and RELIABILITY. QUERY PROCESSING in Distributed DBMSs (DDBMSs)

Distributed systems can more easily allow for graceful (and unlimited) growth simply by adding additional sites. The DDBMS software should allow for adding sites easily. Reliability can be provided by replicating data. The DDBMS should allow for replication to enhance reliability and availability in the presence of failures of sites or links. DISTRIBUTED QUERIES Query Optimization Methods can be STATIC: strategy of transmissions and local processing activities is fully determined before execution begins (at compile time). DYNAMIC: Each step is decided after seeing results of previous steps. Response time usually is dominated by transmission costs (i.e., local processing times are negligible by comparison - assumed 0?). One model is to take RESPONSE time to be linear in number of bytes, X, sent: R(X) = AX + B B is the fixed (setup?) cost of the transmission and AX is the variable cost (depending on message size only, not distance). What assumptions does this make? (next slide) QUERY PROCESSING in Distributed DBMSs (DDBMSs)

71 Bandwidth = The number of fbits per second that can be sent. delay Time to send a message from point A to point B Propagation = Distance / SpeedOfLight: The time between when the last bit enters and last bit leaves the link. Transmit = Size / Bandwidth: The time between when 1 st bit enters and last bit enters the link. Components of delay = Propagation + Transmit + Queue (=delays in send and demultiplexing queues) Propagation versus Transmit delay If you’re sending 1 byte, propagation delay dominates. If you’re sending 500 MB, transmit delay dominates

A STATIC, QUERY PROCESSING ALGORITHM usually takes as input: database statistics such as relation sizes attribute sizes projected sizes of attributes produces as output: a strategy for answering the query (a pattern of what transmissions to make, when, where and what local processing to do, when and where) Usually involves 4 phases: LOCAL PROCESSING phase: do all processing that can be done initially at each site that doesn't require data interchange between sites. (e.g., local selections, joins and projections) The result this phase is that there will be one participating relation at each participating site. REDUCTION phase: selected "semijoins" to be done to reduce the size of participating relations by eliminating tuples that are not needed in answering the query. TRANSPORT phase: send one relation from each participating site (the result of the reduction phase) to the querying site. COMPLETION phase: finishing up processing using those relations to get final answer (e.g., final projects, selects, joins) See "Other Materials" http://www.cs.ndsu.nodak.edu/~perrizo/classes/765/09query.html for more details.http://www.cs.ndsu.nodak.edu/~perrizo/classes/765/09query.html QUERY PROCESSING in Distributed DBMSs (DDBMSs)

What is the SEMIJOIN of R 1 (A,B) to R 2 (A,C) on A? (written: R 1 :A→ R 2 ). 1. projection R onto A (again, the result is written as R 1 [A]) 2. R 1 [A]:A→ R 2 (Which selects those tuples of R 2 that will participate in the join). The result of R 1 [A]:A→ R 2 is the sub-relation of R 2 of only those R 2 -tuples which will participate in the full join of R 1 JOIN A R 2 on A (eliminates non-participants at the cost of generating (and sending, if R 2 is located at a different site than R 1 ) the R 1 -join attribute values). A semijoin can be viewed as a special SELECTION operator also, since it selects out those tuples of R 2 that have a matching A-value in R 1. Thus the semijoin is perfect for reducing the size of relations before they are sent to the querying site. But note that semijoins don't always end up reducing the size of a relation. See "Other Materials" http://www.cs.ndsu.nodak.edu/~perrizo/classes/765/09query.html for more details.http://www.cs.ndsu.nodak.edu/~perrizo/classes/765/09query.html QUERY PROCESSING in Distributed DBMSs (DDBMSs)

For example, STUDENT-FILE S#|SNAME |LCODE 25|CLAY |NJ5101 32|THAISZ|NJ5102 38|GOOD |FL6321 17|BAID |NY2091 57|BROWN |NY2092 ENROLL FILE S#|C#|GRADE 32|8 | 89 32|7 | 91 32|6 | 62 38|6 | 98 STATIC QUERY PROCESSING Example in DDBMSs ENROLL S# →STUDENT 1. project ENROLL onto the S# attribute: S# 32 38 2. Join the two relations on S# S# S# |SNAME |LCODE |32| 25|CLAY |NJ5101 |38| join 32|THAISZ|NJ5102 38|GOOD |FL6321 17|BAID |NY2091 57|BROWN |NY2092 resulting in: S# |SNAME |LCODE 32|THAISZ|NJ5102 38|GOOD |FL6321

STATIC QUERY PROCESSING Example in DDBMSs semijoins don't always end up reducing the size of a relation. Consider STUDENT S#→ ENROLL Project STUDENT onto the S# attribute and join it with ENROLL: S# S#|C#|GRADE 25 join 32|8 | 89 32 32|7 | 91 38 32|6 | 62 17 38|6 | 98 57 resulting in the entirety of STUDENT again (no tuples eliminated)! So let's make it a rule: - never semijoin the primary key to a foreign key, because it will always result in no reduction.

Distributed Semijoin of R 1 at site1 to R 2 at site2 along A At site1: R 1 At site 2: R 2 A 1 A 2 A 3 A 4 A 5 A 6 A 7 A 8 A 9 A 1 A 2 a A A B C C E A F d 1 a C D D E A A B B e 2 b A B C D B A B A g 3 c D D B B A C A C e E B A A C C D D Assume response time for transmission of X bytes between any 2 sites is R(X) = X + 10 time units. 1. projection R 1 [A] 2. transmission of R 1 [A] to the site2. 3. R 1 [A] A-join R 2 (select R 2 -tuples that participate in join) Consider the following distributed query: Assume SELECT R 1.A 2, R 2.A 2 FROM R 1,R 2 WHERE R 1.A 1 = R 2.A 1 arrives at site3.

a A A B C C E A F a C D D E A A B B b A B C D B A B A c D D B B A C A C e E B A A C C D D Distributed Semijoin of R 1 at site1 to R 2 at site2 along A STRATEGY 1 Strategy-1: (No reduction phase). 1. Send R 1 to site3: 45 bytes sent. Cost is R(45)=45+10 = 55 2. Send R 2 to site3: 6 bytes sent. Cost of R(6)= 6+10 = 16 3. Final join (cost = 0) site2: R 2 A 1 A 2 a A A B C C E A F a C D D E A A B B b A B C D B A B A c D D B B A C A C e E B A A C C D D At site1: R 1 A 1 A 2 A 3 A 4 A 5 A 6 A 7 A 8 A 9 site3 d 1 e 2 g 3 result: e EBAACCDD2 Response time= 71 Strategy 1': If 1. and 2. are done in parallel, the response time= 55

degdeg e E B A A C C D D a A A B C C E A F a C D D E A A B B b A B C D B A B A c D D B B A C A C e E B A A C C D D Distributed Semijoin of R 1 at site1 to R 2 at site2 along A; STRATEGY 2 1. Send R 2 [A] to site1; site2: R 2 A 1 A 2 At site1: R 1 A 1 A 2 A 3 A 4 A 5 A 6 A 7 A 8 A 9 site3 result: e EBAACCDD2 Response time = 48 Strategy 2': If 1. and 3. are (can be?) done in parallel, the response time = 32 2. Send R 2 [A]  R 1 to site3. 9 bytes sent. Cost=R(9)= 19 3. Send R 2 to site3; 6 bytes sent. Cost=R(3) = 16 4. JOIN R 2 [A]  R 1 and R 2 on A 1 at site3. Cost = 0 do R 2 [A]  R 1 d 1 e 2 g 3 3 bytes sent. Cost=R(3)= 13 d 1 e 2 g 3

degdeg e E B A A C C D D a A A B C C E A F a C D D E A A B B b A B C D B A B A c D D B B A C A C e E B A A C C D D Distributed Semijoin of R 1 at site1 to R 2 at site2 along A; STRATEGY 3 1. Send R 1 [A] to site2; site2: R 2 A 1 A 2 At site1: R 1 A 1 A 2 A 3 A 4 A 5 A 6 A 7 A 8 A 9 site3 result: e EBAACCDD2 Response time = 122 Strategy 3': If 1. and 3. are (can be?) done in parallel, the response time = 67 2. Send R 1 [A]  R 2 to site3. 2 bytes sent. Cost=R(2)= 12 3. Send R 1 to site3; 45 bytes sent. Cost=R(45) = 55 4. JOIN R 1 [A]  R 2 and R 1 on A 1 at site3. Cost = 0 do R 1 [A]  R 2 d 1 e 2 g 3 45 bytes sent. Cost=R(45)= 55 d 1 e 2 g 3

Distributed Query Processor DQP) must pick the strategy! For static algorithms, the hardest job of the Distributed Query Processor (which is at site3 where the query came in and must be processed) is to pick among these 6 alternatives (if other transmission and local processing cost are used, there would be a vastly different set of alternative strategies). The DQP at site3 must pick a strategy without seeing the data at aites 1 and 2. E.g., if the DQP decides that a "one semijoin strategy is best, should it be 2, versus 3 (or 2' versus 3' if the network accomodates parallel transmissions from a given send site). Note the vast difference is cost (2 costs 48 and 3 costs 122, though both are 1 semijoin strategies! 3 costs more than the no semijoin strategy which is 1 at a cost of 55). The DQP has a need for estimates of the two semijoin result sizes, since the actual results are not known in advance at site 3. That estimation method is important, but difficult, since the situation can be very different than the above.

d 1 e 2 g 3 q 4 q 5 v 7 d 1 e 2 g 3 q 4 q 5 v 7 d A A B C C E A F d C D D E A A B B e A B C D B A B A g D D B B A C A C e E B A A C C D D degqqvdegqqv d A A B C C E A F d C D D E A A B B e A B C D B A B A g D D B B A C A C e E B A A C C D D STRATEGY 2 with different R 1 and R 2 data 1. Send R 2 [A] to site1; site2: R 2 A 1 A 2 At site1: R 1 A 1 A 2 A 3 A 4 A 5 A 6 A 7 A 8 A 9 site3 result: dEBAACCDD1 Response time = 93 dCDDEAABB1 eABCDBABA2 gDDBBACAC3 eEBAACCDD2 Strategy 1 has same cost=71 so Strategy 1 is better! How should semijoin results be estimated? 2. Send R 2 [A]  R 1 to site3. 45 bytes sent. Cost=R(45)= 55 3. Send R 2 to site3; 12 bytes sent. Cost=R(12) = 22 4. JOIN R 2 [A]  R 1 and R 2 on A 1 at site3. Cost = 0 do R 2 [A]  R 1 6 bytes sent. Cost=R(3)= 16

Selectivity Theory for estimating semijoin results. The work of Hevner and Yao assumes data values are uniformly distributed and attribute-distributions are independent of each other. Results estimated as follows: (assuming A 1 has domain {a,b,...z}). The Selectivity of attributed R 1 is the ratio of the number of values present (size of the extant domain) over the number of values possible (size of full domain). Therefore the selectivity of R 1.A is 3/26. Using selectivity theory, we estimate the size of semijoin, R 1 A-semijoin R2 as: (Original size of R 2 )*(selectivity of incoming, R 1.A): 45 * 3/26 = 5.2 Selectivity theory estimates 5.2 bytes of R 1 survive semijoin. This is close for the first example database state and the algorithm proposed by Hevner & Yao (ALGORITHM-GENERAL) would correctly select method 1. However, it is way off in the second database state but ALGORITHM- GENERAL would still select strategy-1 (not best for this DB state).

UPDATE PROPAGATION IN DISTRIBUTED DATABASES UPDATE PROPAGATION: To update any replicated data item, the DDBMS must propagate the new value consistently to all copies. IMMEDIATE method: update all copies (the update fails if even 1 copy is unavailable) PRIMARY method: designate 1 copy as primary for each item. Update is deemed complete (COMMITTED) when primary copy is updated. Primary copy site is responsible for broadcasting the update to the other sites. Broadcast can be done in parallel while the transaction is contining, however that runs counter to local autonomy theme

Query Processing SQL Queries in a high level language such as SQL are processed by Horizontal DBMSs in the following steps: 1. SCAN and PARSE (SCANNER-PARSER):

Similar presentations

Presentation on theme: "Query Processing SQL Queries in a high level language such as SQL are processed by Horizontal DBMSs in the following steps: 1. SCAN and PARSE (SCANNER-PARSER):"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Query Processing SQL Queries in a high level language such as SQL are processed by Horizontal DBMSs in the following steps: 1. SCAN and PARSE (SCANNER-PARSER):

Similar presentations

Presentation on theme: "Query Processing SQL Queries in a high level language such as SQL are processed by Horizontal DBMSs in the following steps: 1. SCAN and PARSE (SCANNER-PARSER):"— Presentation transcript:

Similar presentations

About project

Feedback