CS 245Notes 51 CS 245: Database System Principles Hector Garcia-Molina Notes 5: Hashing and More

CS 245Notes 52 key  h(key) Hashing Buckets (typically 1 disk block)

CS 245Notes Two alternatives records (1) key  h(key)

CS 245Notes 54 (2) key  h(key) Index record key 1 Two alternatives Alt (2) for “secondary” search key

CS 245Notes 55 Example hash function Key = ‘x 1 x 2 … x n ’ n byte character string Have b buckets h: add x 1 + x 2 + ….. x n – compute sum modulo b

CS 245Notes 56  This may not be best function …  Read Knuth Vol. 3 if you really need to select a good function. Good hash  Expected number of function:keys/bucket is the same for all buckets

CS 245Notes 57 Within a bucket: Do we keep keys sorted? Yes, if CPU time critical & Inserts/Deletes not too frequent

CS 245Notes 58 Next: example to illustrate inserts, overflows, deletes h(K)

CS 245Notes 59 EXAMPLE 2 records/bucket INSERT: h(a) = 1 h(b) = 2 h(c) = 1 h(d) = d a c b h(e) = 1 e

CS 245Notes a b c e d EXAMPLE: deletion Delete: e f f g maybe move “g” up c d

CS 245Notes 511 Rule of thumb: Try to keep space utilization between 50% and 80% Utilization = # keys used total # keys that fit If < 50%, wasting space If > 80%, overflows significant depends on how good hash function is & on # keys/bucket

CS 245Notes 512 How do we cope with growth? Overflows and reorganizations Dynamic hashing Extensible Linear

CS 245Notes 513 Extensible hashing: two ideas (a) Use i of b bits output by hash function b h(K)  use i  grows over time…

CS 245Notes 514 (b) Use directory h(K)[i ] to bucket

CS 245Notes 515 Example: h(k) is 4 bits; 2 keys/bucket i = Insert New directory i = 2 2

CS 245Notes Insert: i = Example continued

CS 245Notes i = Insert: 1001 Example continued i = 3 3

CS 245Notes 518 Extensible hashing: deletion No merging of blocks Merge blocks and cut directory if possible (Reverse insert procedure)

CS 245Notes 519 Deletion example: Run thru insert example in reverse!

CS 245Notes 520 Extensible hashing Can handle growing files - with less wasted space - with no full reorganizations Summary + Indirection (Not bad if directory in memory) Directory doubles in size (Now it fits, now it does not) - -

CS 245Notes 521 Linear hashing Another dynamic hashing scheme Two ideas: (a) Use i low order bits of hash grows b i (b) File grows linearly

CS 245Notes 522 Example b=4 bits, i =2, 2 keys/bucket m = 01 (max used block) Future growth buckets If h(k)[i ]  m, then look at bucket h(k)[i ] else, look at bucket h(k)[i ] - 2 i -1 Rule 0101 can have overflow chains! insert 0101

CS 245Notes 523 Example b=4 bits, i =2, 2 keys/bucket m = 01 (max used block) Future growth buckets insert

CS 245Notes 524 Example Continued: How to grow beyond this? m = 11 (max used block) i =

CS 245Notes 525 If U > threshold then increase m (and maybe i )  When do we expand file? Keep track of: # used slots total # of slots = U

CS 245Notes 526 Linear Hashing Can handle growing files - with less wasted space - with no full reorganizations No indirection like extensible hashing Summary + + Can still have overflow chains -

CS 245Notes 527 Example: BAD CASE Very full Very emptyNeed to move m here… Would waste space...

CS 245Notes 528 Hashing - How it works - Dynamic hashing - Extensible - Linear Summary

CS 245Notes 529 Next: Indexing vs Hashing Index definition in SQL Multiple key access

CS 245Notes 530 Hashing good for probes given key e.g., SELECT … FROM R WHERE R.A = 5 Indexing vs Hashing

CS 245Notes 531 INDEXING (Including B Trees) good for Range Searches: e.g., SELECT FROM R WHERE R.A > 5 Indexing vs Hashing

CS 245Notes 532 Index definition in SQL Create index name on rel (attr) Create unique index name on rel (attr) defines candidate key Drop INDEX name

CS 245Notes 533 CANNOT SPECIFY TYPE OF INDEX (e.g. B-tree, Hashing, …) OR PARAMETERS (e.g. Load Factor, Size of Hash,...)... at least in SQL... Note

CS 245Notes 534 ATTRIBUTE LIST  MULTIKEY INDEX (next) e.g., CREATE INDEX foo ON R(A,B,C) Note

CS 245Notes 535 Motivation: Find records where DEPT = “Toy” AND SAL > 50k Multi-key Index

CS 245Notes 536 Strategy I: Use one index, say Dept. Get all Dept = “Toy” records and check their salary I1I1

CS 245Notes 537 Use 2 Indexes; Manipulate Pointers ToySal > 50k Strategy II:

CS 245Notes 538 Multiple Key Index One idea: Strategy III: I1I1 I2I2 I3I3

CS 245Notes 539 Example Record Dept Index Salary Index Name=Joe DEPT=Sales SAL=15k Art Sales Toy 10k 15k 17k 21k 12k 15k 19k

CS 245Notes 540 For which queries is this index good? Find RECs Dept = “Sales” SAL=20k Find RECs Dept = “Sales” SAL > 20k Find RECs Dept = “Sales” Find RECs SAL = 20k

CS 245Notes 541 Interesting application: Geographic Data DATA: x y...

CS 245Notes 542 Queries: What city is at ? What is within 5 miles from ? Which is closest point to ?

CS 245Notes 543 h n b i a c o d Example e g f m l k j h i a b c d e f g n o m l j k Search points near f Search points near b 5 15

CS 245Notes 544 Queries Find points with Yi > 20 Find points with Xi < 5 Find points “close” to i = Find points “close” to b =

CS 245Notes 545 Many types of geographic index structures have been suggested Quad Trees R Trees

CS 245Notes 546 Two more types of multi key indexes Grid Partitioned hash

CS 245Notes 547 Grid Index Key 2 X 1 X 2 …… X n V 1 V 2 Key 1 V n To records with key1=V 3, key2=X 2

CS 245Notes 548 CLAIM Can quickly find records with –key 1 = V i  Key 2 = X j –key 1 = V i –key 2 = X j And also ranges…. –E.g., key 1  V i  key 2 < X j

CS 245Notes 549  But there is a catch with Grid Indexes! How is Grid Index stored on disk? Like Array... X1X2X3X4 X1X2X3X4X1X2X3X4 V1V2V3 Problem: Need regularity so we can compute position of entry

CS 245Notes 550 Solution: Use Indirection Buckets V 1 V 2 V 3 * Grid only V 4 contains pointers to buckets Buckets -- X1 X2 X3

CS 245Notes 551 With indirection: Grid can be regular without wasting space We do have price of indirection

CS 245Notes 552 Can also index grid on value ranges SalaryGrid Linear Scale 123 ToySalesPersonnel 0-20K1 20K-50K2 50K-3 8

CS 245Notes 553 Grid files Good for multiple-key search Space, management overhead (nothing is free) Need partitioning ranges that evenly split keys + - -

CS 245Notes 554 Idea: Key1 Key2 Partitioned hash function h1h

CS 245Notes 555 h1(toy)=0000 h1(sales)=1001 h1(art)= h2(10k)=01100 h2(20k)=11101 h2(30k)=01110 h2(40k)=00111., EX: Insert

CS 245Notes 556 h1(toy)=0000 h1(sales)=1001 h1(art)= h2(10k)=01100 h2(20k)=11101 h2(30k)=01110 h2(40k)= Find Emp. with Dept. = Sales  Sal=40k

CS 245Notes 557 h1(toy)=0000 h1(sales)=1001 h1(art)= h2(10k)=01100 h2(20k)=11101 h2(30k)=01110 h2(40k)= Find Emp. with Sal=30k look here

CS 245Notes 558 h1(toy)=0000 h1(sales)=1001 h1(art)= h2(10k)=01100 h2(20k)=11101 h2(30k)=01110 h2(40k)= Find Emp. with Dept. = Sales look here

CS 245Notes 559 Post hashing discussion: - Indexing vs. Hashing - SQL Index Definition - Multiple Key Access - Multi Key Index Variations: Grid, Geo Data - Partitioned Hash Summary