CS CS4432: Database Systems II Basic indexing

CS Indexing : helps to retrieve data quicker for certain queries value= 1,000,000 Select * FROM Emp WHERE salary = 1,000,000; Chapter 13  value record

CS Topics Sequential Index Files (chap 13.1) Secondary Indexes (chap 13.2)

CS Sequential File

CS Sequential File Dense Index Every record is in index.

CS Sequential File Sparse Index Only first record per block in index.

CS Sequential File Sparse 2nd level

CS Note : DATA FILE or INDEX are “ordered files”. Question: How would we lay them out on disk ? - contiguous layout on disk ? - block-chained layout on disk ?

CS Questions: Do we want to build a dense 2 nd -level index for a dense index? Can we even do this ? Sequential File nd level? st level?

CS Notes on pointers: (1)Block pointer (used in sparse index) can be smaller than record pointer (used in dense index) BP RP

CS K1 K3 K4 K2 R1R2R3R4 say: 1024 B per block if we want K3 block: get it at offset (3-1)*1024 = 2048 bytes Note : If file is contiguous, then we can omit pointers

CS Sparse vs. Dense Tradeoff Sparse: Less index space per record can keep more of index in memory (Later: sparse better for insertions) Dense: Can tell if any record exists without accessing file (Later: dense needed for secondary indexes)

CS Terms Index sequential file Search key (  primary key) Primary index (on sequencing field) Secondary index Dense index (contains all search key values) Sparse index Multi-level index

CS Next: Duplicate keys Deletion/Insertion Secondary indexes

CS Duplicate keys

CS Dense index ! Point to each value ! Duplicate keys

CS Dense index. Point to each distinct value! Duplicate keys

CS Sparse index: point to start of block ! Duplicate keys careful if looking for 20 or 30!

CS Sparse index, another way ? Duplicate keys – place first new key from block should this be 40?

CS Duplicate values, primary index Index may point to first instance of each value only File Index Summary a a a b 

CS Next: Duplicate keys Deletion/Insertion Secondary indexes

CS Deletion from sparse index

CS Deletion from sparse index – delete record 40

CS Deletion from sparse index – delete record 30 40

CS Deletion from sparse index – delete records 30 &

CS 4432lecture #826 Deletion from dense index

CS Deletion from dense index – delete record 30 40

CS Insertion, sparse index case

CS Insertion, sparse index case – insert record our lucky day! we have free space where we need it!

CS Insertion, sparse index case – insert record Immediate reorganization Other variations?

CS Just Illustrated: -Immediate reorganization Now Variation: – insert new block (chained file)

CS Insertion, sparse index case – insert record overflow blocks (reorganize later...)

CS Insertion, dense index case Similar Often more expensive...

CS Next: Duplicate keys Deletion/Insertion Secondary indexes

CS Secondary indexes Sequence field Can I make a secondary index sparse ?

CS Secondary indexes Sequence field Sparse index does not make sense!

CS Secondary indexes Sequence field Must be dense index ! sparse high level allowed?

CS With secondary indexes: Lowest level is dense Other levels are sparse Also: Pointers are record pointers (not block pointers; not computed)

CS Duplicate values & secondary indexes

CS Duplicate values & secondary indexes one option... Problem: excess overhead! disk space search time

CS Duplicate values & secondary indexes another option Problem: variable size records in index!

CS Duplicate values & secondary indexes Another idea : Chain records with same key ! Problems: Need to add fields to data records for each index Need to follow chain to know records

CS Summary : Conventional Indexes –Basic Ideas: sparse, dense, multi-level… –Duplicate Keys –Deletion/Insertion –Secondary indexes

CS Multi-level Index Structures Sequence field first level (dense, if non- sequential) high Level (always sparse)

CS Sequential indexes : pros/cons ? Advantage: - Simple - Index is sequential file good for scans - Search efficient for static data Disadvantage: - Inserts expensive, and/or - Lose sequentiality & balance - Then search time unpredictable

CS ExampleSequential Index continuous free space overflow area (not sequential)

CS Another type of index Give up “sequentiality” of index Predictable performance under updates Achieve always balance of “tree” Automate restructuring under updates

CS Root B+Tree Examplen=

CS Sample non-leaf to keysto keysto keys to keys < 5757  k<8181  k<95 

CS Sample leaf node: From non-leaf node to next leaf in sequence To record with key 57 To record with key 81 To record with key 85

CS In textbook’s notationn=3 Leaf: Non-leaf:

CS Size of nodes:n+1 pointers n keys (fixed)

CS Don’t want nodes to be too empty Use at least Non-leaf:  (n+1)/2  pointers Leaf:  (n+1)/2  pointers to data

CS Full nodemin. node Non-leaf Leaf n= counts even if null Non-leaf:  (n+1)/2  pointers Leaf:  (n+1)/2  pointers to data

CS B+tree rulestree of order n (1) All leaves at same lowest level (balanced tree) (2) Pointers in leaves point to records except for “sequence pointer”

CS (3) Number of pointers/keys for B+tree Non-leaf (non-root) n+1n  (n+1)/ 2   (n+1)/ 2  - 1 Leaf (non-root) n+1n Rootn+1n11 Max Max Min Min ptrs keys ptrs  data keys  (n+ 1) / 2 

CS Root B+Tree Example : Searches

CS Insert into B+tree (a) simple case –space available in leaf (b) leaf overflow (c) non-leaf overflow (d) new root

CS (a) Insert key = 32 n=

CS (a) Insert key = 7 n=

CS (c) Insert key = 160 n=

CS (d) New root, insert 45 n= new root

CS Recap: Insert Data into B+ Tree Find correct leaf L. Put data entry onto L. – If L has enough space, done! – Else, must split L (into L and a new node L2) Redistribute entries evenly, copy up middle key. Insert index entry pointing to L2 into parent of L. This can happen recursively – To split index node, redistribute entries evenly, but push up middle key. (Contrast with leaf splits.) Splits “grow” tree; root split increases height. – Tree growth: gets wider or one level taller at top.

CS (a) Simple case (b) Coalesce with neighbor (sibling) (c) Re-distribute keys (d) Cases (b) or (c) at non-leaf Deletion from B+tree

CS (a) Delete key = 11 n=

CS (b) Coalesce with sibling –Delete n=4 40

CS (c) Redistribute keys –Delete n=4 35

CS (d) Coalese and Non-leaf coalese –Delete 37 n= new root

CS B+tree deletions in practice –Often, coalescing is not implemented –Too hard and not worth it!

CS Delete Data from B+ Tree Start at root, find leaf L where entry belongs. Remove the entry. – If L is at least half-full, done! – If L has only d-1 entries, Try to re-distribute, borrowing from sibling (adjacent node with same parent as L). If re-distribution fails, merge L and sibling. If merge occurred, must delete entry (pointing to L or sibling) from parent of L. Merge could propagate to root, decreasing height.

CS Concurrency control harder in B-Trees B-tree consumes more space DBA does not know when to reorganize DBA does not know how full to load pages of new index Buffering –B-tree: has fixed buffer requirements –Static index: must read several overflow blocks to be efficient (large & variable size buffers needed) Comparison: B-trees vs. static indexed sequential file

CS Speaking of buffering… Is LRU a good policyfor B+tree buffers?  Of course not!  Should try to keep root in memory at all times (and perhaps some nodes from second level)

CS Comparison B-tree vs. indexed seq. file Less space, so lookup faster Inserts managed by overflow area Requires temporary restructuring Unpredictable performance Consumes more space, so lookup slower Each insert/delete potentially restructures Build-in restructuring Predictable performance

CS Interesting problem: For B+tree, how large should n be? … n is number of keys / node

CS assumptions: n children per node and N records in database (1)Time to read B-Tree node from disk is (tseek + tread*n) msec. (2)Once in main memory, use binary search to locate key, (a + b log_2 n) msec (3)Need to search (read) log_n (N) tree nodes (4)t-search = (tseek + tread*n + (a + b*log_2(n)) * log n (N)

CS  Can get: f(n) = time to find a record f(n) n opt n  FIND n opt by f’(n) = 0 øWhat happens to n opt as: Disk gets faster? CPU get faster? …

CS Bulk Loading of B+ Tree For large collection of records, create B+ tree. Method 1: Repeatedly insert records  slow. Method 2: Bulk Loading  more efficient.

CS Bulk Loading of B+ Tree Initialization: – Sort all data entries – Insert pointer to first (leaf) page in new (root) page. 3* 4* 6*9*10*11*12*13* 20*22* 23*31* 35* 36*38*41*44* Sorted pages of data entries; not yet in B+ tree Root

CS Bulk Loading (Contd.) Index entries for leaf pages always entered into right- most index page When this fills up, it splits. Split may go up right-most path to root. 3* 4* 6*9*10*11*12*13* 20*22* 23*31* 35* 36*38*41*44* Root Data entry pages not yet in B+ tree * 4* 6*9*10*11*12*13* 20*22* 23*31* 35* 36*38*41*44* 6 Root not yet in B+ tree Data entry pages

CS Summary of Bulk Loading Method 1: multiple inserts. – Slow. – Does not give sequential storage of leaves. Method 2: Bulk Loading – Has advantages for concurrency control. – Fewer I/Os during build. – Leaves will be stored sequentially (and linked) – Can control “fill factor” on pages.