Relational Database Systems 2 Instructor: Prof. James Cheng Acknowledgement: The slides are extracted from and modified based on the slides provided by Prof. Sourav S. Bhowmick from Nanyang Technological University.

Topics to be covered ER model Relational Algebra SQL Storage and Index Structures Query Processing and Query Optimization

Disks and Files DBMS stores information on “hard” disks Implications on DBMS Design READ: transfer data from disk to main memory (RAM) (in msec) WRITE: transfer data from RAM to disk (in msec) Both are high-cost operations, relative to in-memory operations, so must be planned carefully!

Why Not Store Everything in Main Memory? Main memory is volatile We want data to be saved between runs. Cost too much Main memory is much more expensive! Memory Hierarchy Main memory (RAM) for currently used data Disk for the main database (secondary storage) Tapes for archiving older versions of the data (tertiary storage)

Disks Data is stored and retrieved in units called disk blocks or pages Disk consists of a sequence of blocks All blocks are of the same size A disk block is essentially the same size as a virtual memory page Physical unit of access is always a block even if only a single bit needs to be accessed

Pages and Blocks Pages Blocks Data files are decomposed into pages These are fixed size pieces of contiguous information in the file (usually multiples of 512, 1024, or 2048 bytes) A page is the unit of exchange between disk and main memory (typical page size is 4096 bytes) Blocks Disk are divided into page size blocks of storage A page can be stored in any disk block

? Accessing a Disk Block I want block x block X in memory Time to access a disk block Seek time: Moving arms to position disk head on track (10-20 ms) Rotational delay: Waiting for block to rotate under head (5-10 ms) Transfer time: Actually moving data to/from disk surface Key to lower I/O cost: Reduce seek/rotation delays!

Arranging Pages on Disk Minimizing time cost Blocks in a file should be arranged sequentially on disk For a sequential scan, pre-fetching several pages at a time is advantageous!

Basic Search Value-based search SELECT * FROM R WHERE A = value; SELECT * FROM R, S WHERE R.A = S.B; Range search SELECT * FROM R WHERE A > value; Keyword search SELECT * FROM R WHERE A LIKE value;

What is an index? A data structure Input Output What it does? A property of records (value of one or more fields) Finds the records with that property “quickly” What it does? Index let us find records without having to look at more than a small fraction of all possible records

Index Block holding records Value Index Matching records

B-TREES Structure of B-trees Lookup and range queries Insertion of a node Deletion of a node

B-Trees What is it? Our Focus A type of index (tree) Nonsequential, “balanced” (access paths to different records of equal length) Adapts well to insertions & deletions Consists of blocks holding at most n keys and n+1 pointers, and at least half of this Our Focus We consider a variation called a B+ tree

Example (B+ tree of order p) Non-leaf levels Root p=4 pleaf=3 Non-leaf node 100 Block pointers Leaf node 11 3 5 100 30 35 101 110 120 130 150 156 179 180 200 205 Leaf level 110 150 180 11 Sequential link Pointers in leaf node Non-leaf node point to data blocks (except for pointers that link leaf nodes (index blocks) sequentially) point to nodes (which can be (non-leaf index) blocks once the tree is stored on disk)

Block Pointers in a B+-tree 100 110 150 180 11 3 5 30 35 101 120 130 156 179 200 205 Index in memory Relation on disk … 3 150 100 130 156 120

Leaf Nodes of B+ Tree An entry for every value of the search key If the search key is a primary key Data pointer to the record or to the block containing the record Otherwise Data pointer to a block containing pointers to the data file records Nodes are linked together to provide ordered access

Sample Leaf Node 25 28 Next leaf node 36 To record with k=36 To record

Sample Non-Leaf Node 25 28 31 To keys 28<= K < 31 To keys

Don’t want nodes to be too empty Each block has n search keys Number of pointers At internal nodes at least (n+1)/2 (to child nodes) at leaves at least (n+1)/2 (to data records/blocks)

1 if only one record in the file Summary Max Ptr Keys Min Ptr->data Min Keys Non-leaf (Non-root) Leaf (Non-root) Root n+1 n (n+1)/2 (n+1)/2- 1 n+1 n (n+1)/2 (n+1)/2 1 if only one record in the file n+1 n 2(*) 1

Search 12 10 4 19 21 1 7 10 17 19 20 21 25 28 4 31

Search 28 10 4 19 21 1 7 10 17 19 20 21 25 28 4 31

Usefulness in Query Processing Queries in which a single value of search K is required Range queries SELECT * FROM R WHERE R.k > 20; SELECT * FROM R WHERE R.k >= 17 AND R.k <= 26 ;

R.K > 20 10 4 19 21 1 7 10 17 19 20 21 25 28 4 31

R.K >= 17 AND R.k <= 26 10 4 19 21 1 7 10 17 19 20 21 25 28 4 31

B+ Tree Insertion Problem Construct a B+ tree index of order 4 Database Systems Problem Construct a B+ tree index of order 4 1, 4, 7, 10, 17, 21, 31, 25, 19, 20, 28, 42 n = 3 25 28 31 To keys 28<= K < 31 To keys K < 25 To keys 25<= K < 28 To keys K >= 31

Insert 1, 4, 7 Database Systems Step 1 1 Find a place for the new key in the appropriate leaf and put it there if there is space 1 4 1 4 7

Insert 10 Database Systems Step 2 If there is no space in the leaf we split the leaf into two and divide the keys between the two new nodes so each is half full or just over half full 1 4 7 Overflow Need to split 10

Leaf Node Split Step 1 Step 2 1 4 7 10 1 4 7 10 Database Systems Step 1 Step 2 Create a new node M, which will be the sibling of N, immediately to its right The first j = (n + 1)/2) key-pointer pairs, in sorted order, remains with N Remaining ones are moved to M 1 4 7 Overflow Need to split 10 J= 4/2 = 2 1 4 7 10

Leaf Node Split Step 3 Step 4 7 1 7 10 4 Database Systems Step 3 Step 4 Extra pointer to the new node is created in the parent Must be inserted in the parent in the correct sequence (j+1)th search value is replicated in the parent node (j+1)th search value replicated in parent node 7 1 7 10 4

Insert 17 Database Systems 7 1 10 4 17

Insert 21 7 1 10 4 17 21 Step 2 Overflow Need to split Database Systems 7 1 10 4 17 21 Step 2 Overflow Need to split The first j = (n + 1)/2) key-pointer pairs, in sorted order, remains with N Remaining ones are moved to M J= 4/2 = 2

Insert 21 7 1 10 4 17 21 17 Step 3 Step 4 (J+1)th search Database Systems 7 1 10 4 17 21 (J+1)th search value replicated in parent node 17 Step 3 Step 4 Extra pointer to the new node is created in the parent Must be inserted in the parent in the correct sequence (j+1)th search value is replicated in the parent node

Insert 31 Database Systems 7 1 10 4 17 21 31

Insert 25 25 7 1 10 4 17 21 31 Overflow Need to split J= 4/2 = 2 Database Systems 25 7 1 10 4 17 21 31 Overflow Need to split J= 4/2 = 2

Insert 25 7 1 10 4 17 21 25 31 25 (J+1)th search value replicated Database Systems 7 1 10 4 17 21 25 31 (J+1)th search value replicated in parent node 25

Insert 19 Database Systems 7 1 10 4 17 21 25 31 25 19 7 1 10 4 17 19 25 31 25 21

Insert 20 7 1 10 4 17 19 25 31 20 25 21 Overflow Need to split J= 4/2 Database Systems 7 1 10 4 17 19 25 31 20 25 21 Overflow Need to split J= 4/2 = 2

Insert 20 7 1 10 17 19 20 21 25 31 20 4 Principle root needs Database Systems 7 1 10 17 19 20 21 25 31 root needs to be split 20 4 Insert a pointer to the new leaf into the node above it. Unfortunately, the parent is full! Principle If we try to insert into the root, and there is no room, then we split the root into two nodes and create a new root at the next higher level

Non-Leaf Node Split Step 1 (Node creation) Step 2 (Pointer movement) Database Systems Step 1 (Node creation) Step 2 (Pointer movement) Create a new node M, which will be the sibling of N, immediately to its right Leave at N the first j=(n + 2)/2) pointers , in sorted order Move to M the remaining (n+2)/2 pointers Step 3 (Key movement) The first n/2 keys stay with N, while the last n/2 keys move to M 7 17 25 1 7 10 17 19 20 21 25 31 4

Non-Leaf Node Split Step 4 (Key Movement to the Parent) Database Systems Step 4 (Key Movement to the Parent) One key (say K) always in the middle left over Goes neither to M nor to N K -> smallest key reachable via the first of M’s children K will be used by the parent of N and M to divide searches between these two nodes 20 7 17 25 1 7 10 17 19 20 21 25 31 4

Insert 28 Database Systems 20 7 17 25 1 7 10 17 19 20 21 4 25 31 28

Insert 42 (Leaf node split) Database Systems 20 42 7 17 25 1 7 10 17 19 20 21 4 25 28 31 Overflow Need to split J= 4/2 = 2

Insert 42 (Leaf Node Split) Database Systems 20 7 17 25 31 1 17 19 20 21 31 42 4 7 10 25 28

B+-Tree Deletion (Delete 5) Database Systems Step 1 Step 2 Locate the record to be deleted and its key-pointer pair in the leaf Delete the record itself from the data file and delete the key-pointer pair from the tree Step 3 If the node from which a deletion occurred still has at least the minimum number of keys and pointers then there is nothing more to be done 20 7 17 25 1 7 10 17 19 20 21 4 5 25 31

Minimum-key rule violation Database Systems Case 1 If one of the adjacent siblings of node N has more than the minimum number of keys and pointers then one key-pointer pair can be moved to N Adjust parent key Delete 17 20 7 17 16 25 1 7 10 20 21 4 5 16 17 19 25 31

Minimum-key rule violation Database Systems Case 2 Neither adjacent sibling can be used Merge the two nodes, deleting one of them Adjust the parent Delete 4 20 7 17 25 1 7 10 17 19 20 21 4 7 10 25 31

Efficiency of B+-tree No. of records No. of Disk I/O Assume an average node has 255 pointers A three-level B-tree has 2552 = 65025 leaves with total of 2553 or about 16.6 million pointers to records No. of Disk I/O If root block kept in main memory Each record can be accessed with 2+1 disk I/Os; If all 256 internal nodes are in main memory, record access requires 1+1 disk I/Os (256 x 4 KB = 1 MB; quite feasible!)

HASHING Static Hashing Dynamic Hashing

Hashing What does the hash function do? key  h(key) A hash function takes a search key (hash key) as an input Computes an integer between 0 to B – 1 (B is the number of buckets) key  h(key) <key> Buckets (typically 1 disk block) .

Main-memory Hash Table Bucket Array An array indexed from 0 to B – 1 Holds the header of B linked lists one for each bucket Storage of Record If a record has a search key K then we store the record by linking it to the bucket list for the bucket numbered h(K)

Main memory Hash Table 1 2 3 4 5

Secondary-storage Hash Table Bucket Array Consists of blocks rather than pointers to the header of lists Storage of Record Records that are hashed by the hash function h to a certain bucket are put in the block for that bucket What happens when buckets overflow? Chain of overflow blocks can be added to the bucket to hold more records

Classification Static hash table Dynamic hash table Extensible hashing Number of buckets is fixed and does not change Number of buckets is allowed to vary About one block per bucket Extensible hashing Linear hashing Grows the number of buckets by doubling it Number of buckets grow by 1

Static Hashing - Example Assume that index structure has 8 buckets reserved Each bucket takes only 2 search keys Hashing function: h(k) = h mod 8 1 2 3 4 5 6 7

Insert 3, 23, 19 1 2 3 4 5 6 7 3 23 19 3%8 = 3 23%8 = 7 19%8 = 3

Insert 27, 31, 39 1 2 3 4 5 6 7 3 23 19 31 27 39

Find the record with search key 27 1 2 3 4 5 6 7 3 23 19 31 27 39 Two blocks

Find the record with key >= 27 1 2 3 4 5 6 7 3 23 19 31 Does not Support range queries 27 39 10 blocks

Deletion 27, 29, 3 1 2 3 4 5 6 7 3 20 5 23 19 29 31 39 27 21

Rule of Thumb for Static Hashing Utilization Try to keep space utilization between 50% and 80% Utilization = # keys used total # keys that fit Rule of thumb If < 50%, wasting space If > 80%, overflows significant depends on how good hash function is & on # keys/bucket

How do we cope with growth? Overflows and reorganizations Dynamic hashing: # of buckets may vary Extensible Linear also others ...

Extensible hashing What is it? Rules Array of pointers to blocks represents the buckets, instead of arrays containing data blocks Rules Array of pointers can grow and its length is always a power of 2 There does not have to have a data block for each bucket Certain buckets can share a block

Key ideas 00110101 Principle h(K) The hash function h computes for each key a sequence of k bits for some large k The bucket numbers will at all times use some smaller number of bits, say i bits, from the beginning of the sequence k For example, k=32 00110101 h(K) i (grows with time)

Key ideas Principle h(K)[i ] to bucket The bucket array will have 2i entries when i is the number of bits used h(K)[i ] to bucket . Each bucket stores j, indicating #bits used for placing the records in this block (j  i) .

Insertion of a record with key K Step 1 Step 2 Compute h(K) Take the first i bits of this bit sequence Step 3 Go to the entry of the bucket array indexed by these i bits Follow the pointer and arrive at block B Step 4 If B is not full then insert the record

i = 1 001 1 Block is full 100 101 Variable j 111 001 1 1 Block is full 100 101 Variable j 111 Let j indicates how many bits of the hash value are used to determine membership in block B

Case 1 (j < i) Step 1 Step 2 Step 3 Split B into two Distribute records in B to the two blocks based on the value of their (j+1)st bit Record whose key has 0 in that bit stays in B Those with 1 there goes to the new block Step 2 Put j+1 in each block’s “nub” to indicate number of bits used to determine membership Step 3 Adjust the pointers in the bucket array so entries that formerly point to B now point either to B or to new block depending on their (j+1)st bit Recursively if necessary

Case 2 (j = i) Step 1 Step 2 Step 3 Step 4 Step 5 Increment i by 1 Double the length of the bucket array (2i+1) Step 3 Step 4 Suppose w is a sequence of i bits indexing one of the entries in the previous bucket array In the new bucket array entries are indexed by w0 and w1 Each point to the same block that w used to point to Membership in the block is still determined by whatever number of bits was previously used Step 5 We proceed to split B as in case 1 as (i > j)

Insertion of 111 j=i=1 Step 3 Step 1 Step 2 001 1 100 101 00 001 01 001 j=i=1 1 1 100 101 Step 3 00 1 Step 1 001 Suppose w is a sequence of i bits indexing one of the entries in the previous bucket array In the new bucket array entries are indexed by w0 and w1 Each point to the same block that w used to point to Step 2 Increment i by 1 01 1 100 101 Double the length of the bucket array (2i+1) 10 i=2 j< i Case 1 is true 11

Insertion of 111 Step 3 Step 1 Step 2 00 001 01 i=2 100 101 j=1 10 111 Split B into two Distribute records in B to the two blocks based on the value of their (j+1)st bit Record whose key has 0 in that bit stays in B Those with 1 there goes to the new block Adjust the pointers in the bucket array so entries that formerly point to B now point either to B or to new block depending on their (j+1)st bit Recursively if necessary Put j+1 in each block’s “nub” to indicate number of bits used to determine membership 1 00 001 01 i=2 2 100 101 j=1 10 2 111 11

Deletion (Eg: 0001) Cannot be combined 0001 1001 1010 1100 00 01 10 11 2 0001 1001 1010 1100 00 01 10 Cannot be combined 1 11 2 1001 1010 2 1100

Deletion (Eg: 1100) 00 1 1 1 01 1 1 1001 1010 2 2 1001 1001 1010 1010 10 2 1100 11 Steps Delete value and block pointer Empty bucket can be combined with neighbour Size of array is reduced

Pros and Cons Pros Can handle growing files - without full reorganizations Only one data block examined Cons Indirection (Not bad if directory in memory) Array doubles in size(First it fits in memory, then it does not -> sudden performance degradation)

Indexing vs Hashing Hashing Indexing Hashing good for probes given key SELECT … FROM R WHERE R.A = 5 Indexing Good for range searches SELECT …. FROM R WHERE R.A > 5