CS 440 Database Management Systems Lecture 6: Data storage & access methods 1

Database System Implementation 2 Conceptual Design Physical Storage Schema Entity Relationship(ER) Model Relational Model Files and Indexes User Requirements

The advantage of RDBMS It separates logical level (schema) from physical level (implementation). Physical data independence – Users do not worry about how their data is stored and processes on the physical devices. – It is all SQL! – Their queries work over (almost) all RDBMS deployments. 3

4 DBMS Architecture Query Executor Buffer Manager Storage Manager Storage Transaction Manager Logging & Recovery Lock Manager Buffers Lock Tables Main Memory User/Web Forms/Applications/DBA query transaction Query Optimizer Query Rewriter Query Parser Files & Access Methods Process manager

Challenges in physical level Processor: – MIPS Main memory: around 10 Gb/ sec. Secondary storage: higher capacity and durability Disk random access – Seek time + rotational latency + transfer time – Seek time: 4 ms - 15 ms! – Rotational latency: 2 ms – 7 ms! – Transfer time: at most 1000 Mb/ sec – Read, write in blocks. 5

Gloomy future: Moor’s law Speed of processors and cost and maximum capacity of storage increase exponentially over time. But storage (main and secondary) access time grows much more slowly. 6

Random access versus sequential access Disk random access : Seek time + rotational latency + transfer time. Disk sequential access: reading blocks next to each other – No seek time or rotational latency – Much faster than random access 7

Units of data on physical device Fields: data items Records Blocks Files 8

Fields Fixed size – Integer, Boolean, … Variable length – Varchar, … – Null terminated – Size at the beginning of the string 9

Records: sets of fields Schema – Number of fields, types of fields, order, … Fixed format and length – Record holds only the data items Variable format and length – Record holds fields and their size, type, … information Range of formats in between 10

Record header Pointer to the record schema ( record type) Record size Timestamp Other info … 11

Blocks Collection of records Reduces number of I/O access Different from OS blocks – Why should RDBMS manage its own blocks? It knows the access pattern better than OS. Separating records in a block – Fixed size records: no worry! – Markers between records – Keep record size information in records or block header. 12

Spanned versus un-spanned Un-spanned – Each records belongs to only one block Spanned – Records may be stored across multiple blocks – Saves space – The only way to deal with large records and fields: blob, image, … 13

Block header Data about block File, relation, DB IDs Block ID and type Record directory Pointer to free space Timestamp Other info … 14

Heap versus sorted files Heap files – There is not any order in the file – New blocks are inserted at the end of the file. Sorted files – Order blocks (and records) based on some key. – Physically contiguous or using links to the next blocks. 15

Average cost of data operations Insertion – Heap files are more efficient. – Overflow areas for sorted files. Search for a record or a range of records – Sorted files are more efficient. Deletion – Heap files are more efficient – Although we find the record faster in the sorted file. 16

Row versus column stores We have talked about row store – All fields of a record are stored together. 17

Row versus column stores We can store the fields in columns. – We can store SSNs implicitly. 18

Row versus column store Column store – Compact storage – Faster reads on data analysis and mining operations Row store – Faster writes – Faster reads for record access (OLTP) Further reading – Mike Stonebreaker, et al, “C-Store, a column oriented DBMS”, VLDB’05. 19

Access paths The methods that RDBMS uses to retrieve the data. Attribute value(s)  Tuple(s) 20

Types of search queries Point query over Beers(name, manf) Select * From Beers Where name = ‘Bud’; Range query over Sells(bar, beer, price) Select * From Sells Where price > 2 AND price < 10; 21

Types of access paths Full table scan – Heap files – Inefficient for both point and range queries. Sequential access – Sorted files – Efficient for both point and range queries. – Inefficient to maintain Middle ground? 22

Indexing An old idea 23

Index A data structure that speeds up selecting tuples in a relation based on some search keys. Search key – A subset of the attributes in a relation – May not be the same as the (primary) key Entries in an index – (k, r) – k is the search key. – r is the pointer to a record (record id). 24

Index Data file stores the table data. Index file stores the index data structure. Index file is smaller than the data file. Ideally, the index should fit in the main memory Data FileIndex File

Index categorizations Clustered vs. unclustered – Records are stored according to the index order. – Records are stored in another order, or not any order. Dense vs. sparse – Each record is pointed by an entry in the index. – Each block has an entry in the index. – Size versus time tradeoff. Primary vs. secondary – Primary key is the search key – Other attributes. 26

Index categorizations Clustered and dense DATAINDEX

Index categorizations Clustered and sparse DATAINDEX 70 80

Duplicate search keys Clustered and dense DATAINDEX 40 50

Duplicate search keys Clustered and sparse: – Any problem? DATAINDEX 40 50

Duplicate search keys Clustered and sparse: – Point to the lowest new search key in every block DATAINDEX 40 50

Unclustered Index Dense / sparse? DATAINDEX 10 40

Well known index structures B+ trees: – very popular Hash tables: – Not frequently used 33

B+ trees The index of a very large data file gets too large. How about building an index for the index file? A multi-level index, or a tree 34

B+ trees Degree (order) of the tree: d Each node (except root) stores [d, 2d] keys: [A, 10)[10, 32)[32, 94)[94, B) Non-leaf nodes Leaf nodes Records

36 Example d = 2

B+ tree tuning How to choose the value of d? – Each node should fit in a block. Example – Key value: 8 byte – Record pointer: 16 bytes – Block size: 4096 bytes – 2d * 8 + (2d + 1) * 16 <= 4096 – d <= 85 37

Retrieving tuples using B+ tree Point queries – Start from the root and follow the links to the leaf. Range queries – Find the lowest point in the range. – Then, follow the links between the nodes. The top levels are kept in the buffer pool. 38

B+ tree and index categories B+ tree index could be – Dense / sparse? – Clustered/ unclustered? 39

Inserting a new key Pick the proper leaf node and insert the key. If the node contains more than 2d keys, split the node and insert the extra node in the parent. – If leaf level, add K3 to the right node 40 K1K2K3K4K5 R0R1R2R3R4R5 K1K2 R0R1R2 K4K5 R3R4R5 (K3, ) parent

41 Insertion Insert K = 18

42 Insertion Insert K = 18 18

43 Insertion Insert K=

44 Insertion Need to split the node

45 Insertion Split and update the parent node. What if we need to split the root?

46 Deletion Delete K =

47 Deletion Note: K = 21 may still remain in the internal levels

48 Deletion Delete K =

49 Deletion We need to update the number of keys on the node: Borrow from siblings: rotate

50 Deletion We need to update the number of keys on the node: Borrow from siblings: rotate

51 Deletion We need to update the number of keys on the node: Borrow from siblings: rotate

52 Deletion What if we cannot borrow from siblings? Example: delete K =

53 Deletion What if we cannot borrow from siblings? Merge with a sibling

54 Deletion What if we cannot borrow from siblings? Merge siblings!

55 Deletion What to do with the dangling key and pointer? simply remove them

56 Deletion Final tree

Index creation CREATE TABLE Person(Name varchar(50), Pos int, Age int); CREATE INDEX Person_ID ON Person(ID); CLUSTER Person USING ON Person_ID; CREATE INDEX Pos_Age ON Person(Pos, Age); 57 Default is normally B-tree. Cluster Person_ID index Multi-attribute index

Index selection Let’s index every attribute on every table to speed up all queries! Indexes generally slow down data manipulation – INSERT, DELETE, UPDATE. 58

Index selection Given a query workload and a schema, find the set of indexes that optimize the execution. The query workload: – Queries and their frequencies. – Queries are both data retrieval (SELECT) and data manipulation (INSERT, UPDATE, DELETE). 59

Index selection Part of physical database design – File structure, indexing, tuning queries,… Physical database design may affect logical design! – Change the schema to run the queries faster 60

Index selection Generally a hard problem. RDBMS vendors provide wizards: – Started with AutoAdmin project for SQL Server – SQL Server/ Oracle Index Tuning Wizard – DB2 Index Advisor They try many configurations and pick the one that minimizes the time and overheads. 61