1. Spring 2003 ECE569 Lecture 05.1 ECE 569 Database System Engineering Spring 2003 Yanyong Zhang www.ece.rutgers.edu/~yyzhangwww.ece.rutgers.edu/~yyzhang Course URL www.ece.rutgers.edu/~yyzhang/spring03www.ece.rutgers.edu/~yyzhang/spring03

2. Spring 2003 ECE569 Lecture 05.2 Index  “If you don’t find it in the index, look very carefully through the entire catalog”  An index is a data structure that organizes data records on disk to optimize certain kind of retrieval operations.  A data entry refers to the records stored in an index file. A data entry with search key k, denoted as k*, contains enough information to locate (one or more) data records with search key value k. Three alternatives: 1. K* can be an actual data record 2. K* is a (k, tid) pair 3. K* is a (k, tid-list) pair

3. Spring 2003 ECE569 Lecture 05.3 Clustered Indexes  When is a file is organized so that the ordering of data records is the same as or close to the ordering of data entries in some index, we say that the index is clustered. l Alternative (1) is clustered l Alternatives (2) and (3) can be a clustered index only if the data records are sorted on the search key field => this is expensive => usually they are unclustered.

4. Spring 2003 ECE569 Lecture 05.4 Index Data Structures  Two basic approaches l Hash-based indexing l Tree-based indexing -ISAM tree -B + tree

5. Spring 2003 ECE569 Lecture 05.5 Indexed Sequential Access Method (ISAM)  Highly static  Each node is a disk page  Leaf nodes are first allocated, then index pages, then overflow pages  Once the ISAM file is created, inserts and deletes affect only the contents of leaf pages. Index pages leaf pages overflow pages primary pages

6. Spring 2003 ECE569 Lecture 05.6 ISAM lookup 40 20335163 10*15*20*27*33*37*40*46*51*55*63*97*  Primary leaf pages are allocated sequentially?  Is this assumption reasonable?  no “next-leaf” pointer is necessary

7. Spring 2003 ECE569 Lecture 05.7 ISAM insert 40 20335163 10*15*20*27*33*37*40*46*51*55*63*97*  Insert 23, 48, 41, 42 23*48*41* 42*

8. Spring 2003 ECE569 Lecture 05.8 ISAM delete  Removes the entry  If the page becomes empty l If it is overflow page, then delete it l If it is primary page, just leave it as a place holder

9. Spring 2003 ECE569 Lecture 05.9 ISAM discussion  Pros l We know that the index nodes will not be changed, so that we don’t need to lock them  Cons l Long chains of overflow pages are performance bottleneck

10. Spring 2003 ECE569 Lecture 05.10 B + -tree  The tree grows/shrinks dynamically  Root index fits in one page and directs search for records in index below it  B + -tree is balanced, i.e., every path through tree is same length. Reasonably easy to maintain this property  Large fan-out of index nodes result in few levels. Three levels can address 16M pages (256 records / page)  depth Index entries (to direct search) data entries

11. Spring 2003 ECE569 Lecture 05.11 Format of a node  Index node l An index node contains m entries, with d  m  2d. d is called the order of the tree. The root node is required to have 1  m  2d. l p 0 K 1 p 1 K 2 p 2 … K m p m  Leaf node l Leaf nodes contain the data entries. l A page contains at most 2e-1 records l Records sorted by key value l Doubly linked list

12. Spring 2003 ECE569 Lecture 05.12 Lookup of key K  Assume B + -tree is of depth l  Construct path B 0 B 1 …B l-1 where l B 0 is root node l K j in block B i-1 covers K and j th block pointer in B i-1 is B i.  Example – Find key K = 245 168296 140 220256 303 120136140151168170190220255256271296299303312318 l Path is B0 (168) B1 (220) B7 l 245 is not in B7 => 245 is not in main file B0 B1B2B3 B4 B5 B6B7B8B9B10

13. Spring 2003 ECE569 Lecture 05.13 Insertion of record with key K  Follow lookup procedure to find block in which K belongs. Path is B 0 B 1 …B l-1 l If room in B l-1, then insert there. (Maintain sorted order of B l-1 ) l Otherwise, allocate B’ and split records evenly between B l-1 and B’ l Keys in B l-1 are less than K’ and those in B’ are greater than or equal to K’ -Insert record (K’, B’) in Block B l-1. (This insertion can also cause split) l Splitting the root (maintain B0 as the root) -Allocate two new blocks B l and B r. -Move half of keys in root (keys smaller than K’) to B l and the rest (keys greater than or equal to K’) to B r. -Modify B0 (original root) to contain (B l, K’, B r ) -Depth is increased to l+1

14. Spring 2003 ECE569 Lecture 05.14 Delete record with key K  lookup K and find path B 0 B 1 …B l-1  Delete record from B l-1  If B l-1 now contains fewer than e records l If a neighbor B’ has more than e records, divide the records between B l-1 and B’ as evenly as possible. Update any ancestors necessary to reflect change. l Otherwise, the records of B l-1 and one of its neighbors B’ can be combined. B’ is removed and (K’, B’) is removed from parent. This merge can propagate to the root. l If last two children of root are combined, depth is decreased.

15. Spring 2003 ECE569 Lecture 05.15 Discussion  Merge operations have a high performance penalty; databases tend to grow, so some merges may not be necessary l Remove blocks when empty l Treat merge as a maintenance operation, and do it periodically  What kind of queries can B + -trees help with?

16. Spring 2003 ECE569 Lecture 05.16 Dense Indices  Decouple allocation of tuples from access method l Allocate tuples following a heap organization (good utilization) l Access tuples using hashing, B + -tree, etc.  Access methods must be modified slightly l B + -trees:Keys adjacent in key space need not be physically adjacent. Need tuple pointer for each key value (not key range) in leaf nodes. (each tuple can be in a different page) l Hashing: Hash buckets contain key value, tuple pointer pairs. 168220 140 175 296303 168170

17. Spring 2003 ECE569 Lecture 05.17 Secondary Indices  Primary indices provide access based on primary key  Secondary indices provide access based on search fields other than primary key  Index can be used to cluster tuples l Sparse B-tree can be easily modified 168220 140 175 296303 168170168 175 200180187200

18. Spring 2003 ECE569 Lecture 05.18 Secondary Indexes (cont’d)  Non-clustered indexes 168220 140 175 296303 168170175200180187

19. Spring 2003 ECE569 Lecture 05.19 Performance  Lookup requires l accesses where l is depth  The depth is directly dependant on the fanout of index nodes  Define (sparse B + -tree) – l n = number of records l R = number of records / block (max) l F = number of index entries / block (max0 l u = average node occupancy l R eff = R  u = average number of records / page l F eff = F  u = average number of index entries / page  N  F l-1 eff  R eff   log F eff ( n / R eff )  + 1 = l

20. Spring 2003 ECE569 Lecture 05.20 Performance – cont’d  Utilization l If nodes are merged as described above u  69% l If nodes are removed when empty -# inserts = # deletes, u  40% -60% inserts, 40% deletes, u  60%

21. Spring 2003 ECE569 Lecture 05.21 Example  4000 bytes / block  200 bytes / record  Key requires 20 bytes  Block pointer requires 4 bytes  n = 1000000  What is the depth? ( 4)

22. Spring 2003 ECE569 Lecture 05.22 Key compression  Tree height can be reduced by increasing fan-out of index nodes  Key compression can increase the number of keys that can be stored in an index node l Suffix compression -Store only enough of the key value to discriminate between the children of the index node. For the following keys artful deliver hand access alert amassartful boom dealDeliver everyone fiddleHand integral leaf -Only need to store the following ar del h access alert amassartful boom dealDeliver everyone fiddleHand integral leaf

23. Spring 2003 ECE569 Lecture 05.23 Key compression (cont’d) l Prefix compression -Rather than storing each key value, store the difference from the previous key value -Represent key i as where j is the length of the common prefix shared by key i and key i-1, and key i ’ is the remainder of key i after the common prefix is removed -The following keys (length 36 bytes) – can, cannon, canter, cantor, capacity, capital – can be encoded as (length 27 bytes) -,,,,, -How would you decide which of these two techniques to use in a particular situation?