1. Indexing
CENG 351 File Structures

2. What is an Index?
A simple index is a table containing an ordered list of keys and reference fields.
e.g. the index of a book
Simple indexes are represented using simple arrays of structures that contain keys and references.
In general, indexing is another way to handle the searching problem.
CENG 351 File Structures

3. Uses of an index
An index lets us impose order on a file without rearranging the file.
Indexes provide multiple access paths to a file.
e.g. library catalog providing search for author,book and title
An index can provide keyed access to variable-length record files.
CENG 351 File Structures

4. A simple index for a pile file
Label ID Title Artist 17 62
117
152
LON|2312|Symphony No.9| Beethoven|Giulini
RCA|2626|Romeo and Juliet|Prokofiev|Maazel
WAR|23699|Nebraska| …
ANG|3795|Violin Concerto| …
Address of record
Primary key = company label + record ID.
Index is sorted (in main memory).
Records appear in file in the order they entered.
CENG 351 File Structures

5. Index array: Key Reference ANG3795 152 LON2312 17 RCA2626 62 WAR23699
117
How to search for a recording with given LABEL ID?
Binary search in the index and then seek for the record in position given by the reference field.
CENG 351 File Structures

6. Operations to maintain an indexed file
Create the original empty index and data files.
Load the index file into memory before using it.
Rewrite the index file from memory after using it.
Add data records to the data file.
Delete records from the data file
Update records in the data file.
Update the index to reflect changes in the data file
CENG 351 File Structures

7. Record Addition Append the new record to the end of the data file.
Insert a new entry to the index in the right position.
needs rearrangement of the index: we have to shift all the entries with keys that are larger than the inserted key and then place the new entry in the opened space.
Note: this rearrangement is done in the main memory.
CENG 351 File Structures

8. Record Deletion
This should use the techniques for reclaiming space in files when deleting records from the data file.We must delete the corresponding entry from the index:
Shift all records with keys larger than the key of the deleted record to the previous position; or
Mark index entry as deleted.
CENG 351 File Structures

9. Record Updating There are two cases to consider:
The update changes the value of the key field:
Treat this as a deletion followed by an insertion
The update does not affect the key field:
If record size is unchanged, just modify the data record. If record size is changed treat this as a delete/insert sequence.
CENG 351 File Structures

10. Problems with simple indexes
If index does not fit in memory:
Seeking the index is slow (binary search):
We don’t want more than 3 or 4 seeks for a search.
Insertions and deletions take O(N) disk accesses.
Index rearrangement requires shifting on disk N
Log(N+1)
15 keys 4
1000
~10
100,000
~17
1,000,000
~20
CENG 351 File Structures

11. Alternatives Two main alternatives:
Hashed organization (when access speed is a top priority)
Tree-structured (multi-level) index such as B+trees.
CENG 351 File Structures

12. Multilevel Indexing and B+ Trees
CENG 351 File Structures

13. Indexed Sequential Files
Provide a choice between two alternative views of a file:
Indexed: the file can be seen as a set of records that is indexed by key; or
Sequential: the file can be accessed sequentially (physically contiguous records), returning records in order by key.
CENG 351 File Structures

14. Example of applications
Student record system in a university:
Indexed view: access to individual records
Sequential view: batch processing when posting grades
Credit card system:
Indexed view: interactive check of accounts
Sequential view: batch processing of payments
CENG 351 File Structures

15. The initial idea Maintain a sequence set:
Group the records into blocks in a sorted way.
Maintain the order in the blocks as records are added or deleted through splitting, concatenation, and redistribution.
Construct a simple, single level index for these blocks.
Choose to build an index that contain the key for the last record in each block.
CENG 351 File Structures

16. Maintaining a Sequence Set
Sorting and re-organizing after insertions and deletions is out of question. We organize the sequence set in the following way:
Records are grouped in blocks.
Blocks should be at least half full.
Link fields are used to point to the preceding block and the following block (similar to doubly linked lists)
Changes (insertion/deletion) are localized into blocks by performing:
Block splitting when insertion causes overflow
Block merging or redistribution when deletion causes underflow.
CENG 351 File Structures

17. Example: insertion Block size = 4 Key : Last name ADAMS … BIXBY …
CARSON …
COLE …
Block 1
Insert “BAIRD …”:
Block 1
ADAMS …
BAIRD …
BIXBY …
CARSON ..
COLE …
Block 2
CENG 351 File Structures

18. Example: deletion ADAMS … BAIRD … BIXBY … BOONE … Block 1 Block 2
BYNUM…
CARSON ..
CARTER ..
DENVER…
ELLIS …
Block 3
Block 4
COLE…
DAVIS
Delete “DAVIS”, “BYNUM”, “CARTER”,
CENG 351 File Structures

19. Add an Index set BERNE 1 CAGE 2 DUTTON 3 EVANS 4 FOLK 5 GADDIS 6
Key Block
BERNE 1
CAGE 2
DUTTON 3
EVANS 4
FOLK 5
GADDIS 6
CENG 351 File Structures

20. Tree indexes This simple scheme is nice if the index fits in memory.
If index doesn’t fit in memory:
Divide the index structure into blocks,
Organize these blocks similarly building a tree structure.
Tree indexes:
B Trees
B+ Trees
Simple prefix B+ Trees …
CENG 351 File Structures

21. Separators 1 ADAMS-BERNE Block Range of Keys Separator BOLEN
2 BOLEN-CAGE
CAMP
3 CAMP-DUTTON
EMBRY
4 EMBRY-EVANS
FABER
5 FABER-FOLK
FOLKS
6 FOLKS-GADDIS
CENG 351 File Structures

22. root 1 3 4 6 2 5 Index set EMBRY BOLEN CAMP FABER FOLKS ADAMS-BERNE
CAMP-DUTTON
EMBRY-EVANS
FOLKS-GADDIS 1 3 4 6
BOLEN-CAGE
FABER-FOLK 2 5
CENG 351 File Structures

23. B Trees
B-tree is one of the most important data structures in computer science.
What does B stand for? (Not binary!)
B-tree is a multiway search tree.
Several versions of B-trees have been proposed, but only B+ Trees has been used with large files.
A B+tree is a B-tree in which data records are in leaf nodes, and faster sequential access is possible.
CENG 351 File Structures

24. Formal definition of B+ Tree Properties
Properties of a B+ Tree of order v :
All internal nodes (except root) has at least v keys and at most 2v keys .
The root has at least 2 children unless it’s a leaf..
All leaves are on the same level.
An internal node with k keys has k+1 children
CENG 351 File Structures 17

25. B+ tree: Internal/root node structure
P0 K1 P1 K ……………… Pn-1 Kn Pn
Each Pi is a pointer to a child node; each Ki is a search key value
# of search key values = n, # of pointers = n+1
Requirements:
K1 < K2 < … < Kn
For any search key value K in the subtree pointed by Pi,
If Pi = P0, we require K < K1
If Pi = Pn, Kn  K
If Pi = P1, …, Pn-1, Ki  K < Ki+1
CENG 351 File Structures

26. B+ tree: leaf node structure
L K1 r1 K ……………… Kn rn R
Pointer L points to the left neighbor; R points to the right neighbor
K1 < K2 < … < Kn
v  n  2v (v is the order of this B+ tree)
We will use Ki* for the pair <Ki, ri> and omit L and R for simplicity
CENG 351 File Structures

27. Example: B+ tree with order of 1
Each node must hold at least 1 entry, and at most 2 entries
10*
15*
20*
27*
33*
37*
40*
46*
51*
55*
63*
97* 20 33 51 63 40
Root
CENG 351 File Structures 6

28. Example: Search in a B+ tree order 2
Search: how to find the records with a given search key value?
Begin at root, and use key comparisons to go to leaf
Examples: search for 5*, 16*, all data entries >= 24* ...
The last one is a range search, we need to do the sequential scan, starting from the first leaf containing a value >= 24.
Root 17 24 30 2* 3* 5* 7*
14*
15*
19*
20*
22*
24*
27*
29*
33*
34*
38*
39* 13
CENG 351 File Structures 10

29. Cost for searching a value in B+ tree
Typically, a node is a page (block or cluster)
Let H be the height of the B+ tree: we need to read H+1 pages to reach a leaf node
Let F be the (average) number of pointers in a node (for internal node, called fanout )
Level 1 = 1 page = F0 page
Level 2 = F pages = F1 pages
Level 3 = F * F pages = F2 pages
Level H+1 = …….. = FH pages (i.e., leaf nodes)
Suppose there are D data entries. So there are D/(F-1) leaf nodes
D/(F-1) = FH. That is, H = logF( ) 1 2
level
Height
CENG 351 File Structures

30. B+ Trees in Practice Typical order: 100. Typical fill-factor: 67%.
average fanout = 133 (i.e, # of pointers in internal node)
Can often hold top levels in buffer pool:
Level 1 = page = Kbytes
Level 2 = pages = Mbyte
Level 3 = 17,689 pages = 133 MBytes
Suppose there are 1,000,000,000 data entries.
H = log133( /132) < 4
The cost is 5 pages read
CENG 351 File Structures

31. How to Insert a Data Entry into a B+ Tree?
Let’s look at several examples first.
CENG 351 File Structures 6

32. Inserting 16*, 8* into Example B+ tree
Root 13 17 24 30 2* 3* 5* 7* 8*
14* 15*
16*
You overflow 2* 5* 7* 3* 17 24 30 13 8*
One new child (leaf node) generated; must add one more pointer to its parent, thus one more key value as well.
CENG 351 File Structures 10

33. Inserting 8* (cont.) Copy up the middle value (leaf split) 13 17 24 30
Entry to be inserted in parent node. 5
(Note that 5 is
s copied up and
continues to appear in the leaf.) 2* 3* 5* 7* 8*
You overflow!
CENG 351 File Structures 12

34. Insertion into B+ tree (cont.)
Understand difference between copy-up and push-up
Observe how minimum occupancy is guaranteed in both leaf and index pg splits.
We split this node, redistribute entries evenly, and push up middle key. 
appears once in the index. Contrast
Entry to be inserted in parent node.
this with a leaf split.) 5 24 30 17 13
(Note that 17 is pushed up and only
CENG 351 File Structures 12

35. Example B+ Tree After Inserting 8*
Root 17 5 13 24 30 2* 3* 5* 7* 8*
14*
15*
19*
20*
22*
24*
27*
29*
33*
34*
38*
39*
Notice that root was split, leading to increase in height.
CENG 351 File Structures 13

36. Inserting a Data Entry into a B+ Tree: Summary
Find correct leaf L.
Put data entry onto L.
If L has enough space, done!
Else, must split L (into L and a new node L2)
Redistribute entries evenly, put middle key in L2
copy up middle key.
Insert index entry pointing to L2 into parent of L.
This can happen recursively
To split index node, redistribute entries evenly, but push up middle key. (Contrast with leaf splits.)
Splits “grow” tree; root split increases height.
Tree growth: gets wider or one level taller at top.
CENG 351 File Structures 6

37. Deleting a Data Entry from a B+ Tree
Examine examples first …
CENG 351 File Structures 14

38. Delete 19* and 20* Have we still forgot something? Root You underflow17 5 13 24 30 2* 3* 5* 7* 8*
14*
16*
19*
20*
22*
24*
27*
29*
33*
34*
38*
39*
22*
27* 29*
You underflow
22* 24*
Have we still forgot something?
CENG 351 File Structures 13

39. Deleting 19* and 20* (cont.) Notice how 27 is copied up.
Root 17 5 13 27 30 2* 3* 5* 7* 8*
14*
16*
22*
24*
27*
29*
33*
34*
38*
39*
Notice how 27 is copied up.
But can we move it up?
Now we want to delete 24
Underflow again! But can we redistribute this time?
CENG 351 File Structures 15

40. Deleting 24*
Merge with sibling!
You underflow
Observe the two leaf nodes are merged, and 27 is discarded from their parent, but …
Observe `pull down’ of index entry (below). 30
22*
27*
29*
33*
34*
38*
39*
New root 5 13 17 30 2* 3* 5* 7* 8*
14*
16*
22*
27*
29*
33*
34*
38*
39*
CENG 351 File Structures 16

41. Deleting a Data Entry from a B+ Tree: Summary
Start at root, find leaf L where entry belongs.
Remove the entry.
If L is at least half-full, done!
If L has only d-1 entries,
Try to re-distribute, borrowing from sibling (adjacent node with same parent as L).
If re-distribution fails, merge L and sibling.
If merge occurred, must delete entry (pointing to L or sibling) from parent of L.
Merge could propagate to root, decreasing height.
CENG 351 File Structures 14

42. Example of Non-leaf Re-distribution
Tree is shown below during deletion of 24*. (What could be a possible initial tree?)
In contrast to previous example, can re-distribute entry from left child of root to right child.
Root 22 30 5 13 17 20
14*
16*
17*
18*
20*
33*
34*
38*
39*
22*
27*
29*
21* 7* 5* 8* 3* 2*
CENG 351 File Structures 17

43. After Re-distribution
Intuitively, entries are re-distributed by `pushing through’ the splitting entry in the parent node.
It suffices to re-distribute index entry with key 20; we’ve re-distributed 17 as well for illustration.
Root 17 5 13 20 22 30 2* 3* 5* 7* 8*
14*
16*
17*
18*
20*
21*
22*
27*
29*
33*
34*
38*
39*
CENG 351 File Structures 18

44. Primary vs Secondary Index
Note: We were assuming the data items were in sorted order
This is called primary index
Secondary index:
Built on an attribute that the file is not sorted on.
Can have many different indexes on the same file.
CENG 351 File Structures

45. A Secondary B+-Tree index
Root 17 5 13 20 22 30 33 34 38 39 2 3 5 7 8 14 16 17 18 20 21 22 27 29
Actual data buckets
22* 3* 14* 8*
2* 16* 5* 39*
20* 7* 38* 29* 18

46. More… Hash-based Indexes Grid-files R-Trees etc… Static Hashing
Extendible Hashing
Linear Hashing
Grid-files
R-Trees
etc…
CENG 351 File Structures

47. Terminology
Bucket Factor: the number of records which can fit in a leaf node.
Fan-out : the average number of children of an internal node.
A B+tree index can be used either as a primary index or a secondary index.
Primary index: determines the way the records are actually stored (also called a sparse index)
Secondary index: the records in the file are not grouped in buckets according to keys of secondary indexes (also called a dense index)
CENG 351 File Structures

48. Summary
Tree-structured indexes are ideal for range-searches, also good for equality searches.
B+ tree is a dynamic structure.
Inserts/deletes leave tree height-balanced; High fanout (F) means depth rarely more than 3 or 4.
Almost always better than maintaining a sorted file.
Typically, 67% occupancy on average.
If data entries are data records, splits can change rids!
Most widely used index in database management systems because of its versatility. One of the most optimized components of a DBMS.
CENG 351 File Structures 23