1. Cache-Oblivious B-Trees
Michael A. Bender, Erik D. Demaine, Martin Farach-Colton.
Presented by: Itai Lahan

2. B-Trees – A reminder Balanced search tree
[R. Bayer and E. M. McCreight. 1972]
Balanced search tree
All leaves have the same depth
All nodes have degree at most B
The root has degree at least two
All internal nodes have degree at least B/2
O(B)
O(logBN)

3. B-Trees - example 11 11 5 8 5 8 16 21 16 21 Searching for 18 18
5 8
5 8
16 21
16 21
6 7
6 7
9 10
9 10 18

4. B-Trees Balanced search tree
[R. Bayer and E. M. McCreight. 1972]
Balanced search tree
Fan-out is proportional to memory block size
A single block read determines the next node
Classical two-level memory model solution B B B B B B B B B B B

5. B-Trees in multilevel memory hierarchy ?
B1 > B2 > B3 > … > Bk block sizes between memory levels B1 B2 B3 B1 

6. B-Trees in multilevel memory hierarchy ?
B1 > B2 > B3 > … > Bk block sizes between memory levels B3 B2 B2 B1 B1
 ?

7. Cache Oblivious Algorithms
No variable depends on hardware parameters
Reason about a simple two level memory
Prove results about an unknown multilevel memory
Cache oblivious B-Tree ?
How do we achieve O(logBN) without knowing B ??
It’s all about the memory layout…

8. Van Emde Boas Layout
[P. van Emde Boas, R. Kaas, and E. Zijlstra. 1977]

9. Van Emde Boas Layout
[P. van Emde Boas, R. Kaas, and E. Zijlstra. 1977]

10. Recursive structures are stored contiguously
Van Emde Boas Layout
[P. van Emde Boas, R. Kaas, and E. Zijlstra. 1977]
Recursive structures are stored contiguously A B1 Bk A B1 B2 B3 B4
Bk-1 Bk

11. Van Emde Boas Layout A1 B11 B1k A2 Ak B21 B2k Bk1 Bkk A1 B11 B1k A2
[P. van Emde Boas, R. Kaas, and E. Zijlstra. 1977] A1
B11
B1k A2 Ak
B21
B2k
Bk1
Bkk A1
B11
B1k A2
B21
B2k Ak
Bk1
Bkk

12. Van Emde Boas Layout 1 2 3

13. Van Emde Boas Layout 1 2 7 3 4 5 6 8 9 10 11

14. Van Emde Boas Layout 1 2 17 3 4 18 19 5 8 11 14 20 23 26 29 6 7 9 10 12 13 15 16 21 22 24 25 27 28 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

15. Van Emde Boas Layout

16. Van Emde Boas Layout

17. Van Emde Boas Layout

18. Van Emde Boas Layout

19. Van Emde Boas Layout 1 2 3 7 4 5 6 10 9 19 8 12 11 13 17 15 14 23 18 16 24 25 27 26 20 21 22 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

20. Searching with memory transfer block of size 9
Van Emde Boas Layout
Searching with memory transfer block of size 9 1 2 3 7 4 5 6 10 9 19 8 12 11 13 17 15 14 23 18 16 24 25 27 26 20 21 22 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

21. Searching with memory transfer block of size 9
Van Emde Boas Layout
Searching with memory transfer block of size 9 1 2 3 7 4 5 6 10 9 19 8 12 11 13 17 15 14 23 18 16 24 25 27 26 20 21 22
2 cache misses 10 11 12 13 14 15 16 17 18 1 2 3 4 5 6 7 8 9 19 20 21 22 23 24 25 26 27

22. Searching with memory transfer block of size 3
Van Emde Boas Layout 1 2 3 7 4 5 6 10 9 19 8 12 11 13 17 15 14 23 18 16 24 25 27 26 20 21 22
Searching with memory transfer block of size 3
4 cache misses 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

23. Van Emde Boas Layout 2 L2 cache misses 4 L1 cache misses
Searching with L2 memory transfer block of size 9 1 2 3 7 4 5 6 10 9 19 8 12 11 13 17 15 14 23 18 16 24 25 27 26 20 21 22
and with L1 memory transfer block of size 3
4 L1 cache misses
2 L2 cache misses 10 11 12 13 14 15 16 17 18 1 2 3 4 5 6 7 8 9 19 20 21 22 23 24 25 26 27

24. Static Cache Oblivious Tree
[Prokop, 1999]
LogB
#IOs per search = ?

25. From Static to Dynamic 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Static is fine, but… A Dynamic B-Tree should allow the addition and removal of nodes
How do we add an element efficiently to an ordered array ?

26. Dynamic B-Tree – restating the problem
How do we add an element efficiently to an ordered array ?
Trivial, O(N/B) memory transfers
10.5 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Can we do better ?

27. Dynamic B-Tree – restating the problem
General idea – Array with enough extra space between nodes for future insertions x1 x2 x3 x4 x5 xN
Contradictory goals – nodes should be packed densely for locality of reference
Guideline - Distribute elements evenly in the array
Store any set of k contiguous elements xi1,…, xik in a contiguous subarray of size O(k)

28. Packed Memory Management
[A. Itai, A. G. Konheim, and M. Rodeh 1981]
Weight = #Data/Space x1 x2 x3 x4 x5 x6 1
x’2 N

29. Packed Memory Management
[A. Itai, A. G. Konheim, and M. Rodeh 1981]
6/32
3/16
2/8
2/4
0/4 x1 x2 x3 x4 x5 x6 1
x’2 N

30. Packed Memory Management
[A. Itai, A. G. Konheim, and M. Rodeh 1981]
7/32
4/16
3/8
3/4
0/4 x1 x2 x3 x4 x5 x6 x2
x’2 x3 ! 1 N

31. Packed Memory Management
[A. Itai, A. G. Konheim, and M. Rodeh 1981]
7/32
4/16
3/8 x2
x’2 x3 
2/4
1/4
3/4
0/4 x1 x2 x3 x4 x5 x6 x2
x’2 x3 1 N

32. Packed Memory Management
[A. Itai, A. G. Konheim, and M. Rodeh 1981]
Size of the interval to rebalance ?
Thresholds for interval over/underflows ? x1 x2 x3 x4 x5 x6 x2
x’2 x3 1 N

33. Packed Memory Management
[A. Itai, A. G. Konheim, and M. Rodeh 1981]
Scanning a k consecutive elements
k Elements are stored in O(k) memory cells
O(1+k/B) memory transfers
Inserting/deleting an element
Moves O(log2N) amortized consecutive cells
O(1 + log2N / B) amortized memory transfers x1 x2 x3 x4 x5 x6 x2
x’2 x3 1 N

34. Putting it all together… ?1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

35. Putting it all together… ?9 10 8 8’ 9 10 8 1 2 3 4 5 6 7 8 8’ 9 10 9 10 11 12 13 14 15 8’

36. It’s still a tree ! 8’ Insertion moves O(log2N) amortized nodes3 7 1 2 3 4 5 6 7 8 8’ 9 10 9 10 11 12 13 14 15 8’
Insertion moves O(log2N) amortized nodes
Requires O(log2N) back pointers updates
And what about the van Emde layout ?

37. Splits and Merges

38. Splits and Merges A B1 Bi
Bi+1 Bk A B1 Bi
Bi+1 Bk

39. Splits and Merges A A’
A’’ B1 Bi
Bi+1 Bk A B1 Bi
Bi+1 Bk

40. Splits and Merges A’ A’’ B1 Bi Bi+1 Bk A B1 Bi B1 Bi Bi+1 Bk A’ A’’

41. Splits and Merges A B1 Bi Bk
Bi+1 A’
A’’ B1 Bi Bk
Bi+1 A A’
A’’
Each update requires O(1 + logN/B) amortized memory transfers

42. Putting it all together… againx1 x2 x3 x4 x5 x6
Packed Memory Maintenance 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
van Emde Boas layout
5 8
16 21
6 7
9 10
Weight-balanced B-tree
Search O(logBN)
Scan O(1 + k/B)
Update O(logBN + log2N)
Search for key O(logBN)
Create/delete leaf O(1)
Rearange packed array O(1+log2N/B)
Update back pointers O(1+log2N)
Split/Merge nodes O(1+logN/B)

43. Saving on back pointer updates ?3 7 1 2 3 4 5 6 7 8 8’ 9 10 9 10 11 12 13 14 15 8’
How can we reduce back pointer updates ?

44. Saving on back pointer updates ?
Idea – pack groups of leaves together beneath a single parent

45. Saving on back pointer updates ?
Idea – pack groups of leaves together beneath a single parent
logN

46. Indirection
Idea – pack groups of leaves together beneath a single parent
logN
logN
logN
logN

47. Indirection
Idea – pack groups of leaves together beneath a single parent
logN
logN
logN
logN

48. Asymptotically the same ! But we’ve delayed pointer updates by O(logN)
Indirection
Searching for an element ?
Asymptotically the same ! But we’ve delayed pointer updates by O(logN)
O(logBN)
O(logN / B)
logN
logN
logN
logN

49. Final build – two levels of indirection
Search O(logBN)
Scan O(1 + k/B)
Updates
O(logBN + log2N / B) T L1 L0
L0 contains all the values

50. Presented by: Itai Lahan
The end…
Presented by: Itai Lahan