1. CS 564: Database Management Systems
BadgerDB
Goal: Build key components of a RDBMS
First hand experience building the internals of a simple database system
And have some fun doing so!
Two parts
Buffer manager(Due Date : Feb 17 by 2PM)
B+tree
All projects are individual assignments
11/10/2018
CS 564: Database Management Systems

2. Structure of Database Query optimizer and execution
Relational operators
File and access methods
Buffer manager
I/O manager
Problem it is trying to solve: Disks are slow, especially when it comes to the random access, memory is fast but volatile.
But the whole reason we are using disk is to PERSIST things. Imagine a database that promises you to have saved things or committed a transaction but doesn’t.
Can we come up with a strategy and mechanism to solve this problem?
Note that this nature of problem is not specific to database. Webapplications have similar problems.
11/10/2018
CS 564: Database Management Systems, Jignesh M. Patel

3. CS 564: Database Management Systems, Jignesh M. Patel
Plan for today
Logistics
Quick review of pointer and reference
BadgerDB: BufferManager
Specifications and Code
Q&A
11/10/2018
CS 564: Database Management Systems, Jignesh M. Patel

4. Pointers and references

5. Address-of operator (&)
foo = &myvar
assign the address of variable myvar to foo

6. Address-of operator (&)
foo = &myvar
assign the address of variable myvar to foo
myvar = 25;
foo = &myvar;
bar = myvar;

7. Address-of operator (&)
foo = &myvar
assign the address of variable myvar to foo
myvar = 25;
foo = &myvar;
bar = myvar;
pointer
int *

8. Dereference operator (*)
baz = *foo;

9. Dereference operator (*)
baz = *foo;

10. Dereference operator (*)
baz = *foo;
baz = foo;
baz = *foo;

11. Dereference operator (*)
baz = *foo;
baz = foo; // baz equal to foo (1776)
baz = *foo; // baz equal to value pointed to by foo (25)

12. References (&)
C++ uses & to denote the address-of operator in an expression
C++ assigns an additional meaning to & in declaration to declare a reference variable.

13. References (&)
C++ uses & to denote the address-of operator in an expression
C++ assigns an additional meaning to & in declaration to declare a reference variable.
int main() {
int number = 88;
int & refNumber = number;
cout << number << endl; // (88)
cout << refNumber << endl; // (88) }

14. References (&)
C++ uses & to denote the address-of operator in an expression
C++ assigns an additional meaning to & in declaration to declare a reference variable.
int main() {
int number = 88;
int & refNumber = number;
cout << number << endl; // (88)
cout << refNumber << endl; // (88)
refNumber = 99 }

15. References (&)
C++ uses & to denote the address-of operator in an expression
C++ assigns an additional meaning to & in declaration to declare a reference variable.
int main() {
int number = 88;
int & refNumber = number;
cout << number << endl; // (88)
cout << refNumber << endl; // (88)
refNumber = 99
cout << number << endl; // ? }

16. References (&)
C++ uses & to denote the address-of operator in an expression
C++ assigns an additional meaning to & in declaration to declare a reference variable.
int main() {
int number = 88;
int & refNumber = number;
cout << number << endl; // (88)
cout << refNumber << endl; // (88)
refNumber = 99
cout << number << endl; // (99) }

17. References (&) Likewise with function signatures accepting references
C++ uses & to denote the address-of operator in an expression
C++ assigns an additional meaning to & in declaration to declare a reference variable.
int main() {
int number = 88;
int & refNumber = number;
cout << number << endl; // (88)
cout << refNumber << endl; // (88)
refNumber = 99
cout << refNumber << endl;
cout << number << endl; // (99)
number = 110;
cout << number << endl;
cout << refNumber << endl; // (110) }
Likewise with function signatures accepting references

18. BadgerDB: BufferManager

19. Structure of Database Query optimizer and execution
Relational operators
File and access methods
Buffer manager
I/O manager
11/10/2018
CS 564: Database Management Systems, Jignesh M. Patel

20. CS 564: Database Management Systems
BadgerDB: IO Layer
Allows upper level layer (Buffer Manager) to
Create/destroy files
Allocate/delete pages
Read/write pages
Provided!
11/10/2018
CS 564: Database Management Systems

21. CS 564: Database Management Systems
BadgerDB: IO Layer
File Class(File.h)
Page allocatePage()
create
open
Page readPage(pageNo)
remove
void writePage(newPage)
isOpen
exists
Void deletePage(pageNo)
filename =
Page Class(Page.h)
11/10/2018
CS 564: Database Management Systems

22. CS 564: Database Management Systems, Jignesh M. Patel
Buffer Pool (frames)
In-memory buffer
holds database pages that are read from disk
P1 from File1
<empty>
P4 from File2
unit of transfer = “page”
File1 P1 P2 P3 P4 P5 P6 P7
File2 P1 P2 P3 P4 P5 P6 P7
11/10/2018
CS 564: Database Management Systems, Jignesh M. Patel

23. What does Buffer Manager do?
Control which pages are memory resident
Request Page N From File M
Buffer Manager
P1 from File1
<empty>
P4 from File2
If not:
choose a frame ,
reads in the pages from disk
into the frame
If it is in the buffer pool:
return!
11/10/2018
CS 564: Database Management Systems, Jignesh M. Patel

24. CS 564: Database Management Systems
Clock Algorithm
Requested page in Buffer Pool:
pinCnt++/ Return handle to frame
Else read page in from disk
find space in the buffer pool!
Frame b/k:
pinCnt
dirty
refbit …
Read
Page
(1, T, 0)
(2, T, 1)
(2, T, 0)
Page unpinned
(3, F, 0)
(0, F, 0)
(1, F, 1)
(0, F, 1)
(0, T, 0)
11/10/2018
CS 564: Database Management Systems

25. The Clock Replacement Algorithm
Advance Clock Pointer No
Valid set?
Yes
refbit set?
Yes
Clear refbit No
Yes
Page pinned? No
Dirty bit set?
Yes
Flush page to disk No
Call “Set()” on the Frame
Use Frame
11/10/2018
CS 564: Database Management Systems, Jignesh M. Patel

26. Structure of Buffer Manager
BufHashTbl Class
Keeps mapping information
(file, page number) <-> (buffer pool frame)
// insert entry into hash table mapping (file, pageNo) to frameNo
void insert(const File* file, const int pageNo, const int frameNo);
// Check if (file,pageNo) is currently in the buffer pool (ie. in // the hash table. If so, assign the corresponding frame number in frameNo.
bool lookup(const File* file, const int pageNo, int& frameNo);
// remove entry obtained by hashing (file,pageNo) from hash table.
void remove(const File* file, const int pageNo);
11/10/2018
CS 564: Database Management Systems, Jignesh M. Patel

27. Structure of Buffer Manager
BufDesc Class
Keeps track of the state of each frame in the buffer pool
class BufDesc { private: File* file; // pointer to file object PageId pageNo; // page within file FrameId frameNo; // buffer pool frame number int pinCnt; // number of times this page has been pinned bool dirty; // true if dirty; false otherwise bool valid; // true if page is valid bool refbit; // true if this buffer frame been referenced recently
void Clear(); // initialize buffer frame void Set(File* filePtr, PageId pageNum); //set BufDesc member variable values
void Print() //Print values of member variables
BufDesc(); //Constructor
11/10/2018
CS 564: Database Management Systems, Jignesh M. Patel

28. Structure of Buffer Manager
BufMgr Class (buffer.cpp)
This is where you write your code
class BufMgr { private: FrameId clockHand; // clock hand for clock algorithm BufHashTbl *hashTable; // hash table mapping (File, page) to frame number BufDesc *bufDescTable; // BufDesc objects, one per frame std::uint32_t numBufs; // Number of frames in the buffer pool public:
Page* bufPool;
11/10/2018
CS 564: Database Management Systems

29. CS 564: Database Management Systems, Jignesh M. Patel
Assignment Tasks
Complete member functions of BufMgr class
~BufMgr();
void advanceClock();
void allocBuf(FrameId& frame);
void readPage(
File *file, const PageId PageNo, Page*& page);
void unPinPage(
File *file, const PageID PageNo, const bool dirty)
void allocPage(
File * file, PageID & PageNo, Page *& Page)
void disposePage(File * file, const PageId PageNo)
void flushFile(File *file)
11/10/2018
CS 564: Database Management Systems, Jignesh M. Patel

30. CS 564: Database Management Systems, Jignesh M. Patel
Main function tests
Test1~2 : allocPage, unPinPage, readPage
Test3 : reading a page from a invalid file
InvalidPageException should be thrown
Test4 : unpinning a page whose pinCnt is already zero
PageNotPinnedException should be thrown
Test5 : allocating too many buffers
BufferExceededException should be thrown
Test6 : flushing file when it is still being used
PagePinnedException should be thrown
11/10/2018
CS 564: Database Management Systems, Jignesh M. Patel