1. ItsyBitsyRel: A Small Relational Database (Part II) Implementation Hints Shahin Shayandeh shayande@usc.edu

2. Buffer Pool Manager  Hint 1 – represent the buffer pool using c# N frames pinneddirtypage A Buffer Pool

3. Hint 1 (cont.) Class MyFrame { public bool dirty; public bool pinned; public byte[] page; } Class MyBufferManager { … MyFrame[] frames; }

4. Buffer Pool Manager (cont.)  Hint 2 – Look up the frame given a physical disk page number (PID) quickly –idea: hash table Key: PID Value: a MyFrame object

5. Hint 2 (cont.)  Create a hash table Hashtable ht = new Hashtable();  Insert a (key,value) pair ht.Add(PID, frames[i]);  Remove a (key,value) pair ht.Remove(PID);  Retrieve its value given the key MyFrame frame = (MyFrame)ht[PID]; * The value of frame will be null if no such PID in ht.

6. Buffer Pool Manager (cont.)  Hint 3: Least Frequently Used (LRU) replacement algorithm –Maintain a frequency counter for each frame (say, frames[i].f) Initially, frames[i].f=0. i=0,1,…,N-1 frames[i].f=frames[i].f+1 each time the frame is hit (ReadPage/WritePage) in the buffer pool. frames[i].f=1 when a page is swapped in the ith frame

7. Hint 3 (Cont.) –Psudo Code of selecting a page to swap out Function LFUSwapOut() [ int min_f; int swapout_frame_no = -1; for i=0 to N-1 do if (swapout_frame_no == -1) or ((not frames[i].pinned) and (min_f > frames[i].f)) then [ min_f = frames[i].f; swapout_frame_no = i; ] if (swapout_frame_no <> -1) then return frames[swapout_frame_no] else return null; ]

8. Hint 3(Cont.) 1. ReadPage(0) 2. ReadPage(1) 3. ReadPage(2) 4. ReadPage(0) 5. WritePage(2) 6. ReadPage(3) 7. … (0,0) (0,1)(0,0) (0,1)(1,1)(0,0) (0,1)(1,1)(2,1) (0,2)(1,1)(2,1) (0,2)(1,1) * (2,2) (0,2)(3,1)(2,2) (PID, frequency)

9. Class MyRelation  Hint 1: Where to store the metainfo of a relation? –Design 1: “rel1”5 “rel2”8 “rel10”30... filenamefirst page # Book-keeping Information Relation “rel1” Meta info Page containing records

10. Hint 1 (cont.)  Design 2: “rel1”Meta info of “rel1” “rel2”Meta info of “rel2” “rel80”Meta info of “rel80”... Book-keeping information Relation “Rel1” Page containing records

11. Class MyRelation  Hint 2: Keep track of all pages associated with a relation. –Idea: Double Linked List! Page 5Page 6Page 8Page 11 A relation

12. Class MyRelation  Hint 3: Keep track of records Page Header 101011110001… (record bitmap)* ****** ** * * record hole Records in a page

13. Class MyRelation  Hint 4: record insertion/deletion –Idea: Inspired from free page linked list, we use a linked list to chain holes! –Metainfo stores the RID of the first hole –Each hole stores a RID that points to the next hole

14. Class MyRelation  Hint 5: –Go through our solution for part I and how it implements the management of file entries. Its implementation is similar to management of records.

15. Class MyRelation  Hint 6: Free a page in a relation Problem: how to keep data consistency between the buffer pool and the file system after a page is de-allocated? –Design 1: Invoke MyFileSystem’s routine FreePage to free this page If this page is also in the buffer pool, mark it available so that another page can take this frame next time. You need to implement the routine DropPage in class MyBufferManager!

16. Hint 6 (Cont.)  Design 2 : –Invoke MyFileSystem’s routine FreePage to free this page –If this page is in the buffer pool, update its content intelligently (i.e. do the same thing as what the file system does when freeing a page) and set its dirty bit to true. Remarks: –Higher buffer pool hit rate –More complicated

17. Class MyRelation  Hint 7: Find a record –Idea: Enumerate all records by calling ReadFirstRecord and ReadNextRecord. For each record, determine whether or not it satisfies the given condition.

18. Class MyRelation  Hint 8: Efficient Compact –Idea: Keep two record cursors (RC) rc1 and rc2. rc1 searches holes. It starts from the first record and goes forward. rc2 searches records. It starts from the last record and goes backward. Each time rc1 finds a hole and rc2 finds a record, fill the hole with the record. Compact terminates when rc1 and rc2 hit.

19. Class MyRelation –Implementation hint: Utilize the double linked list!