1. Spring 2004 ECE569 Lecture 03-1.1 ECE 569 Database System Engineering Spring 2004 Yanyong Zhang www.ece.rutgers.edu/~yyzhangwww.ece.rutgers.edu/~yyzhang Course URL www.ece.rutgers.edu/~yyzhang/spring03www.ece.rutgers.edu/~yyzhang/spring03

2. Spring 2004 ECE569 Lecture 03-1.2 Warm-up Discussion  Address space, virtual memory  Memory magement  How does file system interact with virtual memory?  What happens when the user makes a “read” call?

3. Spring 2004 ECE569 Lecture 03-1.3 Questions to answer in this class?  If a query tries to access tuple T, how can the system locate where T is?

4. Spring 2004 ECE569 Lecture 03-1.4 Media and File Management  Abstraction l Array of fixed length blocks l Size varies dynamically, space permitting  Notation l Blocks – File system objects l Pages – Units of virtual address space l Slots – Units of storage on disk l All of these items are identical in size and there is a direct mapping from one to the other. l We will generally use them interchangeably.

5. Spring 2004 ECE569 Lecture 03-1.5 Data structure for Block # defineEIGHTK 8192 typedefunsigned int FILENO; typedefunsigned int BLOCKID; typedefstruct {intflip; FILENO fileno; BLOCKIDblockno; } BLOCKHEAD; typedef struct {BLOCKHEADheader; char contents[EIGHTK-sizeof(header)-2]; int flop; } BLOCK, *BLOCKP;

6. Spring 2004 ECE569 Lecture 03-1.6 File System API STATUS create(filename, allocparmp) -- create and allocate a new file STATUS delete(filename) -- delete a file and deallocate space for it STATUS open(filename,ACCESSMODE,FILEID) -- Open file in desired mode and return file handle STATUS close(FILEID) -- Close an open file STATUS extend(FILEID,allocparamp) -- extend existing file by specified amount STATUS read(FILEID,BLOCKID,BLOCKP) -- read contents of specified block into a buffer STATUS readc(FILEID,BLOCKID,blockcount,BLOCKP) -- read a certain number of block into consecutive buffers in memory. STATUS write(FILEID,BLOCKID,BLOCKP) -- write a buffer to the specified block on disk. STATUS writec(FILEID,BLOCKID,blockcount,BLOCKP) -- write a number of blocks from consecutive pages to disk.

7. Spring 2004 ECE569 Lecture 03-1.7 Mapping Blocks onto Slots  Disk allocation schemes determine the mapping between blocks onto slots  Issues in disk space allocation: l Initial allocation: When a file is created, how many contiguous slots should be allocated to it? l Incremental expansion: If an existing file grows beyond the number of slots currently allocated, how many additional contiguous blocks should be assigned to that file? l Reorganization: When and how should the free space on the disk be reorganized?

8. Spring 2004 ECE569 Lecture 03-1.8 Free Space Management +Bit Map - One bit represents each block - Easy to find contiguous free spaces for allocation +Free List - Link free blocks together into a free list - Can use all techniques for memory free-list management, first-fit, best-fit, etc. + fault-tolerance - extra slots

9. Spring 2004 ECE569 Lecture 03-1.9 Static and Contiguous Allocation  At file creation time, the total number of blocks a file needs is allocated at one time in contiguous slots.  Address translation is easy l s b = the slot holding block 0; l s b +k = the slot holding block k  Advantage: Supports both block-direct access and sequential access  Disadvantages: Files cannot grow, cannot shrink  Discussion: database systems by third-party vendor prefer such file organizations

10. Spring 2004 ECE569 Lecture 03-1.10 Extent-based Allocation  Provides many of the advantages of contiguous allocation without many of the problems  The Idea l Allocate an initial chunk that is probably big enough (primary allocation) l If file runs out of space, allocate another chunk (secondary allocations) l Successive allocations increase in size  Characteristics l Good clustering allows efficient sequential I/O l More complex address translation than contiguous allocation

11. Spring 2004 ECE569 Lecture 03-1.11 Extent-based Allocation Why do we need extent directory? transparency easy lock name

12. Spring 2004 ECE569 Lecture 03-1.12 Single-Slot Allocation  extent-based allocation with extent size of 1 slot.  Used by original UNIX file system, and implemented by the idea of i-node.  10 + n + n2 + n3 blocks can be addressed, where n is the number pointers that can be held by one slot (e.g., 8192 / 4 = 2048).  Poor random access performance for large files.

13. Spring 2004 ECE569 Lecture 03-1.13 Mapping Relations to Disks

14. Spring 2004 ECE569 Lecture 03-1.14 Buffer management  Database buffer is the mediator between the basic file system and the tuple-oriented file system. l A tuple is addressed as  Main purpose l To make the pages addressable in main memory l To coordinate the writing of pages to disks with the log manager and recovery manager l To minimize the number of actual disk accesses.

15. Spring 2004 ECE569 Lecture 03-1.15 Logic of Buffer Manager +Search in buffer: Check if the requested page is in the buffer. If found, return the address F of this frame to the caller. +Find free frame: If the page is not in the buffer, find a frame that holds no valid page. +Determine replacement victim: If no such frame exists, determine a page that can be removed from the buffer (in order to reuse its frame). +Write modified page: If replacement page has been changed, write it. +Establish frame address: Denote the start address of the frame as F. +Determine block address: Translate the requested PAGEID P into a FILEID and a block number. Read the block into the frame selected. +Return: Return the frame address F to the caller.

16. Spring 2004 ECE569 Lecture 03-1.16 Buffer Management

17. Spring 2004 ECE569 Lecture 03-1.17 Buffer Management vs. Read operation  Share vs private (lost update anomaly)  In buffer management, the clients tell whether they are using the page or not.

18. Spring 2004 ECE569 Lecture 03-1.18 Lost Update Anomoly This implies that the buffer should be shared by multiple processes.

19. Spring 2004 ECE569 Lecture 03-1.19 The Need for Synchronization

20. Spring 2004 ECE569 Lecture 03-1.20 The Fix / Use / Unfix Protocol +FIX: The client requests access to a page using the bufferfix interface. +USE: The client uses the page and the pointer to the frame containing the page will remain valid. +UNFIX: The client explicitly waives further usage of the frame pointer; that is, it tells the buffer manager that it no longer wants to use that page.

21. Spring 2004 ECE569 Lecture 03-1.21 The Fix / Use / Unfix Protocol

22. Spring 2004 ECE569 Lecture 03-1.22 Buffer Control Blocks typedef struct { PAGEIDpageid;/* id of page in file*/ PAGEPTRpageaddr;/* base addr. in buffer*/ Intindex;/* record within page */ Semaphore*pagesem;/* pointer to the sem. */ Booleanmodified;/* caller modif. page*/ Booleaninvalid;/* destroyed page*/ } BUFFER_ACC_CB, *BUFFER_ACC_CBP;

23. Spring 2004 ECE569 Lecture 03-1.23 Buffer Structure