Spring 2004 ECE569 Lecture ECE 569 Database System Engineering Spring 2004 Yanyong Zhang Course URL

Spring 2004 ECE569 Lecture Warm-up Discussion  Address space, virtual memory  Memory magement  How does file system interact with virtual memory?  What happens when the user makes a “read” call?

Spring 2004 ECE569 Lecture Questions to answer in this class?  If a query tries to access tuple T, how can the system locate where T is?

Spring 2004 ECE569 Lecture Media and File Management  Abstraction l Array of fixed length blocks l Size varies dynamically, space permitting  Notation l Blocks – File system objects l Pages – Units of virtual address space l Slots – Units of storage on disk l All of these items are identical in size and there is a direct mapping from one to the other. l We will generally use them interchangeably.

Spring 2004 ECE569 Lecture Data structure for Block # defineEIGHTK 8192 typedefunsigned int FILENO; typedefunsigned int BLOCKID; typedefstruct {intflip; FILENO fileno; BLOCKIDblockno; } BLOCKHEAD; typedef struct {BLOCKHEADheader; char contents[EIGHTK-sizeof(header)-2]; int flop; } BLOCK, *BLOCKP;

Spring 2004 ECE569 Lecture File System API STATUS create(filename, allocparmp) -- create and allocate a new file STATUS delete(filename) -- delete a file and deallocate space for it STATUS open(filename,ACCESSMODE,FILEID) -- Open file in desired mode and return file handle STATUS close(FILEID) -- Close an open file STATUS extend(FILEID,allocparamp) -- extend existing file by specified amount STATUS read(FILEID,BLOCKID,BLOCKP) -- read contents of specified block into a buffer STATUS readc(FILEID,BLOCKID,blockcount,BLOCKP) -- read a certain number of block into consecutive buffers in memory. STATUS write(FILEID,BLOCKID,BLOCKP) -- write a buffer to the specified block on disk. STATUS writec(FILEID,BLOCKID,blockcount,BLOCKP) -- write a number of blocks from consecutive pages to disk.

Spring 2004 ECE569 Lecture Mapping Blocks onto Slots  Disk allocation schemes determine the mapping between blocks onto slots  Issues in disk space allocation: l Initial allocation: When a file is created, how many contiguous slots should be allocated to it? l Incremental expansion: If an existing file grows beyond the number of slots currently allocated, how many additional contiguous blocks should be assigned to that file? l Reorganization: When and how should the free space on the disk be reorganized?

Spring 2004 ECE569 Lecture Free Space Management +Bit Map - One bit represents each block - Easy to find contiguous free spaces for allocation +Free List - Link free blocks together into a free list - Can use all techniques for memory free-list management, first-fit, best-fit, etc. + fault-tolerance - extra slots

Spring 2004 ECE569 Lecture Static and Contiguous Allocation  At file creation time, the total number of blocks a file needs is allocated at one time in contiguous slots.  Address translation is easy l s b = the slot holding block 0; l s b +k = the slot holding block k  Advantage: Supports both block-direct access and sequential access  Disadvantages: Files cannot grow, cannot shrink  Discussion: database systems by third-party vendor prefer such file organizations

Spring 2004 ECE569 Lecture Extent-based Allocation  Provides many of the advantages of contiguous allocation without many of the problems  The Idea l Allocate an initial chunk that is probably big enough (primary allocation) l If file runs out of space, allocate another chunk (secondary allocations) l Successive allocations increase in size  Characteristics l Good clustering allows efficient sequential I/O l More complex address translation than contiguous allocation

Spring 2004 ECE569 Lecture Extent-based Allocation Why do we need extent directory? transparency easy lock name

Spring 2004 ECE569 Lecture Single-Slot Allocation  extent-based allocation with extent size of 1 slot.  Used by original UNIX file system, and implemented by the idea of i-node.  10 + n + n2 + n3 blocks can be addressed, where n is the number pointers that can be held by one slot (e.g., 8192 / 4 = 2048).  Poor random access performance for large files.

Spring 2004 ECE569 Lecture Mapping Relations to Disks

Spring 2004 ECE569 Lecture Buffer management  Database buffer is the mediator between the basic file system and the tuple-oriented file system. l A tuple is addressed as  Main purpose l To make the pages addressable in main memory l To coordinate the writing of pages to disks with the log manager and recovery manager l To minimize the number of actual disk accesses.

Spring 2004 ECE569 Lecture Logic of Buffer Manager +Search in buffer: Check if the requested page is in the buffer. If found, return the address F of this frame to the caller. +Find free frame: If the page is not in the buffer, find a frame that holds no valid page. +Determine replacement victim: If no such frame exists, determine a page that can be removed from the buffer (in order to reuse its frame). +Write modified page: If replacement page has been changed, write it. +Establish frame address: Denote the start address of the frame as F. +Determine block address: Translate the requested PAGEID P into a FILEID and a block number. Read the block into the frame selected. +Return: Return the frame address F to the caller.

Spring 2004 ECE569 Lecture Buffer Management

Spring 2004 ECE569 Lecture Buffer Management vs. Read operation  Share vs private (lost update anomaly)  In buffer management, the clients tell whether they are using the page or not.

Spring 2004 ECE569 Lecture Lost Update Anomoly This implies that the buffer should be shared by multiple processes.

Spring 2004 ECE569 Lecture The Need for Synchronization

Spring 2004 ECE569 Lecture The Fix / Use / Unfix Protocol +FIX: The client requests access to a page using the bufferfix interface. +USE: The client uses the page and the pointer to the frame containing the page will remain valid. +UNFIX: The client explicitly waives further usage of the frame pointer; that is, it tells the buffer manager that it no longer wants to use that page.

Spring 2004 ECE569 Lecture The Fix / Use / Unfix Protocol

Spring 2004 ECE569 Lecture Buffer Control Blocks typedef struct { PAGEIDpageid;/* id of page in file*/ PAGEPTRpageaddr;/* base addr. in buffer*/ Intindex;/* record within page */ Semaphore*pagesem;/* pointer to the sem. */ Booleanmodified;/* caller modif. page*/ Booleaninvalid;/* destroyed page*/ } BUFFER_ACC_CB, *BUFFER_ACC_CBP;

Spring 2004 ECE569 Lecture Buffer Structure