Exam I Grades uMax: 96, Min: 37 uMean/Median:66, Std: 18 uDistribution: w>= 90 : 6 w>= 80 : 12 w>= 70 : 9 w>= 60 : 9 w>= 50 : 7 w>= 40 : 11 w>= 30 : 5 u Letter Grades: w>=85: A w>= 60: B w>= 40: C w< 40: D 1

Buffer Management Record Formats, Files & Indexing

3 Disks, Memory, and Files Query Optimization and Execution Relational Operators Files and Access Methods Buffer Management Disk Space Management DB The BIG picture…

Focus on: “Typical Disk” Terms: Platter, Head, Actuator Cylinder, Track Sector (physical), Block (logical), Gap … Disk Controller BUS

Top View Sector Gap Often different numbers of sectors per track Track Block ( typically multiple sectors )

block x in memory ? I want block X Time = Seek Time (locate track) + Rotational Delay (locate sector)+ Transfer Time (fetch block) + Other (disk controller, …) Key Performance Metric: Time to Fetch Block

Arranging Pages on Disk uBlocks are multiples of sector size uNext block concept: w blocks on same track, followed by w blocks on same cylinder, followed by w blocks on adjacent cylinder uBlocks in a file should be arranged sequentially on disk (by ‘next’), to minimize seek and rotational delay. uFor a sequential scan, pre-fetching several pages at a time is a big win!

Disk Space Management uLowest layer of DBMS software manages space on disk. uHigher levels call upon this layer to: w allocate/de-allocate a page w read/write a page uRequest for a sequence of pages must be satisfied by allocating the pages sequentially on disk! Higher levels don’t need to know how this is done, or how free space is managed.

9 Context Query Optimization and Execution Relational Operators Files and Access Methods Buffer Management Disk Space Management DB

Data Items Records Blocks/Pages Files Memory Disk Storage Overview

11 Reducing Number of Page Transfers uKeep cache of recently accessed pages in main memory wGoal: request for page can be satisfied from cache/buffer pool instead of disk wPurge pages when buffer pool is full E.g., Use LRU algorithm Record clean/dirty state of page

12 Accessing Data Through Buffer Buffer pool DBMS Application page frame

Buffer Management in a DBMS uData must be in RAM for DBMS to operate on it! uTable of pairs is maintained. DB MAIN MEMORY DISK disk page free frame Page Requests from Higher Levels BUFFER POOL choice of frame dictated by replacement policy

When a Page is Requested... uIf requested page is not in pool: w Choose a frame for replacement w If frame is dirty, write it to disk w Read requested page into chosen frame uPin the page and return its address. * If requests can be predicted (e.g., sequential scans) pages can be pre-fetched several pages at a time!

More on Buffer Management uRequestor of page must unpin it, and indicate whether page has been modified: w dirty bit is used for this. uPage in pool may be requested many times, w a pin count is used. A page is a candidate for replacement iff pin count = 0.

Buffer Replacement Policy uFrame is chosen for replacement by a replacement policy: w Least-recently-used (LRU), w Clock, w MRU, etc. uPolicy can have big impact on # of I/O’s; depends on the access pattern.

17 LRU Replacement Policy uLeast Recently Used (LRU) wfor each page in buffer pool, keep track of time when last unpinned wreplace the frame which has the oldest (earliest) time wvery common policy: intuitive and simple Works well for repeated accesses to popular pages uProblems? uProblem: Sequential flooding wLRU + repeated sequential scans. w# buffer frames < # pages in file means each page request causes an I/O. wIdea: MRU better in this scenario?

18 “Clock” Replacement Policy uAn approximation of LRU uArrange frames into a cycle, store one reference bit per frame wCan think of this as the 2nd chance bit uWhen pin count reduces to 0, turn on ref. bit uWhen replacement necessary do for each page in cycle { if (pincount == 0 && ref bit is on) turn off ref bit; else if (pincount == 0 && ref bit is off) choose this page for replacement; } until a page is chosen; A(1) B(p) C(1) D(1)

Physical vs. Logical Addresses Device ID E.g., RecordCylinder # Address=Track # or IDBlock # Offset in block Block ID

Logical Addresses E.g., Record ID is arbitrary bit string map rec ID raddress a Physical addr. Rec ID

Swizzling Memory Disk Rec A block 1 Rec A block 2 block 1

TranslationDB Addr Mem Addr Table Rec-A Rec-A-inMem One Option:

In memory pointers - need “type” bit to disk to memory M Another Option:

Swizzling Automatic On-demand No swizzling / program control

Record Formats, Files & Indexing

26 Context Query Optimization and Execution Relational Operators Files and Access Methods Buffer Management Disk Space Management DB

Disk & Buffer Manager uDisks provide cheap, non-volatile storage. w Random access, but cost depends on location of page on disk; important to arrange data sequentially to minimize seek and rotation delays. uBuffer manager brings pages into RAM. w Page stays in RAM until released by requestor. w Written to disk when frame chosen for replacement (which is sometime after requestor releases the page). w Choice of frame to replace based on replacement policy. w Tries to pre-fetch several pages at a time.

28 Buffer Management and Files uStorage of Data wFields, either fixed or variable length... wStored in Records... wStored in Pages... wStored in Files uIf data won’t fit in RAM, store on Disk wNeed Buffer Pool to hold pages in RAM wDifferent strategies decide what to keep in pool

Record Formats: Fixed Length uInformation about field types same for all records in a file; stored in system catalogs. uFinding i’th field does not require scan of record. Base address (B) L1L2L3L4 F1F2F3F4 Address = B+L1+L2 Header

Record Formats: Variable Length uTwo alternative formats (# fields is fixed): * Second offers direct access to i’th field, efficient storage of nulls (special don’t know value); small directory overhead. 4$$$$ Field Count Fields Delimited by Special Symbols F1 F2 F3 F4 Header: Array of Field Offsets

Page Formats: Fixed Length Records *Record id =. In first alternative, moving records for free space management changes rid; may not be acceptable. Slot 1 Slot 2 Slot N... N M1 0 M PACKED UNPACKED, BITMAP Slot 1 Slot 2 Slot N Free Space Slot M 11 number of records number of slots

Page Formats: Variable Length Records *Can move records on page without changing rid; so, attractive for fixed- length records too. Page i Rid = (i,N) Rid = (i,2) Rid = (i,1) Pointer to start of free space SLOT DIRECTORY N N # slots

Unspanned: records must be within one block block 1 block 2... Spanned block 1 block 2... Spanned vs. Unspanned Records R1R2 R1 R3R4R5 R2 R3 (a) R3 (b) R6R5R4 R7 (a)

Block header - data at beginning that describes block May contain: - File ID (or RELATION or DB ID) - This block ID - Record directory - Pointer to free space - Type of block (e.g. contains recs type 4; is overflow, …) - Pointer to other blocks “like it” - Timestamp...