CS CS4432: Database Systems II Record and Page Formats Chapter 12

CS Data Items Records Blocks Files Memory Overview

CS What are the data items we want to store? a salary a name a picture What we have available: Bytes 8 bits

CS To represent: Integer (short): 2 bytes e.g., 35 is

CS Boolean e.g., TRUE FALSE To represent: Enumeration types: e.g., RED  1 GREEN  3 BLUE  2 YELLOW  4 … Can we use less than 1 byte/code? Yes, but only if desperate...

CS Characters  Various coding schemes suggested (ASCII) To represent: Example: A: a: : LF:

CS –Null terminated e.g., –Length given e.g., -Fixed length e.g., In Oracle define the string length. e.g., name CHAR(20), ctacta3 To represent: String of characters

CS Key Points Fixed length items Variable length items - usually length given at beginning Type of an item : - tells us how to interpret (plus size if fixed)

CS Data Items Records Blocks Files Memory Overview

CS Record - Collection of related data items (called FIELDS) E.g.: Employee record: name CHAR (20), salary NUMBER, date-of-hire DATE,...

CS Types of records: Main choices: –FIXED vs VARIABLE FORMAT –FIXED vs VARIABLE LENGTH

CS A SCHEMA contains information such as: - # fields (attributes) - type of each field (length) - order of attributes in record - meaning of each field (domain) - constraints (primary key, etc). Fixed format Not associated with each record.

CS Example: fixed format & fixed length Employee record (1) E.id, 2 byte integer (2) E.name, 10 char. Schema (3) Dept, 2 byte code We can simply concatenate fields. 55 s m i t h j o n e s 01 Records

CS What : –Not all fields are included in the record, –and/or, fields possibly in different orders. Then : –Record itself must contain format, i.e., it is “self-describing”: Variable format

CS Why Variable Format ? “sparse” records repeating fields evolving formats

CS Example: variable format and length 4I524SDROF46 Field name codes could also be strings, i.e., TAGS # Fields Code identifying field as E# Integer type Code for Ename String type Length of str.

CS EXAMPLE: variable format record with repeating fields e.g., Employee has one or more children 3E_name: FredChild: SallyChild: Tom Do repeating fields always require variable format and length?

CS Example : a person and her hobbies. MarySailingChess-- Then allocate maximum number of repeating fields If not used, set to null Repeating fields with fixed format & length

CS Many variants between fixed - variable format: Example1: Include record type in record recordtype record length tells me what to expect (i.e., points to schema)

CS Record header - data at beginning that describes record May contain: - pointer to schema (record type) - length of record - time stamp (create time, mod. time) - other stuff (e.g., ROW-ID in Oracle)

CS Example2: Variant btw FIXED/VAR format Hybrid format : one part is fixed, other is variable E.g.: All employees have E#, name, dept; and other fields vary. 25SmithToy2retiredHobby:chess # of var fields

CS Also, many variations in internal organization of record Just to show one: length of field 3F310F1 5 F212 * * * F1F2F3 total size offsets

CS Question: We have seen examples for : * Fixed format and length records * Variable format and length records (a) Does fixed format and variable length make sense? (b) Does variable format and fixed length make sense?

CS Data Items Records Blocks Files Memory Next:

CS Goal : placing records into blocks blocks... a file assume fixed length blocks assume a single file (for now) records

CS (1) separating records (2) spanned vs. unspanned (3) mixed record types – clustering (4) split records (5) sequencing (6) indirection Options for storing records in blocks:

CS Block (a) no need to separate if fixed size records. (b) or, use special marker (c) or, give record lengths (or offsets) - within each record - in block header (1) Separating records R2R1R3

CS Unspanned: records within one block block 1 block 2... Spanned : records wrap across 2 blocks block 1 block 2... (2) Spanned vs. Unspanned R1R2 R1 R3R4R5 R2 R3 (a) R3 (b) R6R5R4 R7 (a)

CS Unspanned is much simpler, but may waste space… Spanned essential if record size > block size Spanned vs. unspanned:

CS Example 10 6 records each of size 2,050 bytes (fixed) block size = 4096 bytes block 1 block bytes wasted bytes wasted 2046 R1R2 Utiliz = 50% -> ½ of space is wasted

CS Mixed - records of different types (e.g., EMPLOYEE, DEPT) allowed in same block e.g., a block (3) Mixed versus uniform record types EMP e1 DEPT d1 DEPT d2

CS Why do we want to mix? Answer: CLUSTERING Records that are frequently accessed together should be placed into the same block Problems Creates variable length records in block Aim to avoid duplicates (how to cluster?) Insert/deletes are harder

CS Example Clustering Q1: select C_NAME, C_CITY, AMOUNT, … from DEPOSIT, CUSTOMER where DEPOSIT.C_NAME = CUSTOMER.C.NAME a block layout: CUSTOMER,NAME=SMITH DEPOSIT,NAME=SMITH CUSTOMER,NAME=JONES Question: Good idea or bad idea ?

CS If Q1 frequent with join on customer and deposit relations, then clustering good But if instead Q2 frequent with : Q2: SELECT * FROM CUSTOMER then clustering is counter-productive

CS Compromise: No mixing, but keep related records in same cylinder...

CS (1) Separating records (2) Spanned vs. Unspanned (3) Mixed record types - Clustering (4) Split records (5) Sequencing (6) Indirection So Far: Storing records in blocks

CS (1) separating records (2) spanned vs. unspanned (3) mixed record types – clustering (4) split records (5) sequencing (6) indirection Options for storing records in blocks:

CS Fixed part in one block Typically for hybrid format Variable part in another block (4) Split records

CS Block with fixed recs. R1 (a) R1 (b) Block with variable recs. R2 (a) R2 (b) R2 (c)

CS Ordering records in file (and block) by some key value –Sequential file ( -  sequenced file) Why sequencing ? –Typically to make it possible to efficiently read records in order (5) Sequencing

CS Sequencing Options (a) Next record physically contiguous... (b) Linked What about INSERT/ DELETE ? Next (R1)R1 Next (R1)

CS (c)Overflow area Records in sequence R1 R2 R3 R4 R5 Sequencing Options header R2.1 R1.3 R4.7

CS How does one refer to records? Problem: Records can be on disk or in (virtual) memory. Need common address, but have different physical locations. (6) Indirection Addressing Rx Many options: PhysicalIndirect

CS Purely Physical Addressing Device ID E.g., RecordCylinder # Address=Track # ( ID ) Block # Offset in block Block ID

CS Fully Indirect Addressing Solution: Record ID (Oracle: ROWID) as global address, maintain a map table. Map Table rec ID raddress a Physical addr. Rec ID

CS Tradeoff Flexibility Cost to move recordsof indirection (for deletions, insertions) (lookup) What to do : Options inbetween ? Physical Indirect

CS Ex #1 : Indirection in block Block Header A block:Free space R3 R4 R1R2

CS Block header - data at beginning that describes block May contain: - File ID (or RELATION or DB ID) - This block ID - Record directory - Pointer to free space - Type of block (e.g. contains recs type 4; is overflow, …) - Pointer to other blocks “like it” - Timestamp...

CS Ex. #2 Use logical block #’s understood by file system instead of direct disk access REC ID File ID Block # Record # or Offset File ID,Physical Block #Block ID File System Map

CS (1) Separating records (2) Spanned vs. Unspanned (3) Mixed record types - Clustering (4) Split records (5) Sequencing (6) Indirection Recap: Storing records in blocks

CS (1) Insertion/Deletion (2) Buffer Management (3) Comparison of Schemes Other Topics in Chapter 12

CS Block Deletion Rx

CS Options: (a)Deleted and immediately reclaim space (b)Mark deleted –May need chain of deleted records (for re-use) –Need a way to mark: special characters delete field in map

CS As usual, many tradeoffs... How expensive is to move valid record to free space for immediate reclaim? How much space is wasted? –e.g., deleted records, delete fields, free space chains,...

CS Dangling pointers Note: If pointers point to physical locations (rather than ROWIDs), storing new data in deleted block corrupts data. Concern with deletions R1?

CS Solution #1: Do not worry

CS E.g., Leave “MARK” in map or old location Solution #2: Tombstones Physical IDs A block This spaceThis space can never re-usedbe re-used

CS Logical IDs IDLOC 7788 map Never reuse ID 7788 nor space in map... E.g., Leave “MARK” in map or old location Solution #2: Tombstones

CS Place record ID within every record When you follow a pointer, check if it leads to correct record Solution #3 (?): Does this work??? If space reused, won’t new record have same ID? to 3-77 rec-id: 3-77

CS Easy case: Records fixed length/not in sequence  Insert new record at end of file  or, in deleted slot A little harder:  If records are variable size, not as easy  may not be able to reuse space – fragmentation Hard case: records in sequence  If free space “close by”, not too bad...  Or, use overflow idea...  Or worst case, reorganize file... Insert

CS Interesting problems: How much free space to leave in each block, track, cylinder? How often do I reorganize file + overflow? Free space

CS DB features needed Why LRU may be bad Read Pinned blocks Textbook! Forced output Double buffering Swizzling Buffer Management

CS Pointer Swizzling Memory Disk Rec A block 1 Rec A block 2 block 1 Issue : If records (objects) contain pointers to other objects, translate locations when load objects into memory.

CS TranslationDB Addr Mem Addr Table Rec-A Rec-A-inMem One Option: Solution: Insert fields that represent pointers into map table. Translate pointers as needed.

CS In memory pointers - need “type” bit to disk to memory M Another Option:

CS Swizzling Issues Automatic On-demand No swizzling / program control Swizzling Options Must ‘unswizzle’ Updating/writing of records

CS There are 1,000,001 ways to organize my data on disk… Which is right for me? Comparison

CS Issues: FlexibilitySpace Utilization ComplexityPerformance

CS To evaluate a given strategy, compute following parameters: -> space used for expected data - on average -> expected time to : - fetch record given key - fetch record with next key - insert/delete/update record - read complete file - reorganize file (maybe sort) -> usage patterns / workload: - how many/which user queries/updates

CS Chapter 13 in book How to find a record quickly, given a key NEXT