1. 1 Representing Data Elements Fields, Records, Blocks Variable-length Data Modifying Records Source: our textbook

2. 2 Overview uAttributes are represented by sequences of bytes, called fields uTuples are represented by collections of fields, called records uRelations are represented by collections of records, called files uFiles are stored in blocks, using specialized data structures to support efficient modification and querying

3. 3 Representing SQL Data Types uintegers and reals: built-in uCHAR(n): array of n bytes uVARCHAR(n): array of n+1 bytes (extra byte is either string length or null char) udates and times: fixed length strings uetc.

4. 4 Representing Tuples uFor now, assume all attributes (fields) are fixed length. uConcatenate the fields uStore the offset of each field in schema 0 30286287297 name CHAR(30) 30 bytes address VARCHAR(255) 256 bytes gender CHAR(1) 1 byte birthdate DATE 10 bytes

5. 5 More on Tuples uDue to hardware considerations, certain types of data need to start at addresses that are multiples of 4 or 8 uPrevious example becomes: 0 32288292304 name CHAR(30) 30 bytes + 2 address VARCHAR(255) 256 bytes gender CHAR(1) 1 byte + 3 birthdate DATE 10 bytes + 2

6. 6 Record Headers uOften it is convenient to keep some "header" information in each record: wa pointer to schema information (attributes/fields, types, their order in the tuple, constraints) wlength of the record/tuple wtimestamp of last modification

7. 7 Packing Records into Blocks uStart with block header: wtimestamp of last modification/access woffset of each record in the block, etc. uFollow with sequence of records uMay end with some unused space headerblock 1block 2 … block n-1block n

8. 8 Representing Addresses uOften addresses (pointers) are part of records: wthe application data in object-oriented databases was part of indexes and other data structures supporting the DBMS uEvery data item (block, record, etc.) has two addresses: wdatabase address: address on the disk (typically 8-16 bytes) wmemory address, if the item is in virtual memory (typically 4 bytes)

9. 9 Translation Table uProvides mapping from database addresses to memory addresses for all blocks currently in memory uLater we'll discuss how to implement it

10. 10 Pointer Swizzling uWhen a block is moved from disk into main memory, change all the disk addresses that point to items in this block into main memory addresses. uNeed a bit for each address to indicate if it is a disk address or a memory address. uWhy? Faster to follow memory pointers (only uses a single machine instruction).

11. 11 Example of Swizzling Block 1 Block 2 DiskMain Memory read into main memory

12. 12 Swizzling Policies uAutomatic swizzling: as soon as block is brought into memory, swizzle all relevant pointers uSwizzling on demand: only swizzle a pointer if and when it is actually followed uNo swizzling uProgrammer control

13. 13 Automatic Swizzling uLocating all pointers within a block: wrefer to the schema, which will indicate where addresses are in the records wfor index structures, pointers are at known locations uUpdate translation table with memory addresses of items in the block uUpdate pointers in the block (in memory) with memory addresses, when possible, as obtained from translation table

14. 14 Unswizzling uWhen a block is moved from memory back to disk, all pointers must go back to database (disk) addresses uUse translation table again uImportant to have an efficient data structure for the translation table

15. 15 Pinned Records and Blocks uA block in memory is pinned if it cannot be safely written back to disk uIndicate with a bit in the block header uReasons for pinning: wrelated to failure recovery (more later) wbecause of pointer swizzling uIf block B1 has swizzled pointer to an item in block B2, then B2 is pinned.

16. 16 Unpinning a Block uConsider each item in the block to be unpinned uKeep in the translation table the places in memory holding swizzled pointers to that item (e.g., with a linked list) uUnswizzle those pointers (i.e., use translation table to replace the memory addresses with database (disk) addresses

17. 17 Variable Length Data uData items with varying size (e.g., if maximum size of a field is large but most of the time the values are small) uVariable-format records (e.g., NULLs method for representing a hierarchy of entity sets as relations) uRecords that do not fit in a block (e.g., an MPEG of a movie)

18. 18 Variable-Length Fields uStore the fixed-length fields before the variable-length fields in each record uKeep in the record header wrecord length wpointers to the beginnings of all the variable-length fields uBook discusses variations on this idea

19. 19 Variable-Format Records uRepresent by a sequence of tagged fields uEach tagged field contains wname wtype wlength, if not deducible from the type wvalue

20. 20 Splitting Records Across Blocks uCalled spanned records uUseful when wrecord size exceeds block size wputting an integral number of records in a block wastes a lot of the block (e.g., record size is 51% of block size) uEach record or fragment header contains wbit indicating if it is a fragment wif fragment then pointers to previous and next fragments of the record (i.e., a linked list)

21. 21 Record Modification uModifications to records: winsert wdelete wupdate uissues even with fixed-length records and fields ueven more involved with variable- length data

22. 22 Inserting New Records uIf records need not be any particular order, then just find a block with enough empty space uLater we'll see how to keep track of all the tuples of a given relation uBut what if blocks should be kept in a certain order, such as sorted on primary key?

23. 23 Insertion in Order If there is space in the block, then add the record (going right to left), add a pointer to it (going left to right) and rearrange the pointers as needed.

24. 24 What if Block is Full? uRecords are stored in several blocks, in sorted order uOne approach: keep a linked list of "overflow" blocks for each block in the main sequence uAnother approach is described in the book

25. 25 Deleting Records uTry to reclaim space made available after a record is deleted uIf using an offset table, then rearrange the records to fill in any hole that is left behind and adjust the pointers uAdditional mechanisms are based on keeping a linked list of available space and compacting when possible

26. 26 Tombstones uWhat about pointers to deleted records? uWe place a tombstone in place of each deleted record uTombstone is permanent uIssue of where to place the tombstone uKeep a tombstone bit in each record header: if this is a tombstone, then no need to store additional data

27. 27 Updating Records uFor fixed-length records, there is no effect on the storage system uFor variable-length records: wif length increases, like insertion wif length decreases, like deletion except tombstones are not necessary