1. CS522 Advanced database Systems
9/12/2018
CS522 Advanced database Systems
5. Files and ISAM
Huiping Guo
Department of Computer Science
California State University, Los Angeles

2. Files of records page record field 5. Files and ISAM CS522_S16
9/12/2018
Files of records
page
record
field
Page or block is OK when doing I/O, but higher levels of DBMS operate on records, and files of records.
5. Files and ISAM CS522_S16

3. Implementing heap files
As file grows and shrinks, disk pages are allocated and de-allocated.
Supported operations on a heap file:
Create/destroy files
Insert/delete/update a record with a given rid
Scan all records in the file
How to find the page that contains a given record
Rid
5. Files and ISAM CS522_S16

4. Unordered (Heap) Files
To support scans, we must
keep track of the pages in a file
To support insertions efficiently, we must
keep track of free space on pages
To support record level operations, we must:
keep track of the records on a page
There are many alternatives for keeping track of this
5. Files and ISAM CS522_S16

5. Linked list of pages
Maintain a heap file as a doubly linked list of pages
Header page – the first page
DBMS keeps a table containing pairs of <heap_file_name, header_page_addr)
How to keep track of free space within a page
To be discuss later
How to keep track of pages that have some free space?
Maintain a doubly linked list of pages with free space and another doubly linked list of full pages
5. Files and ISAM CS522_S16

6. Linked list of pages Each page contains 2 `pointers’ plus data
Full Pages
Header
Page
Data
Page
Data
Page
Data
Page
Pages with
Free Space
Each page contains 2 `pointers’ plus data
Each pointer is actually the page id
Disadvantage
All pages in a file will be on the free list of records of variable length
5. Files and ISAM CS522_S16

7. Heap File Using a Page Directory
The directory is itself a collection of pages
DBMS keeps the location of the header page
Each directory entry identifies a page ( or a sequence of pages) in the heap file
As the heap file grows or shrinks, the number of entries and the number of directory pages grow or shrink correspondingly
Free space can be managed by maintaining
A bit per entry, indicating whether the corresponding page has any free space or not
A count per entry, indicating the amount of free space on the page.
Works well for variable – length records
5. Files and ISAM CS522_S16

8. Heap File Using a Page Directory
Data
Page 1
Header
Page
Data
Page 2
Data
Page N
DIRECTORY
Since several entries fit on a directory page, we can efficiently search for a data page with enough space to hold a record to be inserted
5. Files and ISAM CS522_S16

9. Page Formats
The page abstraction is appropriate when dealing with I/O issue
Higher levels of DBMS see data as a collection of records
How arrange a collection of records on a page?
A page is treated as a collection of slots
Each slot contains a record
A record is identified by a pair (page id, slot number) – this is the record id(rid)
5. Files and ISAM CS522_S16

10. Page Formats: Fixed Length Records
Slot 1
Slot 2
Slot N
. . . N M 1 M
PACKED
UNPACKED, BITMAP
Free
Space
Slot M
number
of records
of slots
rid = (page id, slot number)
5. Files and ISAM CS522_S16

11. Page Formats: Variable Length Records
Page i
Rid = (i,N)
Rid = (i,2)
Rid = (i,1)
Pointer
to start
of free
space
SLOT DIRECTORY N 20 16 24 N
# slots
5. Files and ISAM CS522_S16

12. Page Formats: Variable Length Records
For each page, maintain a directory of slots
Each slot has a pair <record offset, record length>
Record offset is the offset in bytes from the start of the data area on the page to the start of record
Deletion is rapidly accomplished by setting the record offset to -1
Records can be moved around on the page
The rid, which is the page number and slot number(the position in the directory) doesn’t change when the record is moved
Only the offset in the slot changes
5. Files and ISAM CS522_S16

13. Page Formats: Variable Length Records
The page is NOT preformatted into slots
How to maintain free space for new records?
Maintain a pointer that indicates the start of the free space area.
When a new record is too large to fit into the remaining free space, we have to move records on the page to reclaim the space freed by deleted records.
After reorganization, all records appear in contiguous order, followed by the available free space
5. Files and ISAM CS522_S16

14. Page Formats: Variable Length Records
A slot for deleted record cannot always be removed from the slot directory
Slot numbers are used to identify records
If a slot is deleted, the slot numbers of subsequent slots in the slot directory are also changed, and the rids of records pointed by subsequent slots are changed
The only way to remove slots from slot directory is to remove the last slot if the record that it points to is deleted
When a record is inserted, the slot directory should be scanned for an element that currently does not point to any record
This slot should be used for the new record
A new slot is inserted only if all existing slots point to records
5. Files and ISAM CS522_S16

15. Record format How to organized fields within a record? System catalog
The fields of record are of fixed length or variable length?
The cost of various operations on the record: retrieving and modification of fields
System catalog
A description of the contents of a database, maintained by DBMS
Information common to all records of a given record type is stored in the system catalog
5. Files and ISAM CS522_S16

16. Record Formats: Fixed Length
9/12/2018
Record Formats: Fixed Length
Base address (B) L1 L2 L3 L4 F1 F2 F3 F4
Address = B+L1+L2
5. Files and ISAM CS522_S16 9

17. Record Formats: Variable Length
9/12/2018
Record Formats: Variable Length 4 $
Field
Count
Fields Delimited by Special Symbols
F F F F4
F F F F4
Array of Field Offsets
Require a scan of the records to locate a desired field
5. Files and ISAM CS522_S16 10

18. Record Formats: Variable Length
9/12/2018
Record Formats: Variable Length
Reserve some space at the beginning of a record for use as an array of integer offset
The ith integer in this array is the starting address of the ith field value relative to the start of the record
Get direct access to any field for the overhead of the offset array 4 $
Field
Count
Fields Delimited by Special Symbols
F F F F4
F F F F4
Array of Field Offsets
5. Files and ISAM CS522_S16 10

19. Record Formats: Variable Length
9/12/2018
Record Formats: Variable Length
Null values
A null value is a special value used to denote that the value for a field is unavailable or inapplicable
How to deal with NULL values?
If a field contains a null value, the pointer to the end of the field is set to be the same as the pointer to the beginning of the field
No space is used for representing the null values
A comparison of the pointers to the beginning and the end of the field is used to determine that the value in the field is null
5. Files and ISAM CS522_S16 10

20. Variable Length: some issues
Modifying a field may cause it grow
All subsequent fields need to be shifted to make space for the modification
A modified record may no longer fit into the space remaining on its page
A record may grow so large that it no longer fits on any one page
5. Files and ISAM CS522_S16

21. Review of data storage Disks Disk space manager Buffer manager
Structure
What can we learn?
Disk space manager
Buffer manager
Files of records
File format
Page format
Record format
5. Files and ISAM CS522_S16

22. Keys to reduce I/O costs
Reduce number of page I/Os
Indexing
Reduce the access time of retrieved pages
Sequentially store the pages
5. Files and ISAM CS522_S16

23. Index Issues Search key Index pages Are the records sorted, hashed??
What is stored in an entry?
How are data entries organized?
Are the records sorted, hashed??
Clustered or un-clustered
Primary or secondary
5. Files and ISAM CS522_S16

24. Data entries in an index file
Actual data record
<key,rid>
<key, list of rid>
5. Files and ISAM CS522_S16

25. Organization of data Entries
Tree-structured
ISAM, B+tree, R-tree, Quad-tree,
Hash-based
Static, dynamic
Other
Bitmap, signature,...
5. Files and ISAM CS522_S16

26. Tree-structured indexing techniques
Two structures
ISAM: Indexed Sequential Access Method
static structure
B+ tree:
dynamic, adjusts gracefully under inserts and deletes
Benefits of tree indexing
Reduce search time
Reduce number of page I/Os
Fan-out
Average number of children for a non-leaf node
5. Files and ISAM CS522_S16

27. Intuition for tree indexing
9/12/2018
Intuition for tree indexing
``Find all employees with sal > 50k’’
Binary search
Suppose all records are sorted by sal
Page I/O: log2N
Page 1
Page 2
Page N
Page 3
5. Files and ISAM CS522_S16 3

28. Intuition for tree indexing (cont.)
K1’’
K1’
Km’
Index File k1 k2 kN
Page 1
Page 2
Page 3
Page N
Data File
5. Files and ISAM CS522_S16

29. Format of an index page <key, pointer> index entry P K 1 2 mK 1 2 m
index entry
5. Files and ISAM CS522_S16

30. ISAM Leaf pages contain data entries Alternative 1 is used.
9/12/2018
ISAM
Non-leaf
Pages
Leaf
Pages
Index entries: <search key value, page id>; they `direct’ search for data entries, which are in leaf pages.
Overflow
page
Primary pages
Leaf pages contain data entries
Alternative 1 is used.
All leaf pages are allocated sequentially and sorted on the search key
5. Files and ISAM CS522_S16 4

31. More on ISAM Operation on ISAM File creation Search Insert Delete
9/12/2018
More on ISAM
Operation on ISAM
File creation
Search
Insert
Delete
5. Files and ISAM CS522_S16 5

32. 9/12/2018
Example ISAM Tree
Each node can hold 2 entries; no need for `next-leaf-page’ pointers. (Why?)
10*
15*
20*
27*
33*
37*
40*
46*
51*
55*
63*
97* 20 33 51 63 40
Root
5. Files and ISAM CS522_S16 6

33. Inserting 23*, 48*, 41*, 42* ... Root Index Pages Primary Leaf Pages
9/12/2018
Inserting 23*, 48*, 41*, 42* ...
Root
Index 40
Pages 20 33 51 63
Primary
Leaf
10*
15*
20*
27*
33*
37*
40*
46*
51*
55*
63*
97*
Pages
23*
48*
Overflow
41*
Pages
42*
5. Files and ISAM CS522_S16 7

34. Then deleting 42*, 51*, 97*,55* Root Index Pages Primary Leaf Pages
9/12/2018
Then deleting 42*, 51*, 97*,55*
Root
Index 40
Pages 20 33 51 63
Primary
Leaf
10*
15*
20*
27*
33*
37*
40*
46*
51*
55*
63*
97*
Pages
23*
48*
Overflow
41*
Pages
42*
5. Files and ISAM CS522_S16 8

35. Pros and Cons of ISAM Pros Cons Good for static databases
Good for concurrency control
Delete/insert affect only leaf pages
Cons
Long chain of overflow pages may develop
5. Files and ISAM CS522_S16