1. Database Management 6. course

2. OS and DBMS DMBS DB OS DBMS DBA USER DDL DML WHISHESWHISHES RULESRULES

3. Steps of a query 1.User ”asks” of the DBMS (SQL query) 2.DBMS checks the permission in the schema 3.DBMS checks the permission in the subschema 4.DBMS asks the OS to execute the I/O operation 5.OS looks for the asked record 6.OS imports the record into the system buffer 7.OS notifies the DBMS 8.Record is taken the the user workspace 9.DBMS notifies the user about the recieved data

4. Data storage: disks and files

5. DBMS stores the data on mass storage device (disc, drive) Important consequences on DBMS design (I/O) – READ: reading data from disc to memory (RAM) – WRITE: writing data from memory to disc – Both are time consuming, careful design is needed

6. Why not storing everything in RAM? Costs too much 1GB~10 € (HDD 0,5 €) RAM volatilis: data is lost when unplugged Tipical way of storage: – The actual, used data is in memory – Secondary storage is on HDD (local server, cloud) – Tertiary storage: optical disks, tapes

7. Storage on disks The unit of reading from disc is disc block Speed of reading depends on the location of the block – The order of blocks influences the performance of the DBMS

8. Components Platter rotates Disk head reads the desired track One head is active Size of the sector is fixed Cylinder: parallel tracks

9. Reading a block Access time of a block: – Seek time: moving the arm to the appropriate track – Rotational delay: while the block gets under the head – Transfer time: 1ms/4KB I/O optimization means reducing the seek time and rotational delay

10. Order of data It is worthy to store frequently used blocks close to each other – Same block – Same track, same cylinder – Adjacent cylinder Reading is sequential If multiple blocks are read in sequentially, then much time is saved

11. Way of storage - RAID Redundant Array of Inexpensive/Independent Data Connecting disks logically, storing data redundantly Aims: – Minimizing data loss, increase reliability – Increasing capacity by more smaller/cheaper disks – Increase data access performance – Increase flexibility (can be replaced during usage)

12. Two main techniques Data striping: Data is partitioned into striping units and the partitions are distributed on several disks Redundancy: Data is strored redundantly so that reconstruction of data in case of disk failure is possible

13. Levels of RAID – Level 0 Non redundant If one of the disks fails, data is lost Parallel reading/writing If the capacity of the disks is different then the performance depends on the worst disk

14. RAID Levels – Level 1 Mirrored, the data is the same on every disk If one of the disks fails then data can be reconstructed Parallel reading with increased velocity Parallel writing with normal velocity If the capacity of the disks is different then the performance depends on the worst disk Does not use data striping

15. RAID Levels – Level 2 Uses data striping (unit=1 bit) but some of the disks are used to store error-correcting codes ECC: redundant bits calculated from data bits (compress) In the strip the corresponding strip’s error correcting code is stored. Not used any more (HDDs handle error correction)

16. RAID Levels – Level 3 Bit-Interleaved Parity Cannot identify the failed disk (disk controllers do that) One check disk with parity information The failed disk’s data can be recovered Can process only one I/O at a time Strips=1 bit

17. RAID Levels – Level 4 Block-Interleaved Parity Like RAID 3, with strips as disk blocks Supports serving multiple users Parity disk needs to be updated at every write, can be bottle neck In case of disk failure, reading speed reduces

18. RAID Levels – Level 5 Block-Interleaved Distributed Parity Rotating parity: parity is not stored on a single check disk, but uniformly over all disks Parallel read and write Similar to RAID 3 and 4 depending on the size of strips If a disks fails, it has to be replaced inmediately otherwise if another fails, all data will be lost

19. RAID 5 Capacity= min_capacity*(no of disks-1) Reading speed=min_speed*(no of disks-1)

20. RAID Levels – Level 6 High possibility of the failure of a second disk during disk recovery Needs 2 check disks Able to recover from up to two simultaneous disk failures Read and write speed is equal to RAID 5

21. RAID 0+1 and RAID 10 RAID 0+1: speed of RAID 0 and redundancy of RAID 1 Min 4 disks RAID 10: first mirroring, then connecting If a disk fails, only that RAID 1 is involved

22. Disk space and buffering

23. Disk space management The lowest level of DBMS manages the space Unit of data: page Size of page=size of disk block Higher levels can – Allocate and delete pages – Write / read pages If a query is given for multiple pages, it is worthy to store them sequentially Allows higher levels of DBMS to think of the data as a collection of pages (details are hidden)

24. Keeping track of free blocks As records are deleted holes occur on the disks Disk space manager can – Maintain a list of free blocks with pointer to the first free block – Maintain a bitmap with one bit for each block: block is used or not

25. Using OS to manage disk space Possible, not common Disadvantages: – Not portable: different OS platforms with different file systems – On 32-bit systems the largest file size is 4GB, DB may use bigger files, but OS files cannot span disk devices which is necessary in a DBMS.

26. Buffer manager Data has to be imported into the memory (RAM) to use it pares are stored in tables DB Memory Disc page free frame Page requests BUFFER POOL If a requested page is not in the pool and the pool is full, the buffer manager’s replacement policy controls which existing page is replaced.

27. When a request comes… If the page is not in the buffer: – Choose a frame to replace, incerase its pin count – If the dirty bit for the replacement frame is on, write it on the disk – Reads the requested page into the replacement frame Return the address of the frame to the requestor If it can be predicted that which page will be requested next, then multiple pages can be read (pre-fetching)

28. Buffer management The requestor has to unpin the request Mark if the content of the page is modified – With the dirty bit The page in the buffer can be called multiple times by processes/transactions – Pin_count: page can be replaced if and only if pin_count=0 Concurrency handling and rollback handling can influence the replacement policy

29. Buffer replacement policies Least-recently-used (LRU): counts what was used and when (costs a lot) Clock replacement – Current frame is stored Goes to the next until pin count=0 and referenced bit is off (not used) – After the last, jumps to the first (like a circle) Sequential flooding: ???

30. Files and indexes

31. Records in files Pages and block are low-level definitions, DBMS handles records and files Files: collection of pages containing records They must support – DML (insert, update, delete) – Read records (identified by rid) – Read all the records (satisfying some conditions)

32. Unordered (heap) files Simplest file structure For the record-level operations DBMS must register – pages in the file – free space in the page – records in the page There are many alternative solutions

33. Heap file as a linked list Address of the header page and the name of the heap file must be stored in a known location Every page contains two pointers in addition Header Page Data Page Data Page Data Page Data Page Data Page Data Page Pages with free space Full pages

34. Disadvantages – Every page is in the list of free records if they have variable length – To insert a record, we must examine several pages before finding enough space

35. Directory-based heap file Maintain directory of pages DBMS stores the address of the first page of each heap file Directory=collection of pages (e.g. chained list) Counter for every page: amount of free space/entry Data Page 1 Data Page 2 Data Page N Header Page DIRECTORY

36. Index With heap file it is possible to search for a concrete rid Read the records sequentially We often need records with specific conditions for its attributes (e.g. all CLERCKs) Indexes make possible value-based queries

37. Example, library 1. lokate books of Asimov 2. Search for Foundation

38. Indexed file: Give a search key for the entries (records in files), calculate the index of this key, look for it Goal: speed up search E.g. I am looking for employees of a given age, then I can build an index which might contain pairs The pages of the index files are organized based on the indexes to find the result quickly (access methods)

39. Access methods B trees B+ trees Hash-based structures Discussed in detail later

40. Page formats Higher level of the DBMS handles data as a collection of records Page~collection of slots, each slot contains a record Record identification: – =rid – Number every record and store its location in a table

41. Fixed-length records All records have the same length Insertion: locate empty slot, place there Main issue: – Keep track of empty slots – Locate all records on a page

42. Deletion alternatives – first option Store records in the first N slots without gap If a record is deleted, the last record is moved to the gap Advantage: finding location is easy (just offset calculation) The empty slots remain together at the end of the page Disadvantge: if the moved record is referred externally (the rid changes)

43. Second option Using an array of bits, one bit/slot If record is deleted, its bit turns off Summary: Every page contains additional file- level info (array of bits, address of the next page…)

44. Variable-length records If new record is to be inserted, enough and not too big space is needed (do not waste) If deleted, move the others to fill the hole Most flexible organization: directory of slots for each page

45. Directory of slots Offset (pointer) and length of the records are stored Deletion: set offset to -1 Records can be moved since rid=(page number,slot number) does not change Only the offset of record changes The offset of the free space is stored

46. When new record is inserted and there is not enough space, records are moved If a record is deleted the number of the rest record cannot be changed due to external references If a record is inserted, a missing number should be given to it

48. Record formats Number of fields and field types are stored in the system catalog

49. Fixed-length records Each field has fixed By the offset of the record the offset of each field can be calculated easily: Base address (B) L1L2L3L4 F1F2F3F4 Address = B+L1+L2

50. Variable-length records Variable length fields (e.g. varchar2) Two formats: – Separators are used – Array of integer offsets at the beginning of the record

51. Array of integer offsets The offset of the end of the record is stored Disadvantage – Storage overhead Advantages – Direct access to the fields – NULL: start of the field=end of the field

52. Issues When insert or modify, move the other fields – Page modification may cause a problem – Forwarding address is left on the page When a record is too big for one page – Break record to smaller records – Chain them

53. Thank you for your attention!