CPSC-608 Database Systems Fall 2017 Instructor: Jianer Chen Office: HRBB 315C Phone: 845-4259 Email: chen@cse.tamu.edu Notes #7

Graduate Database DBMS lock table DDL language DDL complier file administrator DDL complier lock table DDL language file manager logging & recovery concurrency control transaction manager database programmer index/file manager buffer manager query execution engine DML complier main memory buffers DML (query) language secondary storage (disks) DBMS Graduate Database

Graduate Database DBMS lock table DDL language DDL complier file administrator DDL complier lock table DDL language file manager logging & recovery concurrency control transaction manager database programmer index/file manager buffer manager query execution engine DML complier main memory buffers DML (query) language secondary storage (disks) DBMS Graduate Database

Computer Memory Hierarchy

A Typical Computer ... CPU main memory Secondary Storage bus Disk controller Secondary Storage disks https://www.youtube.com/watch?v=4iaxOUYalJU

Main Memory fast small capacity (gigabytes) volatile Disks slow large capacity (100’s gigabytes) non-volatile

Typical Disk … Terms: Platter, Head, Cylinder, Track, Sector, Gap

Top View Track Sector Gap

Original image © IBM Corporation Top view of a 36 GB, 10,000 RPM, IBM SCSI server hard disk, with its top cover removed. Note the height of the drive and the 10 stacked platters. (The IBM Ultrastar 36ZX.) Original image © IBM Corporation Video show at https://www.youtube.com/watch?v=9eMWG3fwiEU

A “typical” disk 5 platters (thus 10 surfaces) A surface has 20,000 tracks A track has 500 sectors (million bytes) A sector has several thousand bytes Disk makes 5000 revolutions per minute (so about 10 millisecond per rotation)

Blocks A (logic) block = one or several sectors (typical size 16KB) Block address Physical device Cylinder # Surface # Sector

? Disk Access Time I want block X block X in memory Time = Seek Time + Rotational Delay + Transfer Time + Other

Seek Time 3 or 5x x 1 N Cylinders Traveled Time

Average Random Seek Time   SEEKTIME (i  j) S = N(N-1) i=1 j=1 ji typical seek time: 10 ms  40 ms

Rotational Delay Average Rotational Delay R = 1/2 revolution Head here Block I want Average Rotational Delay R = 1/2 revolution typical rotational delay = 8 ms

Transfer Rate: typical: t = 80 MB/second = 80 KB/millisecond transfer time: block size / t ~ 10/80 < 1 ms

Other Delays Typical value: ≈ 0 CPU time to issue I/O Contention for controller Contention for bus, memory Typical value: ≈ 0

Thus, reading a block of 16K bytes: Time = Seek Time + Rotational Delay + Transfer Time + Other ~ 30 ms + 8 ms + 16/80 ms + 0 ~ 40 ms

slow (read/write: 1~40 millisecond) large capacity (100’s gigabytes) Disks slow (read/write: 1~40 millisecond) large capacity (100’s gigabytes) non-volatile Main Memory fast (read/write: 10-100 nanosecond) small capacity (gigabytes) volatile Disks are about 105~106 times slower than main memory

I/O Model of Computation Dominance of I/O cost: if a block needs to be moved between disk and main memory, then the time taken to perform the read/write is much larger than the time likely to be used to manipulate that data in main memory. The number of disk block reads/writes is a good approximation to the entire computation.

Disk I/O Optimization: Example I

Disk I/O Optimization: Example I Optimizing Disk Seek Time (by disk controller):

Disk I/O Optimization: Example I Optimizing Disk Seek Time (by disk controller): On a (dynamic) sequence of disk I/O requests, how do we order the requests to minimize the seek time?

Disk I/O Optimization: Example I Optimizing Disk Seek Time (by disk controller): On a (dynamic) sequence of disk I/O requests, how do we order the requests to minimize the seek time? (seeking tracks is the most time consuming component of disk I/O. Moving to a nearer track takes less time.)

Disk I/O Optimization: Example I Optimizing Disk Seek Time (by disk controller): On a (dynamic) sequence of disk I/O requests, how do we order the requests to minimize the seek time? (seeking tracks is the most time consuming component of disk I/O. Moving to a nearer track takes less time.) Elevator Algorithm: Let the head move along its current direction, process each encountered request on the way until no request is ahead. Then reverse the direction.

Elevator Algorithm Keep an UpperQ and a LowerQ, and elevator’s current Direction and Position. Repeat 1. If Direction = Up Then If UpperQ   Then x = Min(UpperQ); Position = x; Delete(UpperQ, x); Else Direction = Down; 2. Else If LowerQ   Then x = Max(LowerQ); Position = x; Delete(LowerQ, x); Else Direction = Up.

Disk I/O Optimization: Example II Reducing the number of disk I/Os. Example. Sorting on disk Each tuple (with a key) takes 160 bytes Each block holds 100 tuples (16KB) A relation R has 10M tuples (1.6 GB, 100K blocks) Main memory has 100MB (6400 blocks) A disk read/write: 40 ms

Main memory sorting algorithms disk read/write: 40 ms a tuple: 160 bytes a block: 16KB (100 tuples) a relation R: 1.6 GB (10M tuples, 100K blocks) main memory: 100MB (6400 blocks) Main memory sorting algorithms heap sort: 10M * log2 (10M) = 230M disk block read/write = 9200M ms = 9200000 seconds > 100 day quick sort and merge sort: 2 * 100K (blocks) * log2 (10M) = 4.6M disk block read/write = 184M ms = 184000 seconds > 2 day

Two-phase Multiway MergeSort disk read/write: 40 ms a tuple: 160 bytes a block: 16KB (100 tuples) a relation R: 1.6 GB (10M tuples, 100K blocks) main memory: 100MB (6400 blocks) Two-phase Multiway MergeSort Phase 1. making sorted sublist repeat fill the main memory with remaining tuples in R and sort them; write the sorted sublist (of 6400 blocks) back to disk Phase 2. Merging bring in a block from each of the sorted sublist; merge them and put in an “output” block; write the “output” block back to disk when it is full

Two-phase Multiway MergeSort Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Sort it Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Sort it Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Sort it Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Sort it Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Main memory Two-phase Multiway MergeSort Disk

Second Phase Main memory Disk

Two-phase Multiway MergeSort Second Phase Main memory One block per sublist Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort Main memory One block per sublist Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort Main memory One block per sublist Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort Main memory One block per sublist Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort Main memory One block per sublist Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort Main memory One block per sublist Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort disk read/write: 40 ms a tuple: 160 bytes a block: 16KB (100 tuples) a relation R: 1.6 GB (10M tuples, 100K blocks) main memory: 100MB (6400 blocks) Two-phase Multiway MergeSort # sublists = 100K/6400 = 16 thus, in phase 2, we can easily hold a block for each sublist in the main memory Disk block read/write: 100K (blocks) * 4 = 400K disk block read/write = 16M ms = 16000 seconds < 4.5 hours

Graduate Database DBMS lock table DDL language DDL complier file administrator DDL complier lock table DDL language file manager logging & recovery concurrency control transaction manager database programmer index/file manager buffer manager query execution engine DML complier main memory buffers DML (query) language secondary storage (disks) DBMS Graduate Database

Read Chapter 13 for more details on memory structures