CPSC-608 Database Systems Fall 2018 Instructor: Jianer Chen Office: HRBB 315C Phone: 845-4259 Email: chen@cse.tamu.edu Notes #11

Graduate Database DBMS lock table DDL language DDL complier file administrator DDL complier lock table DDL language file manager logging & recovery concurrency control transaction manager database programmer index/file manager buffer manager query execution engine DML complier main memory buffers DML (query) language secondary storage (disks) DBMS Graduate Database

Graduate Database DBMS lock table DDL language DDL complier file administrator DDL complier lock table DDL language file manager logging & recovery concurrency control transaction manager database programmer index/file manager buffer manager query execution engine DML complier main memory buffers DML (query) language secondary storage (disks) DBMS Graduate Database

Two-phase Multiway MergeSort disk read/write: 40 ms a tuple: 160 bytes a block: 16KB (100 tuples) a relation R: 1.6 GB (10M tuples, 100K blocks) main memory: 100MB (6400 blocks) Two-phase Multiway MergeSort Phase 1. making sorted sublists repeat fill the main memory with remaining tuples in R and sort them; write the sorted sublist (of 6400 blocks) back to disk Phase 2. Merging bring in a block from each of the sorted sublist; merge them and put in an “output” block; write the “output” block back to disk when it is full

Two-phase Multiway MergeSort Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Sort it Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Sort it Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Sort it Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Sort it Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort First Phase Main memory Two-phase Multiway MergeSort Disk

Second Phase Main memory Disk

Two-phase Multiway MergeSort Second Phase Main memory One block per sublist Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort Main memory One block per sublist Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort Main memory One block per sublist Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort Main memory One block per sublist Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort Main memory One block per sublist Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort Main memory One block per sublist Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort Main memory Two-phase Multiway MergeSort Disk

Two-phase Multiway MergeSort disk read/write: 40 ms a tuple: 160 bytes a block: 16KB (100 tuples) a relation R: 1.6 GB (10M tuples, 100K blocks) main memory: 100MB (6400 blocks) Two-phase Multiway MergeSort # sublists = 100K/6400 = 16 thus, in phase 2, we can easily hold a block for each sublist in the main memory Disk block read/write: 100K (blocks) * 4 = 400K disk block read/write = 16M ms = 16000 seconds < 4.5 hours

Read Chapter 13 for more details on memory structures

Graduate Database DBMS lock table DDL language DDL complier file administrator DDL complier lock table DDL language file manager logging & recovery concurrency control transaction manager database programmer index/file manager buffer manager query execution engine DML complier main memory buffers DML (query) language secondary storage (disks) DBMS Graduate Database

Graduate Database DBMS lock table DDL language DDL complier file administrator DDL complier lock table DDL language file manager logging & recovery concurrency control transaction manager database programmer index/file manager buffer manager query execution engine DML complier main memory buffers DML (query) language secondary storage (disks) DBMS Graduate Database

Graduate Database DBMS lock table DDL language DDL complier file administrator DDL complier lock table DDL language file manager logging & recovery concurrency control transaction manager database programmer index/file manager buffer manager query execution engine DML complier main memory buffers DML (query) language secondary storage (disks) DBMS Graduate Database

What Does DBMS Do?

What Does DBMS Do? Execute input database programs; Manage system efficiency, data consistency, and system/data reliability.

What Does DBMS Do? How? Execute input database programs; Manage system efficiency, data consistency, and system/data reliability. How?

What Does DBMS Do? How? Execute input database programs; Manage system efficiency, data consistency, and system/data reliability. How?

What Does DBMS Do? Prepare a collection A of efficient algorithms for operations in relational algebra;

What Does DBMS Do? Prepare a collection A of efficient algorithms for operations in relational algebra; For a given database program P:

What Does DBMS Do? Prepare a collection A of efficient algorithms for operations in relational algebra; For a given database program P: 1. understand the program P;

What Does DBMS Do? Prepare a collection A of efficient algorithms for operations in relational algebra; For a given database program P: 1. understand the program P; 2. translate the program P into an expression E in relational algebra;

What Does DBMS Do? Prepare a collection A of efficient algorithms for operations in relational algebra; For a given database program P: 1. understand the program P; 2. translate the program P into an expression E in relational algebra; 3. convert E into an algorithm using algorithms in the collection A;

What Does DBMS Do? Prepare a collection A of efficient algorithms for operations in relational algebra; For a given database program P: 1. understand the program P; 2. translate the program P into an expression E in relational algebra; 3. convert E into an algorithm using algorithms in the collection A; take care of issues in optimization, consistency, and security.

What Does DBMS Do? Prepare a collection A of efficient algorithms for operations in relational algebra; For a given database program P: 1. understand the program P; 2. translate the program P into an expression E in relational algebra; 3. convert E into an algorithm using algorithms in the collection A; take care of issues in optimization, consistency, and security.

Graduate Database DBMS lock table DDL language DDL complier file administrator DDL complier lock table DDL language file manager logging & recovery concurrency control transaction manager database programmer index/file manager buffer manager query execution engine DML complier main memory buffers DML (query) language secondary storage (disks) DBMS Graduate Database

What Does DBMS Do? Prepare a collection A of efficient algorithms for operations in relational algebra; For a given database program P: 1. understand the program P; 2. translate the program P into an expression E in relational algebra; 3. convert E into an algorithm using algorithms in the collection A; 4. take care of issues in optimization and security.

What Does DBMS Do? Prepare a collection A of efficient algorithms for operations in relational algebra; For a given database program P: 1. understand the program P; Break the program P into “words” (lexical analysis);

WHERE bar = ’Joe’’s Bar’ AND Frequents.drinker = Likes.drinker; Frequents(drinker, bar) Likes(drinker, beer) Example: SELECT beer FROM Likes, Frequents WHERE bar = ’Joe’’s Bar’ AND Frequents.drinker = Likes.drinker;

WHERE bar = ’Joe’’s Bar’ AND Frequents.drinker = Likes.drinker; Frequents(drinker, bar) Likes(drinker, beer) Example: SELECT beer FROM Likes, Frequents WHERE bar = ’Joe’’s Bar’ AND Frequents.drinker = Likes.drinker;

How? Example: SELECT beer FROM Likes, Frequents Frequents(drinker, bar) Likes(drinker, beer) Example: SELECT beer FROM Likes, Frequents WHERE bar = ’Joe’’s Bar’ AND Frequents.drinker = Likes.drinker; How?

How? Example: SELECT beer FROM Likes, Frequents Frequents(drinker, bar) Likes(drinker, beer) Example: SELECT beer FROM Likes, Frequents WHERE bar = ’Joe’’s Bar’ AND Frequents.drinker = Likes.drinker; How? Using the lexical rules of the programming language

lexical rules TinySQL grammar Characters a-z, 0-9, <, >, =, ., ,, ;, `, ’, (, ), [, ], +, -, *, / Lexical Rules letter ::= a | b | c | d | e | f | g | h | i | j | k | l | m | n | o | p | q | r| s | t | u | v | w | x | y | z digit ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 integer ::= digit+ comp-op ::= < | > | = table-name ::= letter(digit | letter)* attribute-name ::= letter(digit | letter)* column-name ::= [table-name.]attribute-name literal ::= “whatever-except-(`)-and-(’)” Keywords CREATE, TABLE, INT, STR20, DROP, SELECT, DISTINCT, FROM, WHERE, ORDER, BY, DELETE, FROM, INSERT, INTO, VALUES, NULL, OR, AND, NOT, TinySQL grammar lexical rules

*: means “zero or more” lexical rules |: means “or” Characters a-z, 0-9, <, >, =, ., ,, ;, `, ’, (, ), [, ], +, -, *, / Lexical Rules letter ::= a | b | c | d | e | f | g | h | i | j | k | l | m | n | o | p | q | r| s | t | u | v | w | x | y | z digit ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 integer ::= digit+ comp-op ::= < | > | = table-name ::= letter(digit | letter)* attribute-name ::= letter(digit | letter)* column-name ::= [table-name.]attribute-name literal ::= “whatever-except-(`)-and-(’)” Keywords CREATE, TABLE, INT, STR20, DROP, SELECT, DISTINCT, FROM, WHERE, ORDER, BY, DELETE, FROM, INSERT, INTO, VALUES, NULL, OR, AND, NOT, |: means “or” +: means “one or more” *: means “zero or more” [ ]: means “optional” TinySQL grammar lexical rules

*: means “zero or more” lexical rules |: means “or” Characters a-z, 0-9, <, >, =, ., ,, ;, `, ’, (, ), [, ], +, -, *, / Lexical Rules letter ::= a | b | c | d | e | f | g | h | i | j | k | l | m | n | o | p | q | r| s | t | u | v | w | x | y | z digit ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 integer ::= digit+ comp-op ::= < | > | = table-name ::= letter(digit | letter)* attribute-name ::= letter(digit | letter)* column-name ::= [table-name.]attribute-name literal ::= “whatever-except-(`)-and-(’)” Keywords CREATE, TABLE, INT, STR20, DROP, SELECT, DISTINCT, FROM, WHERE, ORDER, BY, DELETE, FROM, INSERT, INTO, VALUES, NULL, OR, AND, NOT, SELECT beer FROM Likes, Frequents WHERE bar = ’Joe’’s Bar’ AND Frequents.drinker = Likes.drinker; |: means “or” +: means “one or more” *: means “zero or more” [ ]: means “optional” TinySQL grammar lexical rules

lexical rules TinySQL grammar SELECT beer FROM Likes, Frequents Characters a-z, 0-9, <, >, =, ., ,, ;, `, ’, (, ), [, ], +, -, *, / Lexical Rules letter ::= a | b | c | d | e | f | g | h | i | j | k | l | m | n | o | p | q | r| s | t | u | v | w | x | y | z digit ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 integer ::= digit+ comp-op ::= < | > | = table-name ::= letter(digit | letter)* attribute-name ::= letter(digit | letter)* column-name ::= [table-name.]attribute-name literal ::= “whatever-except-(`)-and-(’)” Keywords CREATE, TABLE, INT, STR20, DROP, SELECT, DISTINCT, FROM, WHERE, ORDER, BY, DELETE, FROM, INSERT, INTO, VALUES, NULL, OR, AND, NOT, SELECT beer FROM Likes, Frequents WHERE bar = ’Joe’’s Bar’ AND Frequents.drinker = Likes.drinker; SCANNER: read the next char c; if c is not a letter or digit then return c; else if c is a digit call INTEGER else call WORD. TinySQL grammar lexical rules

lexical rules TinySQL grammar SELECT beer FROM Likes, Frequents Characters a-z, 0-9, <, >, =, ., ,, ;, `, ’, (, ), [, ], +, -, *, / Lexical Rules letter ::= a | b | c | d | e | f | g | h | i | j | k | l | m | n | o | p | q | r| s | t | u | v | w | x | y | z digit ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 integer ::= digit+ comp-op ::= < | > | = table-name ::= letter(digit | letter)* attribute-name ::= letter(digit | letter)* column-name ::= [table-name.]attribute-name literal ::= “whatever-except-(`)-and-(’)” Keywords CREATE, TABLE, INT, STR20, DROP, SELECT, DISTINCT, FROM, WHERE, ORDER, BY, DELETE, FROM, INSERT, INTO, VALUES, NULL, OR, AND, NOT, SELECT beer FROM Likes, Frequents WHERE bar = ’Joe’’s Bar’ AND Frequents.drinker = Likes.drinker; SCANNER: read the next char c; if c is not a letter or digit then return c; else if c is a digit call INTEGER else call WORD. WORD: 1. read the next char c; 2. while c is a digit/letter do add c to the output; read the next char c; 3. if c = “.” …… TinySQL grammar lexical rules

What Does DBMS Do? Prepare a collection A of efficient algorithms for operations in relational algebra; For a given database program P: 1. understand the program P; Break the program P into “words” (lexical analysis);