6340 DBMS Components

DBMS OS, application, middleware Components: storage, query optimizer, recovery manager, transaction processor, security

DBMS Storage Linear storage/order storage hierarchy, rand/seq access, disk, block bfr, seek algorithms, buffer, ord (bins) /non-ord files (pile), sorting (int ext) column store, RAID Records: fix/var, unspanned/spanned

DBMS indexing PI, SI Single-level, Multilevel Indexing: hashing (reg dynamic, overflow), clustered, B-tree, bitmap, R-tree

Query Optimizer Query plan Optimal plan Query trees Order of operators

Query Optimization Heuristic: transformation rules Cost-based

Physical operators Primitive: block seek/read/write Table scan; table search (linear, binary) Index search; index scan Sort Hashing

Cost-based optimization Cost models: I/Os, memory access, CPU, channel transfer; temporary tables Intermediate table sizes; uniform pdf, skewed pdf

Estimates Domain cardinality: |pi_A(T)| r(T): |T| R(T): rec size Bfr Statistics pdfs

One pass vs Two pass One pass: load smaller table in memory Two pass: block sort in memory, sort on disk, hashing

Join reordering |Ti| R=Ti*Tj first, then R*Tk Incorrectly formed Theta-joins; X products Left, right outer joins are not commutative

Join algorithms Nested-loop: memory, disk Single-loop=index-based Merge-sort: cheap sort or if sorted; block pairs Hashing: 2 phases, memory, M partitioning, uniform/skewed partitions

Relational operators implementation Combining joins with operators Project: dup elimination Set ops: sort-merge, hashing Aggregations: –entire table –Partition: sorting, hasing Outer joins: relational operators or extend join algorithms

Pushing relational ops Push select: as early as possible, depending on indexing; nested selects= and’; through union/interesection Push project: with required attributes; not convenient when there are indexes Push aggregation through join: depends on query semantics

Concurrency control Interleaved vs parallel processing Ops: r(X),w(X) Lost update; dirty read Atomic statement Binary, shared locks Well-formed transactions

Transaction processor Serializability: serial S, conflict, precedence graph P SQL BT implicit; ET explicit Locking vs timestamping Deadlock, starvation Granularity: row, column, block, table

Transaction States: active, partially committed, committed, failed, terminated Final operation: commit, rollback ACID properties Isolation levels: read uncomm, read comm, repeatable read, serializable Schedules: –serial, serializable; total ordering vs correctness –result, conflict (<), conflict serializable equivalent (< for some S’)

2PL Shared locks; Upgradeable locks 2 phases: no overlap Conservative (lock all), basic, strict (unlock all) Deadlock: wait-die, wound-wait

Timestamping Two TS: read_TS(X), write_TS(X) Starvation possible Basic: restart when *_TS(X)>TS(T); else execute T and update *_TS(X) Strict: < & delay T until T’ ends Thomas: T proceeds without writing when TS(T)<write_TS(X)

Further concurrency control Multi-version Optimistic

Recovery manager Reasons: –Hardware failures: disk, memory, power –Software: unhandled errors, exceptions, transaction abort –Physical events Checkpoint; timeline Cascading rollbacks Redo/undo; idempotent Log: WAL

Recovery Manager Schedule Classification Recoverable: T commits only if all T’ commit first and T reads items written by T’ Cascadeless: read only from committed transactions