H-store: A high-performance, distributed main memory transaction processing system Robert Kallman, Hideaki Kimura, Jonathan Natkins, Andrew Pavlo, Alex Rasin, Stanley B. Zdonik, Evan P. C. Jones, Samuel Madden, Michael Stonebraker, Yang Zhang, John Hugg, Daniel J. Abadi

Paper highlights An experimental main-memory, parallel DBMS Optimized for on-line transaction processing (OLTP) applications Highly distributed, row-store-based relational database Runs on a cluster on shared-nothing, main memory executor nodes.

Background

Relational DBMS Data are stored in tables Each row corresponds to a record No pointers or other links Matching keys

Example PSID is a key Links rows among tables Must be unique Barbara 13339 CS CE Student PSID Major Alan 08887 PSID is a key Links rows among tables Must be unique 08887 89 13339 87 90 PSID Mid Final

Atomic Transactions: You buy a car

What are atomic transactions? A mechanism used in databases and other financial systems. System guarantees that an atomic transaction will either be executed properly or abort without leaving any trace All or nothing semantics Atomic transactions verify the four ACID properties

The ACID properties Atomicity: All or nothing property Consistency: A transaction either brings the data in a new consistent state of data or returns them to their previous state Isolation: A transaction in process and not yet committed has no effect on any other transaction. Durability: Committed data are stored by the system in some kind of crash-proof storage

Importance of atomic transactions Atomicity and consistency properties guarantee that either the transaction is correct or will leave no traces No partial updates No incorrect updates Isolation property allows concurrent execution of transactions Much faster than serial execution Durability property ensures transactions will not be lost

Back to the paper

Motivation Legacy OLTP databases Too many of their architectural components are old Inherited from original System R Mid-seventies! Take advantage of recent trends Multi-core architectures Cheap abundant main memory Dominant use of stored procedures

The focus Reject “one size fits all” approach On-line transaction processing (OLTP) systems have specific proprieties Repetitive short-lived transactions Stored procedures Sole focus of this work

Rely on replication to minimize the risk of data loss Main issue Poor I/O performance of RDBMS Their solution Scale system “horizontally” Partition responsibilities among multiple shared nothing machines Store entire DB in the memory of a large cluster of server machines Rely on replication to minimize the risk of data loss

H-Store Next generation OLTP system Operates on a distributed cluster of shared nothing machines Coordinates the work of multiple single-threaded engines All data are always kept in main memory

System Overview H-Store Cluster containing two or more computational nodes Nodes Single physical component that holds multiple sites Sites Normally run on a dedicated core Single-threaded Do not share any data structure or memory with any collocated site

H-Store system architecture

System deployment Cluster deployment framework takes at inputs A set of stored procedures A database schema A sample workload (used to optimize data layout) A set of available sites in the cluster Two-phase optimization First optimize stored procedures as if the database was not distributed Then come with distributed query plans

Run-time model All sites in the cluster are trusted Any site is able to execute any OLTP application request Execution plan is Annotated with the locations of the target sites Passed to a transaction manager No shared data structures Everything is single-threaded

Database properties Physical layout of DB specifically optimized to execute precompiled transactions Not ad hoc queries Can still be executed but could be very slow

Transaction classes Two important special cases Single-Site Transactions Can be entirely executed on a single site Easy to send the transaction to one of the target sites One-Shot Transactions Each of is individual queries executes on a single site Output of these queries is not reused as inputs for other queries Easy to execute in parallel

Physical layout Replicate frequently-accessed or read-only tables on each site Horizontal partition of tables Partitions can be accessed in parallel Collocate them with related data Protect data against node failures Important for in-memory DBs k-safety Number k of node failures DB must tolerate

DB layout loader Table Replication Replicate all read-only tables on all sites Data Partitioning Divide horizontally each table into four disjoint partitions Each partition is stored on two different sites Accent is on parallelism K-Safety k = 2