WiscKey是一个基于LSM的KV存储引擎，特点是：针对SSD的顺序和随机读写都高效的特点，Key和Value分开存储以最小化IO放大效应。

Outline Introduction LSM-tree and LevelDB Architecture WiscKey

Introduction key-value database A key-value database, also called a key-value store, is the most flexible type of NoSQL database. In a key-value store, there is no schema and the value of the data is opaque. Values are identified and accessed via a key, and stored values can be numbers, strings, XML, HTML, binaries, images,and more. It is the most flexible NoSQL model because the application has complete control over what is stored in the value. Motivation The new information of the network is constantly produced, but the design and table analysis of the relational database makes the incoming data unable to respond in time, and the scalability is low. As a result, the traditional relational query is scalability slow and the benefit is relatively low.

LSM-tree and LevelDB Architecture Log files A log file (*.log) stores a sequence of recent updates. Each update is appended to the current log file. memtable A copy of the current log file is kept in an in-memory structure,This copy is consulted on every read so that read operations reflect all logged updates. SSTable: Sorted Table A sorted table (*.ldb) stores a sequence of entries sorted by key. Each entry is either a value for the key, or a deletion marker for the key.

LSM-tree and LevelDB Architecture Insertion Compaction: When the size of level L exceeds its limit, we compact it in a background thread: The compaction picks a file from level L and all overlapping files from the next level L+1 The range of keys contained between SSTable files does not overlap at any Level other than Level 0 In the Compaction process, all the SSTable files are merged and sorted by key, then the sorted results are written to the new SSTable file, and if the size of the SSTable file is capped, the new SSTable is written again. And so on, until all the data is written.

WiscKey Introduction WiscKey is a storage engine that improves K-V based on leveddb SSD-conscious designs decouple sorting and garbage collection by separating keys from values Motivation While this trade-off between sequential I/O access and I/O amplification is justified for traditional hard disks,they are not optimal for modern hardware utilizing SSDs.

WiscKey Design Goals Low write amplification reasons large write amplification can consume most of the write bandwidth (over 90% is not uncommon) and decrease the SSD’s lifetime due to limited erase cycles. Low read amplification the throughput of lookups is significantly reduced by issuing multiple reads for each lookup the large amount of data loaded into memory decreases the efficiency of the cache SSD optimized(obviously) Feature-rich API (e.g. range queries,snapshots)(obviously) Realistic key-value sizes though value sizes can vary widely . WiscKey aims to provide high performance for this realistic set of key-value sizes

WiscKey Optimizations Value-Log Write Buffer WiscKey prepares a buffer for ValueLog, writes all the data into the buffer and writes to ValueLog when it is full or has a sync request and read the buffer first when reading. Optimizing the LSM-tree Log WiscKey stores a record <'head', head-vlog-offset> in the LSM, and upon opening a DB can begin to recover data from the head-vlog-offset. Keeping head in LSM also ensures consistency at LevelDB, so overall consistency is still at LevelDB.