1. Design of Flash-Based DBMS: An In-Page Logging Approach Sang-Won Lee and Bongki Moon Presented by RuBao Li, Zinan Li

2. Introduction Flash Memory ◦ Introduced 1987 ◦ Mp3s, PDAs, mobile phones, digital cameras… ◦ Advantages over disk  Smaller size  Lighter weight  Better shock resistance  Lower power consumption  Less noise  Faster read performance ◦ Non-volatile and retains contents when powered off. ◦ Already replaced disk in some computers. ◦ Future mobile and embedded systems expected to carry out more larger, complex, data centric tasks.

3. How flash works Uses Flash Translation Layer (FTL) to make linear flash memory appear like disk to upper layers, then conventional disk-based database algorithm and access methods do not need any modification. Two basic types: ◦ NOR – full memory mapped random access interface, dedicated address and data lines. ◦ NAND – no dedicated address line, IO-Like interface (similar to disk).

4. How flash works Before overwriting a sector, system must erasing the erase unit containing the sector. To avoid the high latency of write and erase requests, changes made to a data page are buffered in memory on the per-page basis and the change logs are written sector by sector to the log area in flash memory.

5. Goals Overcome the limitations of and exploit the advantages of flash memory. Achieve the best attainable performance while minimizing the changes made to the database sever architecture.

6. Design Principles Characteristics of flash memory No in-place update erase operation must be performed before overwriting No mechanical latency uniform random access speed no seek and rotational delay Asymmetric speed of read/write read speed faster than write and erase speed

7. Design Principles Problems with conventional designs 1. Update of a single record will cause an entire page 2. Frequently update short the life of memory 3. wear-leveling ◦ Distributes erase cycles evenly across memory. ◦ No erasures as long as a free sector is available. ◦ Optimizes write to detriment of read performance. Response time of the query in the wall clock time

8. In-Page Logging Approach Basic concepts Co-locate a data page and its log records in the same physical location of flash memory, specifically in the same erase unit.

9. Design Only change Buffer and Storage Manager of DBMS. When updating, the operation is the same as a traditional DBMS for the in-memory copy. ◦ IPL Buffer Manager adds physiological log record on per-page basis to in-memory log sector associated with in memory copy of the data page. ◦ These are written to Flash memory when log sector becomes full or when a dirty data page is evicted from the buffer pool.

10. Design (continued…) Write log records instead of rewrite data. Reduce write operations. Each unit of flash memory is divided into two parts by IPL Storage Manager ◦ Data pages ◦ Log sectors pertaining to those data pages

11. Merging What happens when Flash memory log sectors become full? Merging ◦ Log sectors in erase unit are full ◦ Allocate a free erase unit ◦ Compute new version of data page by applying log records to it ◦ Write new version of data page to new erase unit ◦ Erase and free old erase unit Merge is expensive, but is done less times than write and erase operations under in-place updating or sequential logging.

12. Merging Merge algorithm

13. Disk-Based Server Performance Read performance: The description of the queries:

14. Flash Memory vs. Disk Write performance: Most of flash memory are augmented with a DRAM buffer to avoid as many erase operation as possible. The MSD-P35 NAND flash solid state disk has a 16MBytes DRAM buffer, each one MByte segment which store 8 contiguous erase units.

15. Flash Memory vs. Disk Erase numbers: Q4=64000/(8*16); Q5=64000/8; Q6=64000;

16. TPC-C Benchmark Workload generation tool: Hammerora TPC-C Trace Generation 1. 100M.20M.10u:100 Mbyte database, 20 Mbyte buffer pool, 10 users 2. 1G.20M.100u:1Gbyte database, 20 Mbyte buffer pool, 100 users 3. 1G.40M.100u:1Gbyte database, 40Mbyte buffer pool, 100 users when the database sever runs under each configuration, it generates log records during the course of query processing.

17. TPC-C Benchmark Update pattern of the TPC-C benchmark First examine the lengths of log records: Average length of log records is less than 50bytes, so a sector can store up to 10 log records. This means the log sectors will not be flushed until its associated data page gets updated 10 times on average.

18. TPC-C Benchmark Update pattern of the TPC-C benchmark second examine the log reference locality by counting the number of log records that updated individual data pages:

19. TPC-C Benchmark

20. Third, examine the temporal locality of data page updates by running a sliding window of length 16 through each trace of physical write operations. 1G.20M.100u trace: 16 consecutive physical writes be done for 16 distinct data pages was 99.9%. the update patterns of the TPC-C benchmark are expected to cause a large number of erase operations with flash memory.

21. TPC-C Benchmark Write performance estimation

22. TPC-C Benchmark The IPL simulator returns two counters: 1. The total number of sector writes 2. The total number of erase unit merges The number of merges is affected by the size of a log region.

23. TPC-C Benchmark The write time estimated by the formula: 20ms is calculated by adding reading, writing and erasing time.

24. TPC-C Benchmark Examine the performance of a database server with IPL features:

25. TPC-C Benchmark The write time was estimated by formula: α denotes the probability that a page write will cause the container erase unit to be copied and erased.

26. Recovery Conventional system-wide logging must be adopted to keep track of start/end of transactions (commit or abort). Maintain a list of dirty pages in the buffer pool in memory so that a transaction that commits or aborts (other than system failure) can locate in-memory log sectors containing log records added by the transaction.

27. Commit No-Force Policy – only log tail is forced to stable storage, not all data pages modified by the transaction. ◦ Force policy would lead to random disk accesses for an increase volume of data. Write corresponding in-memory log sectors to Flash memory when a transaction commits. No REDO action needed at system restart since redefined IPL read operation applies log records to data on the fly and the committed log records would be in Flash memory.

28. Abort An aborting transaction (not by system failure) has log records that remain in in- memory log sectors that can be located via the list of dirty pages in memory. Once found these are removed from their respective in-memory log sector, and de-applied to corresponding data pages in the buffer pool.

29. Abort (continued…) What if a Merge operation already happened? Selective Merge ◦ Committed – apply log record and write new data ◦ Aborted – ignore log record ◦ Active – write log record to new log sector If log sector remains full (such as all the transactions are active still), allow an overflow log sector in another erase unit. No explicit UNDO operation – changes will just be merely ignored and not merged when a merge operation is encountered on its respective data page. If upon recovery a transaction was found to be active (no commit or abort), add an abort record for the transaction to the system- wide log.

30. Conclusions Flash memory will replace disks in computers. Normal traditional database servers will suffer seriously deteriorated write performance due to erase-before-write limitation of Flash memory. IPL has shown potential for considerably improving performance for OLTP apps by exploiting advantages of Flash memory. IPL also supports a nice recovery mechanism for transactions.