University of Minnesota FAST: A Framework for Flash-Aware Spatial Trees Mohamed Sarwat, Mohamed Mokbel, Xun Zhou Department of Computer Science and Engineering.

Slides:



Advertisements
Similar presentations
M AINTAINING L ARGE A ND F AST S TREAMING I NDEXES O N F LASH Aditya Akella, UW-Madison First GENI Measurement Workshop Joint work with Ashok Anand, Steven.
Advertisements

Recovery CPSC 356 Database Ellen Walker Hiram College (Includes figures from Database Systems by Connolly & Begg, © Addison Wesley 2002)
CSCI 3140 Module 8 – Database Recovery Theodore Chiasson Dalhousie University.
Spatial Indexing I Point Access Methods. PAMs Point Access Methods Multidimensional Hashing: Grid File Exponential growth of the directory Hierarchical.
Boost Write Performance for DBMS on Solid State Drive Yu LI.
1 Overview of Storage and Indexing Chapter 8 (part 1)
CPSC-608 Database Systems Fall 2011 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes #10.
Chapter 6: Database Evolution Title: AutoAdmin “What-if” Index Analysis Utility Authors: Surajit Chaudhuri, Vivek Narasayya ACM SIGMOD 1998.
Data Indexing Herbert A. Evans. Purposes of Data Indexing What is Data Indexing? Why is it important?
CPSC-608 Database Systems Fall 2011 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes #6.
1 Overview of Storage and Indexing Yanlei Diao UMass Amherst Feb 13, 2007 Slides Courtesy of R. Ramakrishnan and J. Gehrke.
I/O-Efficient Structures for Orthogonal Range Max and Stabbing Max Queries Second Year Project Presentation Ke Yi Advisor: Lars Arge Committee: Pankaj.
Chapter 3: Data Storage and Access Methods
U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science Re-thinking Data Management for Storage-Centric Sensor Networks Deepak Ganesan University.
Chapter 4: Transaction Management
CPSC-608 Database Systems Fall 2011 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes #13.
CPSC-608 Database Systems Fall 2010 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes 1.
CPSC-608 Database Systems Fall 2010 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes #6.
R-Trees: A Dynamic Index Structure for Spatial Data Antonin Guttman.
Indexing. Goals: Store large files Support multiple search keys Support efficient insert, delete, and range queries.
Lecture 11: DMBS Internals
Overview of a Database Management System
Modularizing B+-trees: Three-Level B+-trees Work Fine Shigero Sasaki* and Takuya Araki NEC Corporation * currently with 1st Nexpire Inc.
Case study DATABASE MANAGEMENT SYSTEMS Oracle Database 11g Release 2 (11.2) – MySQL 5.5 –
Origianal Work Of Hyojun Kim and Seongjun Ahn
Index tuning-- B+tree. overview © Dennis Shasha, Philippe Bonnet 2001 B+-Tree Locking Tree Traversal –Update, Read –Insert, Delete phantom problem: need.
Some key-value stores using log-structure Zhichao Liang LevelDB Riak.
Logging in Flash-based Database Systems Lu Zeping
DBMS Transactions and Rollback Recovery Helia / Martti Laiho.
VLDB2012 Hoang Tam Vo #1, Sheng Wang #2, Divyakant Agrawal †3, Gang Chen §4, Beng Chin Ooi #5 #National University of Singapore, †University of California,
MySQL. Dept. of Computing Science, University of Aberdeen2 In this lecture you will learn The main subsystems in MySQL architecture The different storage.
1 Moshe Shadmon ScaleDB Scaling MySQL in the Cloud.
Speaker: 吳晋賢 (Chin-Hsien Wu) Embedded Computing and Applications Lab Department of Electronic Engineering National Taiwan University of Science and Technology,
Author : Chin-Hsien Wu Presenter : kilroy. Outline Introduction Related work Motivation Main idea Evaluation Conclusion Q & A.
Optimized Transaction Time Versioning Inside a Database Engine Intern: Feifei Li, Boston University Mentor: David Lomet, MSR.
By: Sang K. Cha, Sangyong Hwang, Kihong Kim and Kunjoo Kwon
Indexing.
Resolving Journaling of Journal Anomaly in Android I/O: Multi-Version B-tree with Lazy Split Wook-Hee Kim 1, Beomseok Nam 1, Dongil Park 2, Youjip Won.
Dilip N Simha, Maohua Lu, Tzi-cher chiueh Park Chanhyun ASPLOS’12 March 3-7, 2012 London, England, UK.
Design of Flash-Based DBMS: An In-Page Logging Approach Sang-Won Lee and Bongki Moon Presented by Chris Homan.
Embedded System Lab. Jung Young Jin The Design and Implementation of a Log-Structured File System D. Ma, J. Feng, and G. Li. LazyFTL:
IN-MEMORY OLTP By Manohar Punna SQL Server Geeks – Regional Mentor, Hyderabad Blogger, Speaker.
1 Tree Indexing (1) Linear index is poor for insertion/deletion. Tree index can efficiently support all desired operations: –Insert/delete –Multiple search.
University of Minnesota FAST: A Framework for Flash-Aware Spatial Trees Mohamed Sarwat, Mohamed Mokbel, Xun Zhou Department of Computer Science and Engineering.
Design of Flash-Based DBMS: An In-Page Logging Approach Sang-Won Lee and Bongki Moon Presented by RuBao Li, Zinan Li.
DMBS Internals I. What Should a DBMS Do? Store large amounts of data Process queries efficiently Allow multiple users to access the database concurrently.
PROBLEM STATEMENT A solid-state drive (SSD) is a non-volatile storage device that uses flash memory rather than a magnetic disk to store data. SSDs provide.
RecStore An Extensible and Adaptive Framework for Online Recommender Queries inside the Database Engine.
Darko Makreshanski Department of Computer Science ETH Zurich
CS 440 Database Management Systems Lecture 6: Data storage & access methods 1.
ICOM 5016 – Introduction to Database Systems Lecture 13- File Structures Dr. Bienvenido Vélez Electrical and Computer Engineering Department Slides by.
CS 540 Database Management Systems
ICOM 6005 – Database Management Systems Design Dr. Manuel Rodríguez-Martínez Electrical and Computer Engineering Department Lecture 7 – Buffer Management.
DMBS Internals I February 24 th, What Should a DBMS Do? Store large amounts of data Process queries efficiently Allow multiple users to access the.
DMBS Internals I. What Should a DBMS Do? Store large amounts of data Process queries efficiently Allow multiple users to access the database concurrently.
1 CSCE 520 Test 2 Info Indexing Modified from slides of Hector Garcia-Molina and Jeff Ullman.
Transactional Flash V. Prabhakaran, T. L. Rodeheffer, L. Zhou (MSR, Silicon Valley), OSDI 2008 Shimin Chen Big Data Reading Group.
Sai – 9 th March’08.  NAND flash becoming more and more ubiquitous with promising features such as:  very low power consumption  compactness  affordability,
CS 540 Database Management Systems
Failure-Atomic Slotted Paging for Persistent Memory
CS522 Advanced database Systems
Indexing Goals: Store large files Support multiple search keys
Chapter Overview Understanding the Database Architecture
HashKV: Enabling Efficient Updates in KV Storage via Hashing
Chapters 17 & 18 6e, 13 & 14 5e: Design/Storage/Index
SQL 2014 In-Memory OLTP What, Why, and How
KISS-Tree: Smart Latch-Free In-Memory Indexing on Modern Architectures
RUM Conjecture of Database Access Method
SQL Statement Logging for Making SQLite Truly Lite
Dong Hyun Kang, Changwoo Min, Young Ik Eom
Presentation transcript:

University of Minnesota FAST: A Framework for Flash-Aware Spatial Trees Mohamed Sarwat, Mohamed Mokbel, Xun Zhou Department of Computer Science and Engineering University of Minnesota Suman Nath Microsoft Research

2 Flash Memory: Already exists Mobile Devices Enterprise Servers

3 Flash Memory Storage: Pros Flash MemoryHard Disk Electronic DeviceMechanical Device Fast Read PerformanceSlow Read Performance High Shock ResistanceLow Shock Resistance Low Power ConsumptionHigh Power Consumption Small Size and Light WeightRelatively Large Size and Heavy Weight Flash Memory is Topnotch !

4 Flash Memory Storage: Cons  Erase-Before-Update Property  Erase latency 1500 microseconds  Read latency 25 microseconds  Write latency 200 microseconds  Limited Number of Erases per Block  100,000 to 1000,000

5 DBMS Tree Indexes & Storage All Database indexing algorithms were built with the implicit assumption that the underlying secondary storage is the hard disk. Making Tree Index Structures Flash-Aware !

6 Existing Flash-Aware Trees ■ Chin-Hsien Wu, Li-Pin Chang, Tei-Wei Kuo. An Efficient R-Tree Implementation over Flash-Memory Storage Systems. In GIS 2003  Designed only for R-tree.  No support for recovery ■ Chin-Hsien Wu, Li-Pin Chang, and Tei-Wei Kuo. An Efficient B-Tree Layer for Flash-Memory Storage Systems. In TECS 2007  Designed only for B-tree.  No support for recovery

7 Existing Flash-Aware Trees ■ Devesh Agrawal, Deepak Ganesan, Ramesh Sitaraman, Yanlei Diao, and Shashi Singh. Lazy- Adaptive Tree: An Optimized Index Structure for Flash Devices. In VLDB 2009  Need to build a new kind of tree indexing. Not very practical..!! ■ Yinan Li, Bingsheng He, Robin Jun Yang, Qiong Luo, Ke Yi. Tree Indexing on Flash Disks. In ICDE 2009 (short), VLDB 2010(full)  Need to build a new kind of tree indexing. Not very practical..!!

8 We need a Framework

9 FAST

10 FAST: A Framework for Flash-Aware Search Trees Recent Version of the tree Flash Storage Flash Memory Index In-memory Buffer Update Module Search Module Crash Recovery Module Flushing Module Search Query Answer Update Transaction Log File Restart after Crash Full Memory Read In-memory Update Update Log Entries Flushing Policy Read Selected Memory Pages Write Flush Log Entry Read Index & Log Entries Write

11 FAST: A Framework for Flash-Aware Search Trees Recent Version of the tree Flash Storage Flash Memory Index In-memory Buffer Update Module Search Module Crash Recovery Module Flushing Module Search Query Answer Update Transaction Log File Restart after Crash Full Memory Read In-memory Update Update Log Entries Flushing Policy Read Selected Memory Pages Write Flush Log Entry Read Index & Log Entries Write

12 Update Operation (Insertion/Deletion) Main MemoryFlash Memory Operation Log ……. Tree Modifications Table Tree Index Structure Insert (O1, MBR ….) || Nodes 1,2Step 1 Step 2 Node Node 1 Node 2 MBR1 - value Obj1 - Inserted Example: Insert (Obj1) in R-tree Step 3

13 FAST: A Framework for Flash-Aware Search Trees Recent Version of the tree Flash Storage Flash Memory Index In-memory Buffer Update Module Search Module Crash Recovery Module Flushing Module Search Query Answer Update Transaction Log File Restart after Crash Full Memory Read In-memory Update Update Log Entries Flushing Policy Read Selected Memory Pages Write Flush Log Entry Read Index & Log Entries Write

14 Search Operation (Find/Range) Main Memory Flash Memory Operation Log ……. Tree Modifications Table Tree Index Structure Insert (O1, MBR ….) || Nodes 1,2 Step1 Step 2 Node Node 1 Node 2 MBR1 - value Obj1 - Inserted Example: Range query in R-tree Step 4 Query Result Step 5 Step 3

15 FAST: A Framework for Flash-Aware Search Trees Recent Version of the tree Flash Storage Flash Memory Index In-memory Buffer Update Module Search Module Crash Recovery Module Flushing Module Search Query Answer Update Transaction Log File Restart after Crash Full Memory Read In-memory Update Update Log Entries Flushing Policy Read Selected Memory Pages Write Flush Log Entry Read Index & Log Entries Write

16 Flushing Main MemoryFlash Memory FAST Log File Tree Index Structure Insert (O1, MBR) || Nodes 1,2 MBR1 - value Node12345 Dirty0 01 Flush 2,3,4,5 75 % Nodes 2,3,4,5 flushed Same Block Node Delete (O2, MBR) || Nodes 12 Insert (O4, MBR) || Nodes 1,4,14 Insert (O5, MBR) || Nodes 4,17 Insert (O4, MBR) || Nodes 1,5,18 Step 1 Step 2 Step 4 Step 3 Step 5

17 FAST: A Framework for Flash-Aware Search Trees Recent Version of the tree Flash Storage Flash Memory Index In-memory Buffer Update Module Search Module Crash Recovery Module Flushing Module Search Query Answer Update Transaction Log File Restart after Crash Full Memory Read In-memory Update Update Log Entries Flushing Policy Read Selected Memory Pages Write Flush Log Entry Read Index & Log Entries Write

18 Crash Recovery Main MemoryFlash Memory FAST Log File..... Tree Index Structure Insert (O1, MBR) || Nodes 1,2 Nodes 1,2,3,4,5, 14,17 flushed Delete (O2, MBR) || Nodes 12 Insert (O4, MBR) || Nodes 1,4,14 Insert (O5, MBR) || Nodes 4,17 Insert (O4, MBR) || Nodes 1,5,18 Log Maintenance Compact Log Insert (O4, MBR) || Nodes 18 Crash Recovery Restart after crash MBR1 - value Node

19 Experimental Evaluation (1/4) ■ We implemented FAST in PostGreSQL ■ We instantiate B-tree and R-tree instances of FAST, termed FAST-Btree and FAST-Rtree ■ We use two synthetic workloads:  Lookup intensive workload (WL): 80% search and 20% update  Update intensive workload, (WU): 20% search and 80% update. ■ number of workload operations to 10 million ■ main memory size to 256 KB ■ Tree index size to 512 MB, and ■ log file size to10 MB (~ 2% of Index size)

20 Experimental Evaluation (2/4) Memory SizeLog File Size 348 lines of code 3070 lines of code 308 lines of code 9230 lines of code 8150 lines of code

21 Experimental Evaluation (3/4) Memory SizeLog File Size

22 Experimental Evaluation (4/4) Log CompactionRecovery

23 Conclusion ■ FAST is a generic framework for flash-aware index structures. ■ FAST guarantees the durability of update transactions applied to the underlying index structure. ■ FAST has good performance compared to its counterparts (LA-tree, FD-tree, and RFTL)

24 Questions