University of Minnesota FAST: A Framework for Flash-Aware Spatial Trees Mohamed Sarwat, Mohamed Mokbel, Xun Zhou Department of Computer Science and Engineering University of Minnesota Suman Nath Microsoft Research

2 Flash Memory: Already exists Mobile Devices Enterprise Servers

3 Flash Memory Storage: Pros Flash MemoryHard Disk Electronic DeviceMechanical Device Fast Read PerformanceSlow Read Performance High Shock ResistanceLow Shock Resistance Low Power ConsumptionHigh Power Consumption Small Size and Light WeightRelatively Large Size and Heavy Weight Flash Memory is Topnotch !

4 Flash Memory Storage: Cons  Erase-Before-Update Property  Erase latency 1500 microseconds  Read latency 25 microseconds  Write latency 200 microseconds  Limited Number of Erases per Block  100,000 to 1000,000

5 DBMS Tree Indexes & Storage All Database indexing algorithms were built with the implicit assumption that the underlying secondary storage is the hard disk. Making Tree Index Structures Flash-Aware !

6 Existing Flash-Aware Trees ■ Chin-Hsien Wu, Li-Pin Chang, Tei-Wei Kuo. An Efficient R-Tree Implementation over Flash-Memory Storage Systems. In GIS 2003  Designed only for R-tree.  No support for recovery ■ Chin-Hsien Wu, Li-Pin Chang, and Tei-Wei Kuo. An Efficient B-Tree Layer for Flash-Memory Storage Systems. In TECS 2007  Designed only for B-tree.  No support for recovery

7 Existing Flash-Aware Trees ■ Devesh Agrawal, Deepak Ganesan, Ramesh Sitaraman, Yanlei Diao, and Shashi Singh. Lazy- Adaptive Tree: An Optimized Index Structure for Flash Devices. In VLDB 2009  Need to build a new kind of tree indexing. Not very practical..!! ■ Yinan Li, Bingsheng He, Robin Jun Yang, Qiong Luo, Ke Yi. Tree Indexing on Flash Disks. In ICDE 2009 (short), VLDB 2010(full)  Need to build a new kind of tree indexing. Not very practical..!!

8 We need a Framework

9 FAST

10 FAST: A Framework for Flash-Aware Search Trees Recent Version of the tree Flash Storage Flash Memory Index In-memory Buffer Update Module Search Module Crash Recovery Module Flushing Module Search Query Answer Update Transaction Log File Restart after Crash Full Memory Read In-memory Update Update Log Entries Flushing Policy Read Selected Memory Pages Write Flush Log Entry Read Index & Log Entries Write

11 FAST: A Framework for Flash-Aware Search Trees Recent Version of the tree Flash Storage Flash Memory Index In-memory Buffer Update Module Search Module Crash Recovery Module Flushing Module Search Query Answer Update Transaction Log File Restart after Crash Full Memory Read In-memory Update Update Log Entries Flushing Policy Read Selected Memory Pages Write Flush Log Entry Read Index & Log Entries Write

12 Update Operation (Insertion/Deletion) Main MemoryFlash Memory Operation Log ……. Tree Modifications Table Tree Index Structure Insert (O1, MBR ….) || Nodes 1,2Step 1 Step 2 Node Node 1 Node 2 MBR1 - value Obj1 - Inserted Example: Insert (Obj1) in R-tree Step 3

13 FAST: A Framework for Flash-Aware Search Trees Recent Version of the tree Flash Storage Flash Memory Index In-memory Buffer Update Module Search Module Crash Recovery Module Flushing Module Search Query Answer Update Transaction Log File Restart after Crash Full Memory Read In-memory Update Update Log Entries Flushing Policy Read Selected Memory Pages Write Flush Log Entry Read Index & Log Entries Write

14 Search Operation (Find/Range) Main Memory Flash Memory Operation Log ……. Tree Modifications Table Tree Index Structure Insert (O1, MBR ….) || Nodes 1,2 Step1 Step 2 Node Node 1 Node 2 MBR1 - value Obj1 - Inserted Example: Range query in R-tree Step 4 Query Result Step 5 Step 3

15 FAST: A Framework for Flash-Aware Search Trees Recent Version of the tree Flash Storage Flash Memory Index In-memory Buffer Update Module Search Module Crash Recovery Module Flushing Module Search Query Answer Update Transaction Log File Restart after Crash Full Memory Read In-memory Update Update Log Entries Flushing Policy Read Selected Memory Pages Write Flush Log Entry Read Index & Log Entries Write

16 Flushing Main MemoryFlash Memory FAST Log File Tree Index Structure Insert (O1, MBR) || Nodes 1,2 MBR1 - value Node12345 Dirty0 01 Flush 2,3,4,5 75 % Nodes 2,3,4,5 flushed Same Block Node Delete (O2, MBR) || Nodes 12 Insert (O4, MBR) || Nodes 1,4,14 Insert (O5, MBR) || Nodes 4,17 Insert (O4, MBR) || Nodes 1,5,18 Step 1 Step 2 Step 4 Step 3 Step 5

17 FAST: A Framework for Flash-Aware Search Trees Recent Version of the tree Flash Storage Flash Memory Index In-memory Buffer Update Module Search Module Crash Recovery Module Flushing Module Search Query Answer Update Transaction Log File Restart after Crash Full Memory Read In-memory Update Update Log Entries Flushing Policy Read Selected Memory Pages Write Flush Log Entry Read Index & Log Entries Write

18 Crash Recovery Main MemoryFlash Memory FAST Log File..... Tree Index Structure Insert (O1, MBR) || Nodes 1,2 Nodes 1,2,3,4,5, 14,17 flushed Delete (O2, MBR) || Nodes 12 Insert (O4, MBR) || Nodes 1,4,14 Insert (O5, MBR) || Nodes 4,17 Insert (O4, MBR) || Nodes 1,5,18 Log Maintenance Compact Log Insert (O4, MBR) || Nodes 18 Crash Recovery Restart after crash MBR1 - value Node

19 Experimental Evaluation (1/4) ■ We implemented FAST in PostGreSQL ■ We instantiate B-tree and R-tree instances of FAST, termed FAST-Btree and FAST-Rtree ■ We use two synthetic workloads:  Lookup intensive workload (WL): 80% search and 20% update  Update intensive workload, (WU): 20% search and 80% update. ■ number of workload operations to 10 million ■ main memory size to 256 KB ■ Tree index size to 512 MB, and ■ log file size to10 MB (~ 2% of Index size)

20 Experimental Evaluation (2/4) Memory SizeLog File Size 348 lines of code 3070 lines of code 308 lines of code 9230 lines of code 8150 lines of code

21 Experimental Evaluation (3/4) Memory SizeLog File Size

22 Experimental Evaluation (4/4) Log CompactionRecovery

23 Conclusion ■ FAST is a generic framework for flash-aware index structures. ■ FAST guarantees the durability of update transactions applied to the underlying index structure. ■ FAST has good performance compared to its counterparts (LA-tree, FD-tree, and RFTL)

24 Questions