1. Phase Change Memory Aware Data Management and Application Jiangtao Wang

2. Outline Introduction Integrating PCM into the Memory Hierarchy −PCM for main memory −PCM for auxiliary memory Conclusion

3. Phase change memory An emerging memory technology Memory(DRAM) −Read/write speeds and Byte-addressable −Lower Idle power Storage(SSD & HDD) −Non-volatile −high capacity (high density)

4. DRAMPCMNAND Flash Page size64B 2KB Page read latency20-50ns~50ns~25us Page write latency20-50ns~1us~500us Endurance∞10 6 -10 8 10 4 -10 5 Idle power~100mW/GB~1mW/GB1-10mW/GB Density1x2-4x4x Phase change memory Cons: −Asymmetry read/write latency −Limited write endurance

5. Phase change memory Read operation 10ns 100ns1us10us 100us 1ms10ms Write operation DRAM HDD PCM FLASH DRAM PCM FLASH HDD

6. Outline Introduction Integrating PCM into the Memory Hierarchy −PCM for main memory −PCM for auxiliary memory Conclusion

7. Integrating PCM into the Memory Hierarchy PCM for main memory – Replacing DRAM with PCM to achieve larger main memory capacity PCM for auxiliary memory – PCM as a write buffer for HDD/SSD DISK Buffering dirty page to minimize the disk write I/Os – PCM as secondary storage Storing log records

8. PCM for main memory Phase Change Memory Memory Controller HDD/SSD Disk CPU L1/L2 Cache (a)PCM-only memory Phase Change Memory Memory Controller HDD/SSD Disk CPU L1/L2 Cache DRAM Cache (b)DRAM as a cache memory Phase Change Memory Memory Controller HDD/SSD Disk CPU L1/L2 Cache DRAM Write buffer (c)DRAM as a write buffer [ISCA’09] [ICCD’11] [DAC’09] [CIDR’11]

9. PCM for main memory Challenges with PCM Major disadvantage – Writes Compared to read operation,PCM writes incur higher energy consumption 、 higher latency and limited endurance Read latency20~50nsWrite latency~1us Read energy1 J/GBWrite energy6 J/GB Endurance10 6 ~10 8 Reducing PCM writes is an important goal of data management on PCM !

10. Optimization: data comparison write Goal: write only modified bits rather than entire cache line Approach: read-compare-write 010110110110111001011011011011101000101101011011011011100101100001101011 CPU cache PCM PCM for main memory Optimization on PCM write [ISCAS’07] [ISCA’09] [MICRO’09] read

11. PCM for main memory PCM-friendly algorithms Motivation Choosing PCM-friendly database algorithms and data structures to reduce the number of writes Rethinking Database Algorithms for Phase Change Memory(CIDR2011)

12. PCM for main memory PCM-friendly DB algorithms Prior design goals for DRAM −Low computation complexity −Good CPU cache performance −Power efficiency (more recently) New goal for PCM −minimizing PCM writes −Low wear, energy and latency −Finer-grained access granularity:bits,words,cache line Two core database techniques −B + -Tree Index −Hash Joins

13. PCM-friendly DB algorithms B + -Tree Index B + -Tree – Records at leaf nodes – High fan out – Suitable for file systems For PCM – Insertion/deletion incur a lot of write operations – K keys and K pointers in a node: 2(K/2)+1=K+1 num keys 524789 pointers num keys 6234789 pointers Insert 3 incurs 11 writes

14. PCM-friendly B + -Tree – Unsorted: all the non-leaf and leaf nodes unsorted – Unsorted leaf: sorted non-leaf and unsorted leaf – Unsorted leaf with bitmap :sorted non-leaf and unsorted leaf with bitmaps PCM-friendly DB algorithms B + -Tree Index num keys 582947 pointers num keys 1011 1010 82947 pointers Unsorted leaf nodeUnsorted leaf node with bitmap

15. Unsorted leaf – Insert/delete incurs 3 writes PCM-friendly DB algorithms B + -Tree Index num keys 582947 pointers num keys 48794 pointers Delete 2 Unsorted leaf with bitmap – Insert incurs 3 writes; delete incurs 1 write num keys 1011 1010 82947 pointers Delete 2 num keys 1001 1010 82947 pointers

16. Experimental evaluation B + -Tree Index Simulation Platform – Cycle-accurate X86-64 simulator: PTLSim – Extended the simulator with PCM support – Modeled data comparison write – CPU cache(8MB), B + -Tree (50 million entrys,75% full,1GB) Three workloads: – Inserting 500K random keys – Deleting 500K random keys – Searching 500K random keys

17. Node size 8 cache lines; 50 million entries, 75% full; Experimental evaluation B + -Tree Index Total wear EnergyExecution time Unsorted schemes achieve the best performance For insert intensive workload: unsorted-leaf For insert & delete intensive workload : unsorted-leaf with bitmap

18. Two representative algorithms – Simple Hash Join – Cache Partitioning PCM-friendly DB algorithms Hash Joins ######## R Build Phase S Probe Relation Hash Table Problem – too many cache misses – Build and probe hash table(exceeds CPU cache size) – Small record size Simple Hash Join

19. Cache Partitioning PCM-friendly DB algorithms Hash Joins Partition Phase S S1 S2 S4 S3 Join Phase R R1 R2 R4 R3 Problems ： Too many writes!

20. Virtual Partitioning （ PCM-friendly DB algorithms ） PCM-friendly DB algorithms Hash Joins Partition phase R Virtual partitioning S Virtual Partitioning R’1 S’1 R’2 R’3 R’4 S’1 Store record ID

21. PCM-friendly DB algorithms Hash Joins Join phase R’1 ######## Hash table BuildProbe S’1 R S Good CPU cache performance Reducing writes Virtual Partitioning (PCM-friendly DB algorithms)

22. Relations R and S are in main memory(PCM) R(50MB) joins S(100MB) (2 matches per R record) Varying record size from 20B to 100B Experimental evaluation Hash Join Total wearPCM energy Execution time

23. PCM for auxiliary memory PCM write buffer Memory Controller HDD/SSD Disk CPU L1/L2 Cache DRAM PCM as a write buffer for HDD/SSD DISK [DAC’09] [CIKM’11] [TCDE’10] [VLDB’11] PCM SSD/HDD CPU L1/L2 Cache DRAM PCM as secondary storage

24. PCM for auxiliary memory PCM as a write buffer for HDD/SSD DISK PCMLogging: Reducing Transaction Logging Overhead with PCM(CIKM2011) PCM as secondary storage – Accelerating In-Page Logging with Non-Volatile Memory(TCDE2010) – IPL-P: In-Page Logging with PCRAM (VLDB2011 demo)

25. Motivation Buffering dirty page and transaction logging to minimize disk I/Os PCM for auxiliary memory PCM as a write buffer for HDD/SSD DISK PCMLogging: Reducing Transaction Logging Overhead with PCM(CIKM2011)

26. PCMBasic Two schemes – PCMBasic – PCMLogging PCMBasic DRAM Buffer pool Log pool Dirty pagesWrite log PCM DISK Cons: −Data redundancy −Space management on PCM PCM for auxiliary memory

27. PCMLogging – Eliminate explicit logs (REDO and UNDO log) – Integrate implicit logs into buffered updated(shadow pages) PCMLogging DRAM PCM DISK MetaData P1P1 P1P1 P1P1 P2P2 P2P2 … P

28. Overview – DRAM Mapping Table(MT):map logial page to physical page – PCM Page format FreePageBitmap ActiveTxList Page ContentMetaData XXXXXXXXXXXIDPID PCMLogging

29. Overview

30. PCMLogging Operation Two additional data structures in the main memory to support undo memory Transaction Table(TT) Record all in-progress transaction and their corresponding dirty pages in DRAM and PCM Dirty Page Table(DPT) Keep track of the previous version for each PCM “overwritten” by an in-progress transaction PCMLogging

31. Flushing Dirty pages to PCM – Add XID to ActiveTxList before writing dirty page to PCM – If page P exists in the PCM, do not overwrite and create an out-of- place P’ PCMLogging T3 update P5

32. Commit – flush all its dirty pages – Modify metadata: PCMLogging Abort – discard its dirty pages and restore previous data – Modify metadata:

33. Tuple-based Buffering – In the PCM the buffer slots be managed in the unit of tuples, To manage the free space, employ a slotted directory instead of a bitmap – In the DRAM Mapping Table, we still keep track of dirty pages, but maintain the mappings for the buffered tuples in each dirty page – Merge tuples with the corresponding page of the disk read/write request move committed tuples from PCM to the external disk PCMLogging

34. Simulator based on DiskSim TPC-C benchmark DRAM 64MB Tuple-based Experimental evaluation (PL=PCMLogging)

35. PCM as secondary storage – Accelerating In-Page Logging with Non-Volatile Memory(TCDE2010) – IPL-P: In-Page Logging with PCRAM (VLDB2011 demo) PCM for auxiliary memory Motivation IPL scheme with PCRAM can improve the performance of flash memory database systems by storing frequent log records in PCRAM Design of Flash-Based DBMS: An In-Page Logging Approach(SIGMOD2007)

36. In-Page Logging Introduction – Updating a single record may result in invalidating the current page – Sequential logging approaches incur expensive merge operation – Co-locate a data page and its log records in the same physical block Design of Flash-Based DBMS: An In-Page Logging Approach(SIGMOD2007)

37. In-Page Logging log … … Database Buffer Flash Memory Physical block(128K) 15 data pages Log region(8K) 16 sectors(512B) in-memory data page (8K) update in-memory log sector (512B) log

38. Database Buffer Flash Memory in-memory data page (8K) update in-memory log sector (512B) log … … + … … merge In-Page Logging

39. Cons – Units of write log is a sector(512B) – Only SLC-type NAND flash supports partial programming – The amount of log records for a page is usually small In-Page Logging

40. Pros – log records can be flushed in a finer granularity – the low latency of flushing log records – PCRAM is faster than flash memory for small reads – SLC or MLC flash memory can be used for IPL policy. In-Page Logging

41. Experimental evaluation A trace-driven simulation Implement an IPL module to the B + -tree based Berkeley DB Million key-value records insert/search Log sector in memory(128B/512B) Accelerating In-Page Logging with Non-Volatile Memory(TCDE2010)

42. Experimental evaluation Hardware platform – PCRAM(512M,the granularity:128B) – Intel X25-M SSD (USB interface) Workload – Million key-value records insert/search/update – B + -tree based Berkeley DB – Page size :8KB IPL-P: In-Page Logging with PCRAM (VLDB2011 demo)

43. Outline Introduction Integrating PCM into the Memory Hierarchy −PCM for main memory −PCM for auxiliary memory Conclusion

44. PCM is expected to play an important role in the memory hierarchy It is important to consider read/write asymmetry of PCM when design PCM-friendly algorithms Integrating PCM into Hybrid memory might be more practical If we use PCM as main memory,we had to revise some system application(e.g. Main Memory Database Systems )to address PCM-specific challenges.

45. Thank You!