NVMeDirect: A User-Space I/O Framework for Application-specific Optimization on NVMe SSD 606410087 Ping-Sung Yeh, 606410145 Te-Hao Hsu Conclusions Results Introduction Impact of the Polling Period Differentiated I/O Service The new emerging technologies are making a remarkable progress in the performance of storage devices. NAND flash-based Solid State Drives (SSDs) are being widely adopted on behalf of hard disk drives (HDDs). The nextgeneration non-volatile memory such as 3D XPoint promises the next step for the storage devices. In accordance with the improvement in the storage performance, the new NVMe (Non-Volatile Memory Express) interface has been standardized to support high performance storage based on the PCI Express (PCIe) interconnect. NVMeDirect supports prioritized I/O without H/W features Prioritized I/O without a weighted round-robin scheduler Using flexible binding between Handles and Queues Sharing a single Queue with multiple Handles Evaluation Polling is not efficient on bandwidth sensitive workload due to th significant invrease int the CPU load Significant performance degradation occurs in a certain polling period Implementation on the Linux kernel 4.3.3 Experimental setop Ubuntu 14.04 LTS 3.3GHz Intel Core i7 CPU(6 cores) & 64GB of DRAM Intel 750 Series 400GB NVMe SSD Compariosn with Kernel I/O SPDK NVMeDirect Differentiated I/O Service One prioritized thread with a dedicated queue,Three threads with a shared queue Each thread performs 4KB random write Baseline Performance Control Polling Period dynamically based on I/O size or hints from applications Figure depicts the IOPS of random reads (Figure a) and random writes (Figure b) on NVMeDirect, SPDK, and Kernel I/O varying the queue depth with a single thread. When the queue depth is sufficiently large, the performance of random reads and writes meets or exceeds the performance specification of the device on both NVMeDirect, SPDK, and Kernel I/O. Design Latency Sensitive Application We develop a user-space I/O framework called NVMeDirect to fully utilize the performance of NVMe SSDs while meeting the diverse requirements from user applications. The figure illustrates the overall architecture of our NVMeDirect framework. Using workload-A in YCSB on Redis Update-heavy workload with Zipf distribution Asynchronous random I/O performance using FIO NVMeDirect First full framework for I/O in the user-space based on stock NVMe devices Can be easily applied to many applications Useful for emerging storage devices, e.g. 3D Xpoint ,etc Future work User-level file systems Porting diverse date-intensive applications over NVMeDirect Protecting the system from illegal access Figure a Figure b Acknowledgments We would like to thank the anonymous reviewers and our shepherd, Nisha Talagala, for their valuable comments. This work was supported by Samsung Research Funding Center of Samsung Electronics under Project Number SRFC-TB1503-03. For further information Please contact : hjkim@csl.skku.edu , yslee@calab.kaist.ac.kr , jinsookim@skku.edu More information on this and related projects can be obtained at https://github.com/nvmedirect Literature cited [1] NVM Express Overview. http://www.nvmexpress.org/about/nvm-express-overview/. [2] Redis. http://redis.io/. [3] AXBOE, J. Flexible IO tester. http://git.kernel.dk/?p=fio.git;a=summary. [4] CAULFIELD, A. M., DE, A., COBURN, J., MOLLOW, T. I.,GUPTA, R. K., AND SWANSON, S. Moneta: A highperformancestorage array architecture for next-generation, nonvolatile memories. In Proc. MICRO (2010), pp. 385–395. [5] CAULFIELD, A. M., MOLLOV, T. I., EISNER, L. A., DE, A.,COBURN, J., AND SWANSON, S. Providing safe, user space access to fast, solid state disks. In Proc. ASPLOS (2012), pp. 387–400. [6] COOPER, B. F., SILBERSTEIN, A., TAM, E., RAMAKRISHNAN,R., AND SEARS, R. Benchmarking cloud serving systems with YCSB. In Proc. SOCC (2010), pp. 143–154. [7] INTEL. Storage performance development kit. https://01.org/spdk. [8] INTEL, AND MICRON. Intel and Micron Produce Breakthrough Memory Technology. http://newsroom.intel. com/community/intel_newsroom/blog/2015/07/28/ intel-and-micron-produce-breakthrough-memory-technology,2015.