Understanding Intrinsic Characteristics and System Implications of Flash Memory based Solid State Drives Feng Chen, David A. Koufaty, and Xiaodong Zhang 2009 ACM SIGMETRICS/Performance Embedded Lab. Kim Sewoog

Motivation Solid State Drive(SSD) “pivotal technology” http://www.youtube.com/watch?v=96dWOEa4Djs http://www.youtube.com/watch?v=pJMGAdpCLVg&feature=fvw HDD SSD

< SSD Block Diagram > SSD Internals Array of flash memory packages Flash Translation Layer(FTL) in the SSD controller Hybrid mapping, Over-Provisioning, Using the original host interface(SATA) for compatibility Interleaving, DMA for data processing, low power comsumption, etc… < SSD Block Diagram >

Belief and betrayal of SSD The common belief Accesses to SSD are uncorrelated with access patterns! Betrayal Unexpected performance issues Uncertain behavior SSD is not just another ‘faster’ disk! We need to understand intrinsic limits unexpected performance behavior

Experiments and analysis 7 questions Access Patterns of workloads Random writes Caching for optimizing performance Interference between read and write operations Background operation effects Internal fragmentation System implications

Measurement environment Solid State Drives Experiment System DellTM PowerEdgeTM 1900 server Benchmarks Intel® Open Storage Toolkit : generate various types of I/O workloads blktrace / blkparse : trace and parse I/O activities (completion event)

General Tests Bandwidths 4 distinct workloads : Random/Sequential Read/Write Random workload : 4KB request size, 1024MB storage space Sequential workload : 256KB request size 32 parallel jobs, direct I/O, 30 seconds Comparison with harddisk (WD1600JS)

Micro-benchmark Workloads Various combinations of factors 3 access patterns : Sequential / Random / Stride 10 seconds running, one job, synchronous I/O Full utilization for initialization (using 256KB sequential write)

Distribution of access latencies Read operations on the SSD SSD-L : uniform distribution of latencies SSD-M/H : non-uniform distribution of latencies Reason : specification a readahead mechanism multi-plane operations interleaving 65% 65%

Distribution of access latencies Write operations on the SSD SSD-L : non-uniform distribution of latencies SSD-M/H : uniform distribution of latencies Independent of workload access patterns 88% over 90%

Distribution of access latencies Sequential vs. non-sequential writes on SSD-L (seems to) use a small buffer Sequential write : stripped Non-sequential write : no-stripped 64 requests initiate the prog. process & write data into flash memory in parallel from buffer to register from host to buffer

Disk cache effect of SSDs Large RAM cache(disk cache) Using hdparm tool to enable and disable the disk cache Disk cache off : increase of latencies both SSD-M/H Performance comparision between SSD-M/H without disk cache SSD-H is good performance -> SLC

Interference between read/write operations Writes : high-cost internal operations Cleaning and asynchronous write-back of dirty data from the disk cache Negatively affect foreground read operations Reads : competition for buffer space with writes Break sequential patterns 4 workload patterns Read(n) + Write(n) Write(n) + Read(n) Read(n) + Write(n+1) Read(n) + Write(n+4MB) only non-sequential pattern simultaneously, sequential pattern

Interference between read/write operations SSD-L Substantial degradation SSD-L optimizes performance for sequential writes Non-sequential write Non-shared buffer

Interference between read/write operations SSD-M/H readahead effect random read latency asynchronous write-back

Background operation effects Writes lead almost background operations Sequential workload using request size of 4KB Request type : random (50% write requests) interval time : 10ms disk cache Background operations are completed during the idle periods !

16MB: individual mapping unit Workload randomness effects Randomness effects (only SSD-L) Random write : random range from 1GB to 30GB Stride write : stride step from 4KB to 128MB Request size : 4KB 16MB: individual mapping unit metadata synchronization log block merging

Internal fragmentation Invalid pages in flash memory blocks Cleaning efficiency : block num x valid page num - read/write, block num - erase Non-continuous physical pages : readahead mechanism is not effective Over-provisioning (25% of the SSD capacity) No readahead effect

Conclusion Many well understood features of SSDs Many unexpected performance issues Access Patterns of workloads Random writes Caching for optimizing performance Interference between read and write operations Background operation effects Internal fragmentation System implications

Reference N. Agrawal, V. Prabhakaran, T. Wobber, J. D. Davis, M. Manasse, and R. Panigrahy. “Design tradeoffs for SSD performance”, In Proc. of USENIX’08, 2008. Blktrace. http://linux.die.net/man/8/blktrace. S. Lee, D. Park, T. Chung, D. Lee, S. Park, and H. Song. “A log buffer based flash translation layer using fully associative sector translation”. In IEEE Tran. on Embedded Computing Systems, 2007. M. Mesnier. Intel open storage toolkit. http://www.sourceforge.org/projects/intel-iscsi. V. Prabhakaran, T. L. Rodeheffeer, and L. Zhou. “Transactional flash.” In Proc. of OSDI’08, 2008.

Thank you !