WMPI 2006, Austin, Texas © 2006 John C. Koob An Empirical Evaluation of Semiconductor File Memory as a Disk Cache John C. Koob Duncan G. Elliott Bruce F. Cockburn VLSI Design Lab ECE Department University of Alberta Edmonton, Alberta Canada

WMPI 2006, Austin, Texas Slide 2John C. Koob, University of Alberta Outline  Motivation  Extended Storage  File Memory  Experimental Platform  Cost/Performance Analysis  Conclusions

WMPI 2006, Austin, Texas Slide 3John C. Koob, University of Alberta Motivation Source: Computer Architecture, Hennessy & Patterson, 2003

WMPI 2006, Austin, Texas Slide 4John C. Koob, University of Alberta Access Time Gap Problem  Use Extended Storage  Cheaper per bit than main memory  Faster than disk  Slower than main memory  Potential for power savings  How to fill the access time gap?

WMPI 2006, Austin, Texas Slide 5John C. Koob, University of Alberta Historical Systems  Extended storage first appeared in expensive systems  IBM 3090 mainframe  Main memory: 0.5 GB  Extended storage: 4 GB  Terminology: Expanded Storage Image courtesy of

WMPI 2006, Austin, Texas Slide 6John C. Koob, University of Alberta Historical Systems  Extended storage first appeared in expensive systems  Cray Y-MP supercomputer  Main memory: 1 GB of 15-ns SRAM  Extended storage: 4 GB of 50-ns DRAM  Terminology: Solid State Disk Image courtesy of the Charles Babbage Institute

WMPI 2006, Austin, Texas Slide 7John C. Koob, University of Alberta Recent Research  Compressed caching ( )  Compression can reduce paging costs  Adaptive sizing of compressed page cache  Multi-level main memory (WMPI 2004)  30% of memory must run at DRAM speed  Remaining memory can be slower

WMPI 2006, Austin, Texas Slide 8John C. Koob, University of Alberta Extended Storage Today?  Emerging technology may prompt a return to extended storage  Semiconductor file memory  Up to 5 times slower than DRAM  Cheaper per bit than DRAM  MEMS probe-based storage  5 times faster than disk  10 times more expensive than disk

WMPI 2006, Austin, Texas Slide 9John C. Koob, University of Alberta What is File Memory?  File memory leverages current DRAM technology  DRAM design constraints increase costs per bit  100% of nominal capacity must be functional  Contiguous address space  File memory relaxes DRAM’s design constraints  Bad block marking to improve yield  Address space is not contiguous  Improve density at the expense of performance (e.g. multi-level DRAM or hardware compression)

WMPI 2006, Austin, Texas Slide 10John C. Koob, University of Alberta Feasibility of File Memory  A precedent for file memory exists in the non-volatile memory market  NOR Flash memory  Limited capacity  Moderate reliability  Random-access supported  NAND Flash memory  High capacity  Low reliability  bad block marking  Restricted to sequential access Contiguous Memory Non-Contiguous Memory

WMPI 2006, Austin, Texas Slide 11John C. Koob, University of Alberta Extended Storage Disk Cache  To evaluate file memory as extended storage:  Require an experimental platform  Modify Linux OS kernel  ESDC Design Summary  High memory support  Page cache containment  Configurable performance  CPU caching issues  Performance metrics

WMPI 2006, Austin, Texas Slide 12John C. Koob, University of Alberta Postmark Results using File Memory

WMPI 2006, Austin, Texas Slide 13John C. Koob, University of Alberta Postmark Results Analysis Need 39% more file memory for equivalent performance

WMPI 2006, Austin, Texas Slide 14John C. Koob, University of Alberta Summary of Postmark Results

WMPI 2006, Austin, Texas Slide 15John C. Koob, University of Alberta Conclusions  Use file memory for extended storage  Leverage DRAM cell technology  Relax DRAM design constraints  Use bad block marking  Preliminary evaluation of ESDC  File memory can be up to 4 times slower than DRAM  Performance improved even with no page cache  Ongoing research  Evaluate hierarchies with file memory and page cache

WMPI 2006, Austin, Texas Slide 16John C. Koob, University of Alberta Selected References Bray. Bonnie Castro et al. Adaptive compressed caching. Symp. on Comp. Arch. And High Performance Computing, Nov Ekman and Stenstrom. A case for multi-level main memory. WMPI Hennessy and Patterson. Computer architecture: A quantitative approach. Third Edition, Katcher. PostMark: A new filesystem benchmark. TR3022, Network Appliance, Oct Koob et al. Test and characterization of a variable capacity multilevel DRAM. In Proc. VLSI Test Symp., pp , May Uysal et al. Using MEMS-based storage in disk arrays. FAST 2003, pp

WMPI 2006, Austin, Texas Slide 17John C. Koob, University of Alberta Configurable Performance  How to model different file memory access times?  Use multiple page copies  Gives accurate file memory slowdown ratios  Problem:  Repeated page copies would be cached  Solution:  Disable CPU caches for ESDC – Use IA-32 memory type range registers (MTRRs)

WMPI 2006, Austin, Texas Slide 18John C. Koob, University of Alberta Experimental Setup  Experimental Platform  Processor 2.4 GHz Pentium 4  Memory2 GB DDR SDRAM  Hard disk18-GB Seagate SCSI  Disk buffer4-MB  Experimental Suite  PostMark – benchmark for many small files  Bonnie – file system benchmark  Kernel compilation – Linux kernel build

WMPI 2006, Austin, Texas Slide 19John C. Koob, University of Alberta Postmark Results for Original Hierarchy