WMPI 2006, Austin, Texas © 2006 John C. Koob An Empirical Evaluation of Semiconductor File Memory as a Disk Cache John C. Koob Duncan G. Elliott Bruce F. Cockburn VLSI Design Lab ECE Department University of Alberta Edmonton, Alberta Canada

WMPI 2006, Austin, Texas Slide 2John C. Koob, University of Alberta Outline  Motivation  Extended Storage  File Memory  Experimental Platform  Cost/Performance Analysis  Conclusions  Future Work

WMPI 2006, Austin, Texas Slide 3John C. Koob, University of Alberta Motivation Source: Computer Architecture, Hennessy & Patterson, 2003

WMPI 2006, Austin, Texas Slide 4John C. Koob, University of Alberta Access Time Gap Problem  Use Extended Storage  Cheaper per bit than main memory  Faster than disk  Slower than main memory  Not necessarily semiconductor media  Potential for power savings  How to fill the access time gap?

WMPI 2006, Austin, Texas Slide 5John C. Koob, University of Alberta Historical Systems  Extended storage first appeared in expensive systems  IBM 3090 mainframe  Main memory: 0.5 GB  Extended storage: 4 GB  Terminology: Expanded Storage Image courtesy of

WMPI 2006, Austin, Texas Slide 6John C. Koob, University of Alberta Historical Systems  Extended storage first appeared in expensive systems  Cray Y-MP supercomputer  Main memory: 1 GB of 15-ns bipolar SRAM  Extended storage: 4 GB of 50-ns DRAM  Terminology: Solid State Disk Image courtesy of the Charles Babbage Institute

WMPI 2006, Austin, Texas Slide 7John C. Koob, University of Alberta Recent Research  Compressed caching ( )  Use compression to reduce paging costs  Adaptive sizing of the compressed portion required  Evaluated using a modified Linux 2.4 kernel  Multi-level main memory (WMPI 2004)  30% of memory must run at DRAM speed  Remaining memory can be slower  Hardware compression  Remote memory box  Portions can be powered down

WMPI 2006, Austin, Texas Slide 8John C. Koob, University of Alberta Extended Storage Today?  Emerging technology may prompt a return to extended storage  Semiconductor file memory  Up to 5 times slower than DRAM  Must be cheaper per bit than DRAM  MEMS probe-based storage  5 times faster than disk  10 times more expensive than disk

WMPI 2006, Austin, Texas Slide 9John C. Koob, University of Alberta What is File Memory?  File memory leverages current DRAM technology  DRAM design constraints increase costs per bit  100% of nominal capacity must be functional  Contiguous address space  Consistently good access time  File memory relaxes DRAM’s design constraints  Bad block marking to improve yield  Address space is not contiguous  Improve density at the expense of performance (e.g. multi-level DRAM or hardware compression)

WMPI 2006, Austin, Texas Slide 10John C. Koob, University of Alberta Feasibility of File Memory  A precedent for file memory exists in the non-volatile memory market  NOR Flash memory  Limited capacity  Moderate reliability  Random-access supported  NAND Flash memory  High capacity  Low reliability  bad block marking  Restricted to sequential access Contiguous Memory Non-Contiguous Memory

WMPI 2006, Austin, Texas Slide 11John C. Koob, University of Alberta Extended Storage Disk Cache  To evaluate file memory as extended storage:  Require an experimental platform  Modify Linux OS kernel  ESDC Design Summary  High memory support  Page cache containment  Configurable performance  CPU caching issues  Performance metrics  Verification

WMPI 2006, Austin, Texas Slide 12John C. Koob, University of Alberta Configurable Performance  Need configurable file memory properties  Capacity  Access time  How to model different file memory access times?  Use multiple page copies  Gives accurate file memory slowdown ratios  Problem:  Repeated page copies would be cached  Solution:  Use IA-32 memory type range registers (MTRRs)  Disable CPU caches for ESDC

WMPI 2006, Austin, Texas Slide 13John C. Koob, University of Alberta Experimental Setup  Experimental Platform  Processor 2.4 GHz Pentium 4  Memory2 GB DDR SDRAM  Hard disk18-GB Seagate SCSI  Disk buffer4-MB  Experimental Suite  PostMark – benchmark for many small files  Bonnie – file system benchmark  Kernel compilation – Linux kernel build

WMPI 2006, Austin, Texas Slide 14John C. Koob, University of Alberta Postmark Results for Original Hierarchy

WMPI 2006, Austin, Texas Slide 15John C. Koob, University of Alberta Postmark Results using File Memory

WMPI 2006, Austin, Texas Slide 16John C. Koob, University of Alberta Postmark Results Analysis Need 39% more file memory for equivalent performance

WMPI 2006, Austin, Texas Slide 17John C. Koob, University of Alberta Summary of Postmark Results

WMPI 2006, Austin, Texas Slide 18John C. Koob, University of Alberta Conclusions  Use non-contiguous file memory for extended storage  Leverage DRAM cell technology  Relax DRAM design constraints  Use bad block marking  Preliminary evaluation of ESDC  Use file memory up to 4 times slower than DRAM  Even though ESDC replaces the page cache, system performance can be improved  Ongoing research  Evaluate hierarchies with file memory and page cache

WMPI 2006, Austin, Texas Slide 19John C. Koob, University of Alberta Selected References Bray. Bonnie Castro et al. Adaptive compressed caching. Symp. on Comp. Arch. And High Performance Computing, Nov Ekman and Stenstrom. A case for multi-level main memory. WMPI Hennessy and Patterson. Computer architecture: A quantitative approach. Third Edition, Katcher. PostMark: A new filesystem benchmark. TR3022, Network Appliance, Oct Koob et al. Test and characterization of a variable capacity multilevel DRAM. In Proc. VLSI Test Symp., pp , May Uysal et al. Using MEMS-based storage in disk arrays. FAST 2003, pp