Mass StorageCS510 Computer ArchitecturesLecture Lecture 18 Mass Storage Devices: RAID, Data Library, MSS

Mass StorageCS510 Computer ArchitecturesLecture Review: Improving Bandwidth of Secondary Storage Processor performance growth phenomenal I/O? I/O certainly has been lagging in the last decade Seymour Cray, Public Lecture (1976) Also, I/O needs a lot of work David Kuck, Keynote Address, (1988)

Mass StorageCS510 Computer ArchitecturesLecture Network Attached Storage High Performance Storage Service on a High Speed Network Decreasing Disk Diameters 14" » 10" » 8" » 5.25" » 3.5" » 2.5" » 1.8" » 1.3" »... high bandwidth disk systems based on arrays of disks Increasing Network Bandwidth 3 Mb/s » 10Mb/s » 50 Mb/s » 100 Mb/s » 1 Gb/s » 10 Gb/s networks capable of sustaining high bandwidth transfers Network provides well defined physical and logical interfaces: separate CPU and storage system! Network File Services OS structures supporting remote file access

Mass StorageCS510 Computer ArchitecturesLecture RAID

Mass StorageCS510 Computer ArchitecturesLecture Manufacturing Advantages of Disk Arrays Disk Product Families 3.5 Disk Array: 1 disk design Conventional: 4 disk designs Low End High End

Mass StorageCS510 Computer ArchitecturesLecture Replace Small Number of Large Disks with Large Number of Small Disks IBM 3390 (K) 20 GBytes 97 cu. ft. 3 KW 15 MB/s 600 I/Os/s 250 KHrs $250K IBM 3.5" MBytes 0.1 cu. ft. 11 W 1.5 MB/s 55 I/Os/s 50 KHrs $2K x70 23 GBytes 11 cu. ft. 1 KW 120 MB/s 3900 IOs/s ??? Hrs $150K Disk Arrays have potential for large data and I/O rates high MB per cu. ft., high MB per KW awful reliability Data Capacity Volume Power Data Rate I/O Rate MTTF Cost

Mass StorageCS510 Computer ArchitecturesLecture Redundant Arrays of Disks Files are "striped" across multiple spindles to gain throughput Increase of the number of disks reduces the reliability Redundancy yields high data availability Disks will fail Contents reconstructed from data redundantly stored in the array – Capacity penalty to store it – Bandwidth penalty to update Techniques: Mirroring/Shadowing (high capacity cost) Horizontal Hamming Codes (overkill) Parity & Reed-Solomon Codes Failure Prediction (no capacity overhead!) VaxSimPlus - Technique is controversial

Mass StorageCS510 Computer ArchitecturesLecture Array Reliability Reliability of N disks = Reliability of 1 Disk N 50,000 Hours 70 disks = 700 hours Disk system MTTF: Drops from 6 years to 1 month! Arrays without redundancy too unreliable to be useful! Hot spares support reconstruction in parallel with access: very high media availability can be achieved

Mass StorageCS510 Computer ArchitecturesLecture Redundant Arrays of Disks(RAID) High I/O Rate Parity Array Interleaved parity blocks Independent reads and writes Logical write = 2 reads + 2 writes Parity + Reed-Solomon codes Disk Mirroring, Shadowing Each disk is fully duplicated onto its "shadow" Logical write = two physical writes 100% capacity overhead Shadow Parity Data Bandwidth Array Parity computed horizontally Recovery purpose instead of fault detection Logically a single high data bw disk Parity

Mass StorageCS510 Computer ArchitecturesLecture Problems of Disk Arrays: Small Writes RAID-5: Small Write Algorithm 1 Logical Write = 2 Physical Reads + 2 Physical Writes D0D1D2 D3 P D0' D1D2 D Read old parity XOR + 1. Read old data XOR D0' new data 3. Write new data P' 4. Write new parity

Mass StorageCS510 Computer ArchitecturesLecture Redundant Arrays of Disks: RAID 1: Disk Mirroring/Shadowing Each disk is fully duplicated onto its "shadow" Very high availability can be achieved Bandwidth sacrifice on write: Logical write = two physical writes Reads may be optimized Most expensive solution: 100% capacity overhead Targeted for high I/O rate, high availability environments recovery group

Mass StorageCS510 Computer ArchitecturesLecture Redundant Arrays of Disks: RAID 3: Parity Disk logical record Striped physical records Parity computed across recovery group to protect against hard disk failures 33% capacity cost for parity in this configuration wider arrays reduce capacity costs, decrease expected availability, increase reconstruction time Arms logically synchronized, spindles rotationally synchronized logically a single high capacity, high transfer rate disk Targeted for high bandwidth applications: Scientific, Image Processing P

Mass StorageCS510 Computer ArchitecturesLecture Redundant Arrays of Disks: RAID 5+: High I/O Rate Parity Independent accesses occur in llel A logical write is 4 physical I/Os; 2 Reads and 2 Writes Independent writes, 1 data and 1 parity, possible because of interleaved parity Reed-Solomon Codes ("Q") for protection during reconstruction Targeted for mixed applications Stripe Unit Stripe Unit D0D1D2 D3 P0P0 D4D5D6 P1P1 D7 D8D9P2P2 D10 D11 D12P3P3 D13 D14 D15 P4P4 D16D17 D18 D19 D20D21D22 D23 P5P Disk Columns a disk

Mass StorageCS510 Computer ArchitecturesLecture Subsystem Organization host array controller single board disk controller single board disk controller single board disk controller single board disk controller host adapter physical device control often piggy-backed in small format devices Striping software off-loaded from host to array controller No applications modifications No reduction of host performance control, buffering, parity logic manages interface to host, DMA

Mass StorageCS510 Computer ArchitecturesLecture System Availability: Orthogonal RAIDs Redundant Support Components: fans, power supplies, controller, cables End to End Data Integrity: internal parity protected data paths Data Recovery Group: unit of data redundancy

Mass StorageCS510 Computer ArchitecturesLecture System-Level Availability Recovery Group Goal: No Single Points of Failure with duplicated paths, higher performance can be obtained when there are no failures Fully dual redundant I/O Controller Array Controller host

Mass StorageCS510 Computer ArchitecturesLecture Magnetic Tapes

Mass StorageCS510 Computer ArchitecturesLecture Memory Hierarchies General Purpose Computing Environment Memory Hierarchy Access Time Capacity Cost per bit File Cache Hard Disk Tapes

Mass StorageCS510 Computer ArchitecturesLecture Memory Hierarchies File Cache Hard Disk Tapes General Purpose Computing Environment Memory Hierarchy 1980 Off Line Storage On-Line Near-Line Disk Arrays Memory Hierarchy 1995 File Cache SSD High I/O Rate Disks High Data Rate Disks Optical Juke Box Automated Tape Libraries Low $/Actuator Low $/MB Remote Archive

Mass StorageCS510 Computer ArchitecturesLecture Storage Trends: Distributed Storage Storage Hierarchy 1980 Storage Hierarchy 1990 Local Magnetic Disk File Cache MagneticTape Server Remote Magnetic Disk LocalArea Network Server Cache Client Workstation File Server Declining $/MByte Increasing Access Time Capacity File Cache Magnetic Disk MagneticTape

Mass StorageCS510 Computer ArchitecturesLecture Storage Trends: Wide-Area Storage 1995 Typical Storage Hierarchy Conventional disks replaced by disk arrays Near-line storage emerges between disk and tape Local Area Network DiskArray Server Cache Shelved Magnetic or OpticalTape Optical Disk Jukebox Magnetic or OpticalTape Library Client Cache On-line Storage Near-line Storage Off-line Storage WideArea Network Internet

Mass StorageCS510 Computer ArchitecturesLecture What's All This About Tape? Tape is used for: Backup Storage for Hard Disk Data – Written once, very infrequently (hopefully never!) read Software Distribution – Written once, read once Data Interchange – Written once, read once File Retrieval – Written/Rewritten, files occasionally read – Near Line Archive – Electronic Image Management Relatively New Application For Tape

Mass StorageCS510 Computer ArchitecturesLecture Alternative Data Storage Technologies * Second Generation 8mm: 5000 MB, 500KB/s ** Second Generation 4mm: GB Conventional Tape: Reel-to-Reel (.5") minutes Cartridge (.25") minutes CapBPITPIBPI*TPIData Xfer Acc Time Technology(MB)(Million)(KByte/s) Helical Scan Tape: VHS (.5") minutes Video (8mm)* minutes DAT (4mm)** seconds Disk: Hard Disk (5.25") ms Floppy Disk (3.5") second CD ROM (3.5") second

Mass StorageCS510 Computer ArchitecturesLecture R-DAT Technology Two Competing Standards DDS (HP, Sony) * 22 frames/group * 1870 tpi * Optimized for serial writes DataDAT (Hitachi, Matsushita, Sharp) * Two modes: streaming (like DDS) and update in place * Update in place sacrifices transfer rate and capacity Spare data groups, inter-group gaps, preformatted tapes

Mass StorageCS510 Computer ArchitecturesLecture R-DAT Technology Advantages: * Small Formfactor, easy handling/loading * 200X speed search on index fields (40 sec. max, 20 sec. avg.) * 1000X physical positioning (8 sec. max, 4 sec. avg.) * Inexpensive media ($10/GBytes) * Volumetric Efficiency: 1 GB in 2.5 cu. in; 1 TB in 1 cu. ft. Disadvantages: * Two incompatible standards (DDS, DataDAT) * Slow XFER rate * Lower capacity vs. 8mm tape * Small bit size (13 x 0.4 sq. micron) effect on archive stability

Mass StorageCS510 Computer ArchitecturesLecture R-DAT Technical Challenges Tape Capacity * Data Compression is key Tape Bandwidth * Data Compression * Striped Tape

Mass StorageCS510 Computer ArchitecturesLecture MSS Tape: No Perfect? Tape Drive Best 2 out of 3 Cost, Size, Speed Expensive (Fast & big) Cheap (Slow & big)

Mass StorageCS510 Computer ArchitecturesLecture Data Compression Issues Peripheral Manufacturer Approach: Host SCSI HBA Embedded Controller Transport Host SCSI HBA Video Compression Audio Compression Image Compression Text Compression... Embedded Controller Transport System Approach: Compression Done Here 20:1 2,3:1 Data Specific Compression Hints from Host

Mass StorageCS510 Computer ArchitecturesLecture Striped Tape Speed Matching Buffers Embedded Controller Transport Embedded Controller Transport Embedded Controller Transport Embedded Controller Transport To/From Host 180 KB/s Challenges: * Difficult to logically synchronize tape drives * Unpredictable write times R after W verify, Error Correction Schemes, N Group Writing, Etc.

Mass StorageCS510 Computer ArchitecturesLecture Automated Media Handling Tape Carousels Gravity Feed 3.5" formfactor tape reader 19" Carousel 4mm Tape Reader

Mass StorageCS510 Computer ArchitecturesLecture Automated Media Handling Front View Side View Tape Readers Tape Cassette Tape Pack: Unit of Archive

Mass StorageCS510 Computer ArchitecturesLecture MSS: Automated Tape Library 116 x 5 GB 8 mm tapes = 0.6 TBytes (1991) 4 tape readers 1991, 8 half height readers now 4 x.5 MByte/second = 2 MBytes/s $40,000 O.E.M. Price Predict 1995: 3 TBytes; 2000: 9 TBytes

Mass StorageCS510 Computer ArchitecturesLecture Open Research Issues Hardware/Software attack on very large storage systems » File system extensions to handle terabyte sized file systems » Storage controllers able to meet bandwidth and capacity demands Compression/decompression between secondary and tertiary storage » Hardware assist for on-the-fly compression » Application hints for data specific compression » More effective compression over large buffered data » DB indices over compressed data Striped tape: is large buffer enough? * Applications: Where are the Terabytes going to come from? » Image Storage Systems » Personal Communications Network multimedia file server

Mass StorageCS510 Computer ArchitecturesLecture MSS: Applications of Technology Robo-Line Library Books/Bancroft x Pages/book x bytes/page = Bancroft 372, = 0.54 TB Full text Bancroft Near Line = 0.5 TB; Pages images ? 20 TB Predict: "RLB" (Robo-Line Bancroft) = $250,000 Bancroft costs: Catalogue a book: $20 / book Reshelve a book: $1/ book % new books purchased per year never checked out: 20%

Mass StorageCS510 Computer ArchitecturesLecture MSS: Summary Access Time (ms) $0.00$0.01$0.10$1.00 $10.00 $ Robo-Line Tape Magnetic Disk DRAM Access Gap #1 Access Gap #2 $ / MB