1. 1 COMP 206: Computer Architecture and Implementation Montek Singh Mon., Dec. 1, 2003 Topic: Storage Systems (Disk Technology) – contd.

2. 2 Ensembles of Disks  Key idea Use collection of disk drives to improve characteristics of disk systems (storage capacity, bandwidth, etc.) Use collection of disk drives to improve characteristics of disk systems (storage capacity, bandwidth, etc.)  Used in mainframes for a long time  RAID Redundant Array of Inexpensive Disks (original 1988 acronym) Redundant Array of Inexpensive Disks (original 1988 acronym) Redundant Array of Independent Disks (redefined in 1992) Redundant Array of Independent Disks (redefined in 1992)

3. 3 Improving Bandwidth with Disk Arrays  Arrays of independent disk drives Similar to high-order interleaving in main memories Similar to high-order interleaving in main memories Each file assigned to different disk drive Each file assigned to different disk drive Simultaneous access to files Simultaneous access to files Load balancing issues Load balancing issues  File striping/disk striping/disk interleaving Single file distributed across array of disks Single file distributed across array of disks Similar to low-order interleaving in main memories Similar to low-order interleaving in main memories Each logical I/O request corresponds to a data stripe Each logical I/O request corresponds to a data stripe  Data stripe divided into number of equal sized stripe units  Stripe units assigned to different disk units Two kinds of striping depending on size of stripe unit Two kinds of striping depending on size of stripe unit  Fine-grained striping: Stripe unit chosen to balance load  Coarse-grained striping: Larger stripe unit

4. 4 Improving Availability with Disk Arrays  Failure rates of Disk Arrays can be quite high: MTTF of large-system disks approaches 1,000,000 hours MTTF of large-system disks approaches 1,000,000 hours MTTF of PC-class disks approaches 150,000 hours MTTF of PC-class disks approaches 150,000 hours But, array of 1,000 PC-class disks has MTTF of 150 hours But, array of 1,000 PC-class disks has MTTF of 150 hours  All schemes to cope with low MTTF aim to “fail soft” Operation should be able to continue while repair is made Operation should be able to continue while repair is made Always depends on some form of redundancy Always depends on some form of redundancy

5. 5RAID-0 Strip 12 Strip 8 Strip 4 Strip 0 Strip 13 Strip 9 Strip 5 Strip 1 Strip 14 Strip 10 Strip 6 Strip 2 Strip 15 Strip 11 Strip 7 Strip 3  Striped, non-redundant Parallel access to multiple disks Parallel access to multiple disks  Excellent data transfer rate (for small strips)  Excellent I/O request processing rate (for large strips)  Typically used for applications requiring high performance for non-critical data

6. 6RAID-1 Strip 3 Strip 2 Strip 1 Strip 0 Strip 3 Strip 2 Strip 1 Strip 0  Mirrored/replicated (most costly form of redundancy)  I/O request rate: good for reads, fair for writes  Data transfer rate: good for reads; writes slightly slower Read can be serviced by the disk with the shorter seek distance Read can be serviced by the disk with the shorter seek distance Write must be handled by both disks Write must be handled by both disks  Typically used in system drives and critical files Banking, insurance data Banking, insurance data Web (e-commerce) servers Web (e-commerce) servers

7. 7 Combining RAID-0 and RAID-1 Strip 12 Strip 8 Strip 4 Strip 0 Strip 13 Strip 9 Strip 5 Strip 1 Strip 14 Strip 10 Strip 6 Strip 2 Strip 15 Strip 11 Strip 7 Strip 3 Strip 12 Strip 8 Strip 4 Strip 0 Strip 13 Strip 9 Strip 5 Strip 1 Strip 14 Strip 10 Strip 6 Strip 2 Strip 15 Strip 11 Strip 7 Strip 3  Can combine RAID-0 and RAID-1: Mirrored stripes (RAID 0+1, or RAID 01) Mirrored stripes (RAID 0+1, or RAID 01)  Example: picture above Striped Mirrors (RAID 1+0, or RAID 10) Striped Mirrors (RAID 1+0, or RAID 10)  Data transfer rate: fair for reads and writes  Reliability: good  Efficiency: poor (100% overhead in terms of disk utilization)

8. 8RAID-3 b0b0 b1b1 b2b2 b3b3 P(b)  Fine-grained (bit) interleaving with parity E.g., parity = sum modulo 2 (XOR) of all bits E.g., parity = sum modulo 2 (XOR) of all bits  Disks are synchronized, parity computed by disk controller  When one disk fails… (how do you know?) Data is recovered by subtracting all data in good disks from parity disk Data is recovered by subtracting all data in good disks from parity disk Recovering from failures takes longer than in mirroring, but failures are rare, so is okay Recovering from failures takes longer than in mirroring, but failures are rare, so is okay Hot spares used to reduce vulnerability in reduced mode Hot spares used to reduce vulnerability in reduced mode  Performance: Poor I/O request rate Poor I/O request rate Excellent data transfer rate Excellent data transfer rate Typically used in large I/O request size applications, such as imaging or CAD Typically used in large I/O request size applications, such as imaging or CAD

9. 9RAID-2 b0b0 b1b1 b2b2 b3b3 f 0 (b)f 1 (b)f 2 (b)  Hamming codes capable of correcting two or more erasures E.g., single error-correcting, double error-detecting (SEC-DED) E.g., single error-correcting, double error-detecting (SEC-DED)  Problem with small writes (similar to DRAM cycle time/access time)  Poor I/O request rate  Excellent data transfer rate

10. 10RAID-4  Coarse-grained striping with parity  Unlike RAID-3, not all disks need to be read on each read New parity computed by computing difference between old and new data New parity computed by computing difference between old and new data  Drawback: Like RAID-3, parity disk involved in every write; serializes small reads Like RAID-3, parity disk involved in every write; serializes small reads  I/O request rate: excellent for reads, fair for writes  Data transfer rate: good for reads, fair for writes Blk 12 Blk 8 Blk 4 Blk 0 Blk 13 Blk 9 Blk 5 Blk 1 Blk 14 Blk 10 Blk 6 Blk 2 Blk 15 Blk 11 Blk 7 Blk 3 P(12-15) P(8-11) P(4-7) P(0-3)

11. 11RAID-5 Blk 12 Blk 8 Blk 4 Blk 0 P(12-15) Blk 9 Blk 5 Blk 1 Blk 13 P(8-11) Blk 6 Blk 2 Blk 14 Blk 10 P(4-7) Blk 3 Blk 15 Blk 11 Blk 7 P(0-3)  Key Idea: reduce load on parity disk Block-interleaved distributed parity Block-interleaved distributed parity Multiple writes can occur simultaneously Multiple writes can occur simultaneously Block 0 can be accessed in parallel with Block 5 Block 0 can be accessed in parallel with Block 5  First needs disks 1 and 5; second needs disks 2 and 4  I/O request rate: excellent for reads, good for writes  Data transfer rate: good for reads, good for writes  Typically used for high request rate, read-intensive data lookup

12. 12 Removable Media  Magnetic Tapes: Have been used in computer systems as long as disks Have been used in computer systems as long as disks Similar magnetic technology as disks Similar magnetic technology as disks  have followed similar density improvements as disks Long strips wound on removable spools Long strips wound on removable spools Benefits: can be essentially of “unlimited” length; removable Benefits: can be essentially of “unlimited” length; removable Drawbacks: only offer sequential access; wear out Drawbacks: only offer sequential access; wear out Used for archival backups; but on the way out Used for archival backups; but on the way out  Optical Disks: CD’s and DVD’s CD-ROM/DVD-ROM: removable, inexpensive, random access CD-ROM/DVD-ROM: removable, inexpensive, random access CD-R/RW, DVD-R/RAM: writable/re-writable CD-R/RW, DVD-R/RAM: writable/re-writable Becoming immensely popular: replacing floppy drives, tape drives Becoming immensely popular: replacing floppy drives, tape drives  Flash: flash cards, memory sticks, digital film  Microdrives: small portable hard disks (up to 1GB)

13. 13 Interfacing I/O Subsystem to CPU Where do we connect I/O devices? Cache? Cache? Memory? Memory?  Low-cost systems: I/O bus = memory bus! –CPU and I/O compete for bus How does CPU address I/O devs.? Memory-mapped I/O Memory-mapped I/O  Portions of address space assigned to I/O devices I/O opcodes (not very popular) I/O opcodes (not very popular) How does CPU synch with I/O? Polling vs. interrupt-driven Polling vs. interrupt-driven Real-time systems: hybrid Real-time systems: hybrid  Clock interrupts periodically  CPU polls during interrupt

14. 14 Bus: Different Options  Bus width High-perf: separate address and data lines High-perf: separate address and data lines Low-cost: multiplex address and data lines Low-cost: multiplex address and data lines  Date width High-perf: wider is faster (e.g., 64 bits) High-perf: wider is faster (e.g., 64 bits) Low-cost: narrower is cheaper (e.g., 8 bits) Low-cost: narrower is cheaper (e.g., 8 bits)  Transfer size High-perf: multiple words have less bus overhead High-perf: multiple words have less bus overhead Low-cost: single-word transfer is simpler Low-cost: single-word transfer is simpler  Bus masters High-perf: multiple (requires arbitration) High-perf: multiple (requires arbitration) Low-cost: single master (simpler) Low-cost: single master (simpler)  Split transaction High-perf: Yes (request and reply in separate “packets”) High-perf: Yes (request and reply in separate “packets”) Low-cost: No (continuous connection is cheaper) Low-cost: No (continuous connection is cheaper)  Clocking High-perf: synchronous (though it is changing) High-perf: synchronous (though it is changing) Low-cost: asynchronous Low-cost: asynchronous

15. 15 Examples of Bus Standards IDE/Ultra ATA SCSIPCIPCI-X Data width 16 bits 8 or 16 bits 32 or 64 bits Clock rate Up to 100 MHz 10-160 MHz 33-66 MHz 66-133 MHz # Bus masters 1MultipleMultipleMultiple Peak Bandwidth 200 MB/sec 320 MB/sec 533 MB/sec 1066 MB/sec ClockingAsyncAsyncSyncSync

16. 16 Does I/O Performance Matter?  The “No” Argument: “There is always another process to run while one process waits for I/O to complete” “There is always another process to run while one process waits for I/O to complete”  Counter-arguments: Above argument applies only if throughput is sole goal Above argument applies only if throughput is sole goal Interactive software and personal computers lay more emphasis on response time (=latency), not throughput Interactive software and personal computers lay more emphasis on response time (=latency), not throughput  Interactive multimedia, transaction processing Process switching actually increases I/O (paging traffic) Process switching actually increases I/O (paging traffic) Desktop/Mobile computing: only one person/computer  fewer processes than in time-sharing Desktop/Mobile computing: only one person/computer  fewer processes than in time-sharing Amdahl’s Law: benefits of making CPU faster taper off if I/O becomes the bottleneck Amdahl’s Law: benefits of making CPU faster taper off if I/O becomes the bottleneck

17. 17 Does CPU Performance Matter? Consequences of Moore’s Law: Large, fast CPU’s Large, fast CPU’s Small, cheap CPU’s Small, cheap CPU’s  Main goal: keeping I/O devices busy, not keeping CPU busy –bulk of the hardware cost is in I/O peripherals, not CPU Shift in focus: Shift in focus:  From computation to communication  Reflected in change of terminology: –1960’s-80’s: Computing Revolution –1990’s-present: Information Age

18. 18 Does Performance Matter? Performance is not the problem it once was! 15 years of doubling CPU speed every 18 months (x1000) 15 years of doubling CPU speed every 18 months (x1000) Disks also have steadily become bigger and faster Disks also have steadily become bigger and faster Most people would prefer more reliability than speed: Today’s speeds come at a cost: frequent crashes Today’s speeds come at a cost: frequent crashes Program crashes  frustration Program crashes  frustration Disk crashes  hysteria Disk crashes  hysteria  Reliability, availability, dependability, etc. are the key terms  Client-server model of computing has made reliability the key criterion for evaluation –E.g., UNC-COMPSCI IMAP server: 300 MHz or something, 99.9… % uptime