RAID Systems Ver.2.0 Jan 09, 2005 Syam

RAID Primer Redundant Array of Inexpensive Disks random, real-time, redundant, array, assembly, interconnected, independent, inter-relation, devices, drives, etc. Physical Drive Actual Hard Disks Physical Array One or more physical drive Logical Array Formed by splitting or combining physical arrays Logical Drive One or more logical array

Drive Layout Physical Drive 0 Physical Array 0 Physical Array 1 Logical Array 2 Logical Drive Logical Array 0 Logical Array 1

Why RAID CPU and Memory Disk I/O Disk Reliability MTBF (Mean Time Between Failures) = MTBF Single Drive / # Drives Fault Tolerance Use multiple small, inexpensive disks into array which yields performance exceeding that of Single Large Expensive Disk

RAID Benefits Higher Data Security Fault Tolerance Improved Availability Increased, Integrated Capacity Improved Performance

RAID Costs Planning and Design Hardware Software Setup and Training Maintenance

RAID Tradeoffs

RAID Misconceptions Blanket Statements “Invulnerability Complex” AID 0 not RAID 0

Hardware vs. Software RAID Hardware RAID Hardware manages the RAID independently from the host and gives the host a single drive per array Controller operates simultaneously with system Highly fault tolerant Software RAID Software manages RAID Lives in host memory and consumes CPU cycles Array only functional when array software is loaded (array down, array software load?)

Hardware vs. Software RAID Hardware RAIDSoftware RAID ImplementationDedicated HardwareSoftware Kernel Automatic Failover Yes How SwapYesNo CostHundreds of DollarsNone CPU ImpactNegligibleTypically 5-15%

RAID Concepts Mirroring – One method of data redundancy Data written simultaneously to two hard disks 100% redundancy protects against failure of any of the disks My DATA

RAID Concepts Striping – Disks used in parallel Each drive partitioned into stripes from one sector (512 bytes) to several MBs Partition size referred to as Striping Unit Pieces of files are stored on multiple disks Files can be broken up into bytes or blocks Drive 0 Drive 1 Drive 2 Striping Unit

Parity – Another method of data redundancy Take N pieces of data and calculate another piece and store the N+1 pieces on separate drives If any one of the N+1 pieces of data is lost, it can be recreated from the other N RAID Concepts

Parity Example RAID Concepts D1 = D2 = D3 = D4 = Parity = D1 XOR D2 XOR D3 XOR D4 = ((( XOR ) XOR ) XOR ) = Now five pieces of data are stored on five separate disks Assume D3 becomes corrupt, can be restored by: D3 = D1 XOR D2 XOR D4 XOR Parity = ((( XOR ) XOR ) XOR ) =

RAID operation in degraded state Two-drive mirrored => performance equals that of a single drive Striped array with parity => regenerating lost information Rebuilding Two-drive mirrored => copy entire good drive to replacement drive Striped array with parity => must determine new parity information RAID Degraded Operation and Rebuilding

RAID can continue to operate during rebuild Hardware RAID rebuilds faster than software RAID Automatic rebuild Controller detects failed drive Automatically rebuilds on replacement Manual rebuild Administrator initiates rebuild Can be run in off-peak time RAID Degraded Operation and Rebuilding

RAID Reliability Component Reliability System Reliability Function of reliability of components

RAID Reliability Example RAID with 4 drives with MTBF 500,000 hours Reliability decreased from 500,000 to 88,235 => decreased 82% RAID reliability referred to RAID with fault tolerance

Ability of RAID system to withstand loss of some hardware without loss of data or availability When fault occurs, array enters degraded state Drive must be replaced Array must be rebuilt RAID Fault Tolerance

Ability for users to access data Depends on: Hardware Reliability Fault Tolerance Hot Swapping Automatic Rebuilding Service RAID Availability

Most RAID levels use striping Possible threats to data integrity Unexpected Hard Disk Failure Failures of Support Hardware Physical Damage Software Problems Viruses Human Error RAID Backups

1988 Paper defined RAID levels 1- 5 Now single RAID levels 0 – 7 Multiple RAID levels RAID Levels

JBOD – Just a Bunch Of Disks Spanning multiple physical drives into one logical drive No Fault Tolerance Not a RAID

JBOD – Just a Bunch Of Disks

RAID 0 - Striping Disk Striping No parity Example: Write essay with 3 hands instead of 1 Increased IO Not a valid RAID implementation due to lack of fault tolerance

RAID 0 - Striping

Supported by all hardware controllers Supported by most software Minimum of two hard disks Array Capacity = Smallest Drive Size * # of Drives Storage Efficiency = 100% of drive Fault Tolerance: None Availability: Lowest of all RAID levels Failure results in array down until rebuild and restore Degradation and Rebuilding: Not Applicable

RAID 0 - Striping Random Read Performance: Very Good, increases with larger stripe size Random Write Performance: Very Good, increases with larger stripe size Sequential Read Performance: Very Good to Great Sequential Write Performance: Very Good Cost: Lowest of all RAID levels Special Considerations: Daily Backups Uses: Non-critical data, Hobbyist, high-end gaming

RAID 1 - Mirroring 100% Data Redundancy No IO Speed Increase When a drive fails, the other operates until drive replaced

RAID 1 - Mirroring

Supported by all hardware controllers Supported by most software Exactly two hard disks Array Capacity = Smallest Drive Size Storage Efficiency = 50% of drives Fault Tolerance: Very Good Availability: Very Good Most allow hot spare and automatic rebuilding Degradation and Rebuilding: Slight degradation of read, write improves Rebuilding is relatively fast

RAID 1 - Mirroring Random Read Performance: Good Random Write Performance: Good Sequential Read Performance: Fair Sequential Write Performance: Good Cost: Relatively High Special Considerations: Size Limitation Uses: High Fault Tolerance without high capacity, small databases, accounting and financial data

RAID 2 – Memory-Style ECC Introduces Parity Same principle as ECC memory Bit-level with Hamming Code Not used today Cost, Complexity

RAID 2 - Memory-Style ECC

Special Hardware Controller Required Typically 10 Data Disks and 4 ECC Disks Array Capacity = 10 * Data Disk Size Storage Efficiency = 71% of drives Fault Tolerance: Fair Availability: Very Good “On the fly” error correction Degradation and Rebuilding: In theory little degradation

RAID 2 - Memory-Style ECC Random Read Performance: Fair Random Write Performance: Poor Sequential Read Performance: Very Good Sequential Write Performance: Fair to Good Cost: Very Expensive Special Considerations: Not in modern systems Uses: Not Used in Modern Systems

RAID 3 – Bit-Interleaved Parity Byte Level striping with dedicated parity disk Read requests hit all data disks Write requests hit all data disks and parity disk Great for high bandwidth but not high I/O rates Parity Disk can be bottleneck

RAID 3 - Bit-Interleaved Parity

Medium to high-end hardware controller required Minimum of three hard disks Array Capacity = Smallest Drive Size*(# Drives–1) Storage Efficiency = (# Drives-1)/# Drives Fault Tolerance: Good Availability: Very Good Hot swapping and automatic rebuild Degradation and Rebuilding: Relative little degradation and rebuilds can take many hours

RAID 3 - Bit-Interleaved Parity Random Read Performance: Good Random Write Performance: Poor Sequential Read Performance: Very Good Sequential Write Performance: Fair to Good Cost: Moderate Special Considerations: Not as popular as other Uses: Large Files with High transfer performance, multimedia, publishing

RAID 4 – Block-Interleaved Parity Block-level striping with dedicated parity disk Write requests use read-modify-write i.e. four disks (3 data, 1 parity) Small write request 4 disk IO write the new data to disk 0 (1) read old data from disk 1 & disk 2 (2) Write parity information (1) Parity disk can become bottleneck

RAID 4 – Block-Interleaved Parity

Medium to high-end hardware controller required Minimum of three hard disks Array Capacity = Smallest Drive Size*(# Drives–1) Storage Efficiency = (# Drives-1)/# Drives Fault Tolerance: Good Availability: Very Good Hot swapping and automatic rebuild Degradation and Rebuilding: Moderate if drive fails and potential lengthy rebuild RAID 4 – Block-Interleaved Parity

Random Read Performance: Very Good Random Write Performance: Poor to Fair Sequential Read Performance: Good to Very Good Sequential Write Performance: Fair to Good Cost: Moderate Special Considerations: Performance depends on stripe size Uses: Not as common as level 3 or 5 Large Files with High transfer performance RAID 4 – Block-Interleaved Parity

RAID 5 – Block-Interleaved Distributed Parity One of most popular levels Eliminates the parity disk bottleneck by distributing parity information across the array More efficient with small read and large write requests

RAID 5 – Block-Interleaved Distributed Parity

Moderately high-end hardware controller required Supported by some software solutions Minimum of the hard disks Array Capacity = Smallest Drive Size*(# Drives–1) Storage Efficiency = (# Drives-1)/# Drives Fault Tolerance: Good Availability: Good to Very Good Hot swapping and automatic rebuild Degradation and Rebuilding: Can be substantial due to distributed parity

RAID 5 – Block-Interleaved Distributed Parity Random Read Performance: Very Good to Great Random Write Performance: Fair Sequential Read Performance: Good to Very Good Sequential Write Performance: Fair to Good Cost: Moderate Special Considerations: Software RAID can greatly affect performance due to parity calculations Uses: Seen as Middle of RAID tradeoff triangle

RAID 6 – P+Q Redundancy Block-level striping with dual distributed parity Adds 2D parity information Can handle up to multiple disk failures High data fault tolerance for mission critical applications

RAID 6 – P+Q Redundancy

Special Hardware Controller Required Typically Minimum of 4 Hard Disks Array Capacity = Smallest Drive Size * # Drives-2 Storage Efficiency = (#Drives - 2)/#Drives Fault Tolerance: Very Good to Great Availability: Great Degradation and Rebuilding: Can be substantial due to dual distributed parity

RAID 6 – P+Q Redundancy Random Read Performance: Very Good to Great Random Write Performance: Poor Sequential Read Performance: Good to Very Good Sequential Write Performance: Fair Cost: High Special Considerations: Tends to be used in proprietary systems Uses: Where RAID 5 plus more fault tolerance

Multiple RAID Levels Combine two single levels to obtain improved performance Most common level 01 and 10 RAID level X+Y ≠ Y+X Usually not much impact on capacity More impact on fault tolerance

Multiple RAID Levels RAID 01 vs 10 RAID 01 Strip Drives 1,2 RAID 0 for Stripe A Stripe Drives 3,4 RAID 0 for Stripe B Mirror two sets, if Drive 2 fails Stripe A is lost RAID 10 Mirror Drives 1,2 RAID 1 for Mirror A Mirror Drives 3,4 RAID 1 for Mirror B Stripe across A and B, if Drive 2 fails still have Drive 1 maintaining stripe

RAID 10 – Mirrored Stripe Mirroring and striping without parity Most Common of Multiple levels Large arrays with high performance and high fault tolerance

RAID 10 - Mirrored Stripe

Most Hardware Controllers Support Even Number with Minimum of 4 Hard Disks Array Capacity = Smallest Drive Size * # Drives/2 Storage Efficiency = 50% Fault Tolerance: Very Good to Great Availability: Great Degradation and Rebuilding: Relatively Little

RAID 10 - Mirrored Stripe Random Read Performance: Very Good to Great Random Write Performance: Good to Very Good Sequential Read Performance: Very Good to Great Sequential Write Performance: Good to Very Good Cost: High Special Considerations: Low storage efficiency Uses: High performance and reliability, enterprise servers

RAID Comparison