Chapter 8 Adding Disks

Loops Run instructions again, and again, and again…. 2 types of loops Check some condition If true, repeat 2 types of loops for loop while loop First, the variable used in loop condition must be initialized, then execution of the loop begins. A test (condition) is made at the beginning of each iteration. The body of loop ends with a statement that modifies the value of the test (condition) variable.

For Loop Syntax: for ( init; condition; increment ) { statement(s); } Example #!/usr/bin/perl # for loop execution for( $a = 10; $a < 20; $a = $a + 1 ) { print "value of a: $a\n";

While Loop Syntax: while(condition) { statement(s); } #!/usr/bin/perl # while loop execution while( $a < 20 ) { print "Value of a: $a\n"; $a = $a + 1;

Foreach Loop The foreach loop iterates over a list value and sets the control variable (var) to be each element of the list in turn: Syntax: The syntax of a foreach loop in Perl programming language is: foreach var (list) { ... } #!/usr/local/bin/perl @list = (2, 20, 30, 40, 50); # foreach loop execution foreach $a (@list) { print "value of a: $a\n"; }

Lets just quickly see how to add a disk Attach hard drive and reboot No RAID (will cover later) One logical volume (will cover later) Identify and format the right drive Linux recipe Type “sudo fdisk –l” to list the system disks and identify a new drive Use a partitioning utility to create partition table Put all drives spare in one partition of unspecified type.

Lets just quickly see how to add a disk cont’ Sudo pvcreate /dev/sdc1 Sudo vgcreate vgname /dev/sdc1 Sudo lvcreate –l 100%FREE –n volname vgname Sudo mkfs –t ext4 /dev/vgname/volname Sudo mkdir mountpoint Sudo vi /etc/fstab (device to be mounted is /dev/vgname/volname) Sudo mount mountpoint

Storage Hardware Hard Disks Flash Memory Magnetic Tapes Optical Media SSD Hybrids Magnetic Tapes Optical Media

Hard Disk Failure A 2007 Google Labs study of 100,000 drives Drives 2+ years old average annual failure rate (AFR) of more than 6%! The overall pattern was a few months of infant mortality, a two-year honeymoon of annual failure rates of a few percent, and then a jump up to the 6%–8% AFR range. Overall, hard disks in the Google study had less than a 75% chance of surviving a five-year tour of duty.

Disk Geometry Platter - Stacked circular plates with magnetic coating. Sector – Smallest unit of storage on the disk which the system can access, and is typically 512 bytes. Track – Series of sectors, on the same physical platter of a disk, all of which can be accessed without moving the disk's read/write heads. Cylinder - A cylinder is the set of tracks, each of which is on a different platter and all of which can be accessed without moving the disk's read/write heads. logical partitions (or slices) - start cylinder of each partition plus the number of cylinders it contains.

Disk Geometry Platters rotate inside hard disk around spindle Disk head reads and writes data to sectors (similar to needle and record player) Faster the platter spins (rotation) the higher the bandwidth can be accessed from disk. Disk speeds 5,400 RPMs – 15,000 RPMs

Solid State Drives Flash memory – capable of parallel read and write(s) Using SATA 6Gb/s interface Typical internal laptop SSD 415MB/s (reads) 250MB/s (writes) Long term reliability ????? - YES $$$$ vs hard drives – prices coming down

Disks Interfaces IDE/ATA (Advanced Technology Attachment) PATA – Parallel ATA SATA - Serial ATA SCSI – “skuzzy” Fibre Channel 1-40 Gb/s USB – Universal Serial Bus Thunderbolt

PATA aka IDE Parallel ATA commonly known as IDE Traditional interface found in lower end workstations / PCs (becoming deprecated) Transfer rates up to 133 MB/s 18in. cable length

SATA Serial ATA - SATA1,2,3 6 Gb/s 1 meter cable length SATA3 backwards compatible to SATA1 Commonly found in today’s workstations and PCs

Small Computer System Interface (SCSI) Generic data pipe used by different types of devices (similar to USB) Daisy chain drives together Hot swappable SCSI-3 defines a family of standards including Firewire and Fibre Channel

Universal Serial Bus - USB 3.0 transfer rates up to 5Gbit/s (max theorectical) Data pipe used by different peripherals

Thunderbolt Formally known as LightPeak - developed by Intel Transfer rates up to 10 Gbits/s

The Software Side of Storage Partition – fixed sub-sized section of a storage device RAID - a redundant array of inexpensive/independent disks – combines multiple drives into a virtual disk. (more on RAID later) Volume groups and logical volumes - These systems aggregate physical devices to form pools of storage called volume groups. Then subdivide this pool into logical volumes.

Linux Filesystem Check filesystem “df –T” command Ext2: This is like UNIX file system. It has the concepts of blocks, inodes and directories. Ext3: It is ext2 filesystem enhanced with journalling capabilities. Journalling allows fast file system recovery. Ext4: Supports huge individual file size and overall file system size. Maximum individual file size can be from 16 GB to 16 TB Overall maximum ext4 file system size is 1 EB (exabyte). 1 EB = 1024 PB (petabyte). 1 PB = 1024 TB (terabyte). Directory can contain a maximum of 64,000 subdirectories (as opposed to 32,000 in ext3)

The Software Side of Storage

Storage Details Partitions – fixed size sub section of a storage device RAID array (redudant array of inexpensive disks) – combines multiple storage devices into one virtualized device Volume groups and logical volumes associated with logical volume managers (LVM) - physical devices for pools of storage called volume groups – volume groups can then be subdivided into logical volumes

The Software Side of Storage cont’ Filesystem - mediates between the raw bag of blocks presented by a partition, RAID array, or logical volume and the standard filesystem interface expected by programs: paths such as /var/spool/mail, Linux file types, Linux permissions, etc

Adding a Hard Drive to your system Physically connect the drive to the PC, system, RAID array, etc Hardware verification The BIOS of the PC can tell you if its connected correctly System log files Files located in /dev will update with the new drive information automatically Linux block device /dev/sda Partition /dev/sda1 USB are /dev/sdb Run command parted /dev/sdb print

Time to put on filesystem mkfs.ext4 Formats a disk to ext4 file system mkdir <new mountpoint> Mount <new filesystem drives> <new mountpoint>

LVM Logical Volume Management Essentially abstract dynamic disk partitioning. Groups individual storage devices into “volume groups.” The blocks in a volume group can then be allocated to “logical volumes,” which are represented by block device files and act like disk partitions.

Linux LVM A Linux LVM configuration proceeds in a few distinct phases: Creating (defining, really) and initializing physical volumes Adding the physical volumes to a volume group Creating logical volumes on the volume group

Format & Bad Block Management Low level format – identifies bad blocks, imperfections in the media that result in areas that cannot be reliably read or written. New drives come preformatted. No need to low level format new drives. Rm –rf does not remove the contents of a drive.

hdparm general way to interact with the firmware of SATA, IDE, and SAS hard disks. Among other things, hdparm can set drive power options, enable or disable noise hdparm [options] device

Disk Partitioning There are several options for partitioning a hard disk: fdisk Parted (****) gparted (GUI) An extended partition is the only kind of partition that can have multiple partitions inside. Think of it like a box that contains other boxes, the logical partitions. The extended partition can't store anything, it's just a holder for logical partitions. The extended partitions is a way to get around the fact you can only have four primary partitions on a drive. You can put lots of logical partitions inside it. hda is the whole drive hda1 is a primary partition hda2 is a primary partition hda4 is an extended partition hda5 is an logical partition hda6 is an logical partition You will never see hda4 mounted, just hda5 and hda6, in this case. Note that Linux numbers primary partitions 1-4, logical partitions start at 5 and up, even if there are less than 4 primary partitions.

RAID Redundant Array of Inexpensive Disks Best handled in software by the kernel (although dedicated hardware is available) RAID has two features – striping and replication Striping – spreads data out across multiple disks Replication: Mirroring – byte for byte copy of the data across one or more disks Parity – error correcting checksums of data blocks spread across multiple drives

RAID Levels Linear mode - concatenates the block addresses of multiple drives to create a single, larger virtual drive. (no redundancy or performance improvement) RAID 0 – used strictly to increase performance stripes data across multiple disks

RAID Level 1 RAID 1 Mirroring – write functions are duplicated across two or more drives simultaneously

RAID Level 1 + 0 & 0 + 1 RAID levels 1+0 and 0+1 are stripes of mirror sets or mirrors of stripe sets. The goal of both modes is to simultaneously obtain the performance of RAID 0 and the redundancy of RAID 1.

RAID Level 5 RAID 5 - stripes both data and parity information, adding redundancy while simultaneously improving read performance. RAID 5 is more efficient in its use of disk space than is RAID 1. If there are N drives in an array (at least three are required), N–1 of them can store data.

RAID Levels 6 RAID 6 is similar to RAID 5 with two parity disks. A RAID 6 array can withstand the complete failure of two drives without losing data.

Now what?? parted to set additional partitions and format drives (if needed) mkpart mdadm command Is used to configure your RAID array and levels