1. Chapter 8
Adding Disks

2. Loops Run instructions again, and again, and again…. 2 types of loops
Check some condition
If true, repeat
2 types of loops
for loop
while loop
First, the variable used in loop condition must be initialized, then execution of the loop begins.
A test (condition) is made at the beginning of each iteration.
The body of loop ends with a statement that modifies the value of the test (condition) variable.

3. For Loop Syntax: for ( init; condition; increment ) { statement(s); }
Example
#!/usr/bin/perl
# for loop execution
for( $a = 10; $a < 20; $a = $a + 1 ) {
print "value of a: $a\n";

4. While Loop Syntax: while(condition) { statement(s); } #!/usr/bin/perl
# while loop execution
while( $a < 20 ) {
print "Value of a: $a\n";
$a = $a + 1;

5. Foreach Loop
The foreach loop iterates over a list value and sets the control variable (var) to be each element of the list in turn:
Syntax:
The syntax of a foreach loop in Perl programming language is:
foreach var (list) { ... }
#!/usr/local/bin/perl
@list = (2, 20, 30, 40, 50);
# foreach loop execution
foreach $a {
print "value of a: $a\n"; }

6. Lets just quickly see how to add a disk
Attach hard drive and reboot
No RAID (will cover later)
One logical volume (will cover later)
Identify and format the right drive
Linux recipe
Type “sudo fdisk –l” to list the system disks and identify a new drive
Use a partitioning utility to create partition table
Put all drives spare in one partition of unspecified type.

7. Lets just quickly see how to add a disk cont’
Sudo pvcreate /dev/sdc1
Sudo vgcreate vgname /dev/sdc1
Sudo lvcreate –l 100%FREE –n volname vgname
Sudo mkfs –t ext4 /dev/vgname/volname
Sudo mkdir mountpoint
Sudo vi /etc/fstab (device to be mounted is /dev/vgname/volname)
Sudo mount mountpoint

8. Storage Hardware Hard Disks Flash Memory Magnetic Tapes Optical Media
SSD
Hybrids
Magnetic Tapes
Optical Media

10. Hard Disk Failure A 2007 Google Labs study of 100,000 drives
Drives 2+ years old average annual failure rate (AFR) of more than 6%!
The overall pattern was a few months of infant mortality, a two-year honeymoon of annual failure rates of a few percent, and then a jump up to the 6%–8% AFR range.
Overall, hard disks in the Google study had less than a 75% chance of surviving a five-year tour of duty.

11. Disk Geometry Platter - Stacked circular plates with magnetic coating.
Sector – Smallest unit of storage on the disk which the system can access, and is typically 512 bytes.
Track – Series of sectors, on the same physical platter of a disk, all of which can be accessed without moving the disk's read/write heads.
Cylinder - A cylinder is the set of tracks, each of which is on a different platter and all of which can be accessed without moving the disk's read/write heads.
logical partitions (or slices) - start cylinder of each partition plus the number of cylinders it contains.

12. Disk Geometry Platters rotate inside hard disk around spindle
Disk head reads and writes data to sectors (similar to needle and record player)
Faster the platter spins (rotation) the higher the bandwidth can be accessed from disk.
Disk speeds 5,400 RPMs – 15,000 RPMs

13. Solid State Drives
Flash memory – capable of parallel read and write(s)
Using SATA 6Gb/s interface Typical internal laptop SSD 415MB/s (reads) 250MB/s (writes)
Long term reliability ????? - YES
$$$$ vs hard drives – prices coming down

14. Disks Interfaces
IDE/ATA (Advanced Technology Attachment) PATA – Parallel ATA
SATA - Serial ATA
SCSI – “skuzzy”
Fibre Channel 1-40 Gb/s
USB – Universal Serial Bus
Thunderbolt

15. PATA aka IDE Parallel ATA commonly known as IDE
Traditional interface found in lower end workstations / PCs (becoming deprecated)
Transfer rates up to 133 MB/s
18in. cable length

16. SATA Serial ATA - SATA1,2,3 6 Gb/s 1 meter cable length
SATA3 backwards compatible to SATA1
Commonly found in today’s workstations and PCs

17. Small Computer System Interface (SCSI)
Generic data pipe used by different types of devices (similar to USB)
Daisy chain drives together
Hot swappable
SCSI-3 defines a family of standards including Firewire and Fibre Channel

18. Universal Serial Bus - USB
3.0 transfer rates up to
5Gbit/s (max theorectical)
Data pipe used by different peripherals

19. Thunderbolt Formally known as LightPeak - developed by Intel
Transfer rates up to 10 Gbits/s

20. The Software Side of Storage
Partition – fixed sub-sized section of a storage device
RAID - a redundant array of inexpensive/independent disks – combines multiple drives into a virtual disk. (more on RAID later)
Volume groups and logical volumes - These systems aggregate physical devices to form pools of storage called volume groups. Then subdivide this pool into logical volumes.

21. Linux Filesystem Check filesystem “df –T” command
Ext2: This is like UNIX file system. It has the concepts of blocks, inodes and directories.
Ext3: It is ext2 filesystem enhanced with journalling capabilities. Journalling allows fast file system recovery.
Ext4: Supports huge individual file size and overall file system size. Maximum individual file size can be from 16 GB to 16 TB Overall maximum ext4 file system size is 1 EB (exabyte). 1 EB = 1024 PB (petabyte). 1 PB = 1024 TB (terabyte). Directory can contain a maximum of 64,000 subdirectories (as opposed to 32,000 in ext3)

22. The Software Side of Storage

23. Storage Details
Partitions – fixed size sub section of a storage device
RAID array (redudant array of inexpensive disks) – combines multiple storage devices into one virtualized device
Volume groups and logical volumes associated with logical volume managers (LVM)
- physical devices for pools of storage called volume groups – volume groups can then be subdivided into logical volumes

24. The Software Side of Storage cont’
Filesystem - mediates between the raw bag of blocks presented by a partition, RAID array, or logical volume and the standard filesystem interface expected by programs: paths such as /var/spool/mail, Linux file types, Linux permissions, etc

25. Adding a Hard Drive to your system
Physically connect the drive to the PC, system, RAID array, etc
Hardware verification
The BIOS of the PC can tell you if its connected correctly
System log files
Files located in /dev will update with the new drive information automatically
Linux block device /dev/sda Partition /dev/sda1
USB are /dev/sdb
Run command parted /dev/sdb print

26. Time to put on filesystem
mkfs.ext4 Formats a disk to ext4 file system
mkdir <new mountpoint>
Mount <new filesystem drives> <new mountpoint>

28. LVM Logical Volume Management
Essentially abstract dynamic disk partitioning.
Groups individual storage devices into “volume groups.”
The blocks in a volume group can then be allocated to “logical volumes,” which are represented by block device files and act like disk partitions.

29. Linux LVM A Linux LVM configuration proceeds in a few distinct phases:
Creating (defining, really) and initializing physical volumes
Adding the physical volumes to a volume group
Creating logical volumes on the volume group

30. Format & Bad Block Management
Low level format – identifies bad blocks, imperfections in the media that result in areas that cannot be reliably read or written.
New drives come preformatted.
No need to low level format new drives.
Rm –rf does not remove the contents of a drive.

31. hdparm
general way to interact with the firmware of SATA, IDE, and SAS hard disks.
Among other things, hdparm can set drive power options, enable or disable noise
hdparm [options] device

32. Disk Partitioning
There are several options for partitioning a hard disk:
fdisk
Parted (****)
gparted (GUI)
An extended partition is the only kind of partition that can have multiple partitions inside. Think of it like a box that contains other boxes, the logical partitions. The extended partition can't store anything, it's just a holder for logical partitions. The extended partitions is a way to get around the fact you can only have four primary partitions on a drive. You can put lots of logical partitions inside it. hda is the whole drive hda1 is a primary partition hda2 is a primary partition hda4 is an extended partition hda5 is an logical partition hda6 is an logical partition You will never see hda4 mounted, just hda5 and hda6, in this case. Note that Linux numbers primary partitions 1-4, logical partitions start at 5 and up, even if there are less than 4 primary partitions.

33. RAID Redundant Array of Inexpensive Disks
Best handled in software by the kernel (although dedicated hardware is available)
RAID has two features – striping and replication
Striping – spreads data out across multiple disks
Replication:
Mirroring – byte for byte copy of the data across one or more disks
Parity – error correcting checksums of data blocks spread across multiple drives

34. RAID Levels
Linear mode - concatenates the block addresses of multiple drives to create a single, larger virtual drive. (no redundancy or performance improvement)
RAID 0 – used strictly to increase performance stripes data across multiple disks

35. RAID Level 1
RAID 1
Mirroring – write functions are duplicated across two or more drives simultaneously

36. RAID Level & 0 + 1
RAID levels 1+0 and 0+1 are stripes of mirror sets or mirrors of stripe sets.
The goal of both modes is to simultaneously obtain the performance of RAID 0 and the redundancy of RAID 1.

37. RAID Level 5
RAID 5 - stripes both data and parity information, adding redundancy while simultaneously improving read performance.
RAID 5 is more efficient in its use of disk space than is RAID 1.
If there are N drives in an array (at least three are required), N–1 of them can store data.

38. RAID Levels 6
RAID 6 is similar to RAID 5 with two parity disks. A RAID 6 array can withstand the complete failure of two drives without losing data.

39. Now what??
parted to set additional partitions and format drives (if needed)
mkpart
mdadm command
Is used to configure your RAID array and levels