CS-3013 & CS-502, Summer 2006 More on File Systems1 More on Disks and File Systems CS-3013 & CS-502 Operating Systems.

Slides:



Advertisements
Similar presentations
A CASE FOR REDUNDANT ARRAYS OF INEXPENSIVE DISKS (RAID) D. A. Patterson, G. A. Gibson, R. H. Katz University of California, Berkeley.
Advertisements

CS 346 – April 4 Mass storage –Disk formatting –Managing swap space –RAID Commitment –Please finish chapter 12.
RAID (Redundant Arrays of Independent Disks). Disk organization technique that manages a large number of disks, providing a view of a single disk of High.
RAID Oh yes Whats RAID? Redundant Array (of) Independent Disks. A scheme involving multiple disks which replicates data across multiple drives. Methods.
RAID Redundant Array of Independent Disks
Enhanced Availability With RAID CC5493/7493. RAID Redundant Array of Independent Disks RAID is implemented to improve: –IO throughput (speed) and –Availability.
2P13 Week 11. A+ Guide to Managing and Maintaining your PC, 6e2 RAID Controllers Redundant Array of Independent (or Inexpensive) Disks Level 0 -- Striped.
Lecture 36: Chapter 6 Today’s topic –RAID 1. RAID Redundant Array of Inexpensive (Independent) Disks –Use multiple smaller disks (c.f. one large disk)
REDUNDANT ARRAY OF INEXPENSIVE DISCS RAID. What is RAID ? RAID is an acronym for Redundant Array of Independent Drives (or Disks), also known as Redundant.
Chapter 11: File System Implementation
1 Recap (RAID and Storage Architectures). 2 RAID To increase the availability and the performance (bandwidth) of a storage system, instead of a single.
Other Disk Details. 2 Disk Formatting After manufacturing disk has no information –Is stack of platters coated with magnetizable metal oxide Before use,
More on Disks and File Systems 1 CS502 Spring 2006 More on Disks and File Systems CS-502 Operating Systems Spring 2006.
CS 333 Introduction to Operating Systems Class 18 - File System Performance Jonathan Walpole Computer Science Portland State University.
Cse Feb-001 CSE 451 Section February 24, 2000 Project 3 – VM.
Disks, RAIDs, and Stable Storage CS-3013 A-term Disks, RAIDs, and Stable Storage CS-3013, Operating Systems A-term 2009 (Slides include materials.
More on FilesCS-4513, D-Term More on File Systems CS-4513 Distributed Computing Systems (Slides include materials from Operating System Concepts,
More on Disks and FilesCS-502 Fall More on Disks and File Systems CS-502 Operating Systems Fall 2006 (Slides include materials from Operating System.
I/O Systems and Storage Systems May 22, 2000 Instructor: Gary Kimura.
RAID Systems CS Introduction to Operating Systems.
CSE 451: Operating Systems Winter 2010 Module 13 Redundant Arrays of Inexpensive Disks (RAID) and OS structure Mark Zbikowski Gary Kimura.
The Design and Implementation of a Log-Structured File System Presented by Carl Yao.
Storage System: RAID Questions answered in this lecture: What is RAID? How does one trade-off between: performance, capacity, and reliability? What is.
Transactions and Reliability. File system components Disk management Naming Reliability  What are the reliability issues in file systems? Security.
ICOM 6005 – Database Management Systems Design Dr. Manuel Rodríguez-Martínez Electrical and Computer Engineering Department Lecture 6 – RAID ©Manuel Rodriguez.
Storage Systems CSE 598d, Spring 2007 Lecture 5: Redundant Arrays of Inexpensive Disks Feb 8, 2007.
Chapter 6 RAID. Chapter 6 — Storage and Other I/O Topics — 2 RAID Redundant Array of Inexpensive (Independent) Disks Use multiple smaller disks (c.f.
More on File SystemsCS-502 Fall More on File Systems CS-502, Operating Systems Fall 2007 (Slides include materials from Operating System Concepts,
CS 352 : Computer Organization and Design University of Wisconsin-Eau Claire Dan Ernst Storage Systems.
1 Recitation 8 Disk & File System. 2 Disk Scheduling Disks are at least four orders of magnitude slower than main memory –The performance of disk I/O.
CSE 321b Computer Organization (2) تنظيم الحاسب (2) 3 rd year, Computer Engineering Winter 2015 Lecture #4 Dr. Hazem Ibrahim Shehata Dept. of Computer.
Disk Access. DISK STRUCTURE Sector: Smallest unit of data transfer from/to disk; 512B 2/4/8 adjacent sectors transferred together: Blocks Read/write heads.
Lecture 9 of Advanced Databases Storage and File Structure (Part II) Instructor: Mr.Ahmed Al Astal.
Redundant Array of Inexpensive Disks aka Redundant Array of Independent Disks (RAID) Modified from CCT slides.
CE Operating Systems Lecture 20 Disk I/O. Overview of lecture In this lecture we will look at: Disk Structure Disk Scheduling Disk Management Swap-Space.
1 File Systems: Consistency Issues. 2 File Systems: Consistency Issues File systems maintains many data structures  Free list/bit vector  Directories.
CS 153 Design of Operating Systems Spring 2015 Lecture 22: File system optimizations.
Free Space Management.
Page 111/15/2015 CSE 30341: Operating Systems Principles Chapter 11: File System Implementation  Overview  Allocation methods: Contiguous, Linked, Indexed,
Silberschatz, Galvin and Gagne  Operating System Concepts Chapter 12: File System Implementation File System Structure File System Implementation.
File System Implementation
CS 153 Design of Operating Systems Spring 2015 Lecture 21: File Systems.
The concept of RAID in Databases By Junaid Ali Siddiqui.
12.1 Silberschatz, Galvin and Gagne ©2003 Operating System Concepts with Java Chapter 12: File System Implementation Chapter 12: File System Implementation.
RAID Disk Arrays Hank Levy. 212/5/2015 Basic Problems Disks are improving, but much less fast than CPUs We can use multiple disks for improving performance.
CS333 Intro to Operating Systems Jonathan Walpole.
11.1 Silberschatz, Galvin and Gagne ©2005 Operating System Principles 11.5 Free-Space Management Bit vector (n blocks) … 012n-1 bit[i] =  1  block[i]
Disks, RAIDs, and Stable Storage CS-502 (EMC) Fall Disks, RAIDs, and Stable Storage CS-502, Operating Systems Fall 2009 (EMC) (Slides include materials.
Transactions and Reliability Andy Wang Operating Systems COP 4610 / CGS 5765.
Storage and File structure COP 4720 Lecture 20 Lecture Notes.
Lecture 20 FSCK & Journaling. FFS Review A few contributions: hybrid block size groups smart allocation.
Hands-On Microsoft Windows Server 2008 Chapter 7 Configuring and Managing Data Storage.
Lecture Topics: 11/22 HW 7 File systems –block allocation Unix and NT –disk scheduling –file caches –RAID.
I/O Errors 1 Computer Organization II © McQuain RAID Redundant Array of Inexpensive (Independent) Disks – Use multiple smaller disks (c.f.
CS Introduction to Operating Systems
Jonathan Walpole Computer Science Portland State University
Transactions and Reliability
Disks and RAID.
Operating System I/O System Monday, August 11, 2008.
Journaling File Systems
Disks.
ICOM 6005 – Database Management Systems Design
Overview Continuation from Monday (File system implementation)
Outline Allocation Free space management Memory mapped files
Printed on Monday, December 31, 2018 at 2:03 PM.
UNIT IV RAID.
Overview: File system implementation (cont)
File-System Structure
CSE 451: Operating Systems Winter 2007 Module 18 Redundant Arrays of Inexpensive Disks (RAID) Ed Lazowska Allen Center 570.
CSE 451: Operating Systems Winter 2006 Module 18 Redundant Arrays of Inexpensive Disks (RAID) Ed Lazowska Allen Center 570.
Presentation transcript:

CS-3013 & CS-502, Summer 2006 More on File Systems1 More on Disks and File Systems CS-3013 & CS-502 Operating Systems

CS-3013 & CS-502, Summer 2006 More on File Systems2 Additional Topics Mapping files to VM RAID – Redundant Array of Inexpensive Disks Stable Storage Log Structured File Systems

CS-3013 & CS-502, Summer 2006 More on File Systems3 Reading Assignment(s) RAID – Tanenbaum §5.4.1 Stable Storage – §5.4.5 Log-structured File System – §6.3.8 These topics will be included on exam next week regardless of whether we complete them this evening

CS-3013 & CS-502, Summer 2006 More on File Systems4 Mapping files to VM Instead of “reading” from disk into virtual memory, why not simply use file as the swapping storage for certain VM pages? Called mapping Page tables in kernel point to disk blocks of the file

CS-3013 & CS-502, Summer 2006 More on File Systems5 Memory-Mapped Files Memory-mapped file I/O allows file I/O to be treated as routine memory access by mapping a disk block to a page in memory A file is initially read using demand paging. A page-sized portion of the file is read from the file system into a physical page. Subsequent reads/writes to/from the file are treated as ordinary memory accesses. Simplifies file access by allowing application to simple access memory rather than be forced to use read() & write() calls to file system.

CS-3013 & CS-502, Summer 2006 More on File Systems6 Memory-Mapped Files (continued) A tantalizingly attractive notion, but … Cannot use C/C++ pointers within mapped data structure Corrupted data structures more likely to persist in file Don’t really save anything in terms of Programming energy Thought processes Storage space & efficiency

CS-3013 & CS-502, Summer 2006 More on File Systems7 Memory-Mapped Files (continued) Nevertheless, the idea has its uses 1.Simpler implementation of file operations –read(), write() are memory-to-memory operations –seek() is simply changing a pointer, etc… –Called memory-mapped I/O 2.Shared Virtual Memory among processes

CS-3013 & CS-502, Summer 2006 More on File Systems8 Shared Virtual Memory

CS-3013 & CS-502, Summer 2006 More on File Systems9 Shared Virtual Memory (continued) Supported in –Windows XP –Apollo DOMAIN –Linux?? Synchronization is the responsibility of the sharing applications –OS retains no knowledge

CS-3013 & CS-502, Summer 2006 More on File Systems10 Questions?

CS-3013 & CS-502, Summer 2006 More on File Systems11 Problem Question:– –If mean time to failure of a disk drive is 100,000 hours, –and if your system has 100 identical disks, –what is mean time between drive replacement? Answer:– –1000 hours (i.e., days  6 weeks) I.e.:– –You lose 1% of your data every 6 weeks! But don’t worry – you can restore most of it from backup!

CS-3013 & CS-502, Summer 2006 More on File Systems12 Can we do better? Yes, mirrored –Write every block twice, on two separate disks –Mean time between simultaneous failure of both disks is 57,000 years Can we do even better? –E.g., use fewer extra disks? –E.g., get more performance?

CS-3013 & CS-502, Summer 2006 More on File Systems13 RAID – Redundant Array of Inexpensive Disks Distribute a file system intelligently across multiple disks to –Maintain high reliability and availability –Enable fast recovery from failure –Increase performance

CS-3013 & CS-502, Summer 2006 More on File Systems14 “Levels” of RAID Level 0 – non-redundant striping of blocks across disk Level 1 – simple mirroring Level 2 – striping of bytes or bits with ECC Level 3 – Level 2 with parity, not ECC Level 4 – Level 0 with parity block Level 5 – Level 4 with distributed parity blocks

CS-3013 & CS-502, Summer 2006 More on File Systems15 RAID Level 0 – Simple Striping Each stripe is one or a group of contiguous blocks Block/group i is on disk (i mod n) Advantage –Read/write n blocks in parallel; n times bandwidth Disadvantage –No redundancy at all. System MBTF is 1/n disk MBTF! stripe 8 stripe 4 stripe 0 stripe 9 stripe 5 stripe 1 stripe 10 stripe 6 stripe 2 stripe 11 stripe 7 stripe 3

CS-3013 & CS-502, Summer 2006 More on File Systems16 RAID Level 1– Striping and Mirroring Each stripe is written twice Two separate, identical disks Block/group i is on disks (i mod 2n) & (i+n mod 2n) Advantages –Read/write n blocks in parallel; n times bandwidth –Redundancy: System MBTF = (Disk MBTF) 2 at twice the cost –Failed disk can be replaced by copying Disadvantage –A lot of extra disks for much more reliability than we need stripe 8 stripe 4 stripe 0 stripe 9 stripe 5 stripe 1 stripe 10 stripe 6 stripe 2 stripe 11 stripe 7 stripe 3 stripe 8 stripe 4 stripe 0 stripe 9 stripe 5 stripe 1 stripe 10 stripe 6 stripe 2 stripe 11 stripe 7 stripe 3

CS-3013 & CS-502, Summer 2006 More on File Systems17 RAID Levels 2 & 3 Bit- or byte-level striping Requires synchronized disks Highly impractical Requires fancy electronics For ECC calculations Not used; academic interest only See Silbershatz, § (pp )

CS-3013 & CS-502, Summer 2006 More on File Systems18 Observation When a disk or stripe is read incorrectly, we know which one failed! Conclusion: –A simple parity disk can provide very high reliability (unlike simple parity in memory)

CS-3013 & CS-502, Summer 2006 More on File Systems19 RAID Level 4 – Parity Disk parity 0-3 = stripe 0 xor stripe 1 xor stripe 2 xor stripe 3 n stripes plus parity are written/read in parallel If any disk/stripe fails, it can be reconstructed from others –E.g., stripe 1 = stripe 0 xor stripe 2 xor stripe 3 xor parity 0-3 Advantages –n times read bandwidth –System MBTF = (Disk MBTF) 2 at 1/n additional cost –Failed disk can be reconstructed “on-the-fly” (hot swap) –Hot expansion: simply add n + 1 disks all initialized to zeros However –Writing requires read-modify-write of parity stripe  only 1x write bandwidth. stripe 8 stripe 4 stripe 0 stripe 9 stripe 5 stripe 1 stripe 10 stripe 6 stripe 2 stripe 11 stripe 7 stripe 3 parity 8-11 parity 4-7 parity 0-3

CS-3013 & CS-502, Summer 2006 More on File Systems20 RAID Level 5 – Distributed Parity Parity calculation is same as RAID Level 4 Advantages & Disadvantages –Same as RAID Level 4 Additional advantage: avoids beating up on parity disk Writing individual stripes (RAID 4 & 5) –Read existing stripe and existing parity –Recompute parity –Write new stripe and new parity stripe 12 stripe 8 stripe 4 stripe 0 parity stripe 9 stripe 5 stripe 1 stripe 13 parity 8-11 stripe 6 stripe 2 stripe 14 stripe 10 parity 4-7 stripe 3 stripe 15 stripe 11 stripe 7 parity 0-3

CS-3013 & CS-502, Summer 2006 More on File Systems21 RAID 4 & 5 Very popular in data centers –Corporate and academic servers Built-in support in Windows XP and other systems –Connect a group of disks to fast SCSI port (320 MB/sec bandwidth) –OS RAID support does the rest!

CS-3013 & CS-502, Summer 2006 More on File Systems22 New Topic Problem – how to protect against disk write operations that don’t complete –Power or CPU failure in the middle of a block –Related series of writes interrupted in middle Examples: –Database update of charge and credit –RAID 1, 4, 5 failure between redundant writes

CS-3013 & CS-502, Summer 2006 More on File Systems23 Solution (part 1) – Stable Storage Write everything twice (separate disks) Be sure 1 st write does not invalidate previous 2 nd copy RAID 1 is okay; RAID 4/5 not okay! Read blocks back to validate; then report completion Reading both copies If 1 st copy okay, use it – i.e., newest value If 2 nd copy different, update it with 1 st copy If 1 st copy error; use 2 nd copy – i.e., old value

CS-3013 & CS-502, Summer 2006 More on File Systems24 Stable Storage (continued) Crash recovery Scan disks, compare corresponding blocks If one is bad, replace with good one If both good but different, replace 2 nd with 1 st copy Result:– If 1 st block is good, it contains latest value If not, 2 nd block still contains previous value An abstraction of an atomic disk write of a single block Uninterruptible by power failure, etc.

CS-3013 & CS-502, Summer 2006 More on File Systems25 What about more complex disk operations? E.g., File create operation involves Allocating free blocks Constructing and writing i-node –Possibly multiple i-node blocks Reading and updating directory What if system crashes with the sequence only partly completed? Answer: inconsistent data structures on disk

CS-3013 & CS-502, Summer 2006 More on File Systems26 Solution (Part 2) – Log-Structured File System Make changes to cached copies in memory Collect together all changed blocks Write to log file A circular buffer on disk Fast, contiguous write Update log file pointer in stable storage Offline: Play back log file to actually update directories, i-nodes, free list, etc. Update playback pointer in stable storage

CS-3013 & CS-502, Summer 2006 More on File Systems27 Transaction Data Base Systems Similar techniques –Every transaction is recorded in log before recording on disk –Stable storage techniques for managing log pointers –One log exist is confirmed, disk can be updated in place –After crash, replay log to redo disk operations

CS-3013 & CS-502, Summer 2006 More on File Systems28 Unix LFS Tanenbaum, §6.3.8, pp Everything is written to log i-nodes point to updated blocks in log i-node cache in memory updated whenever i-node is written Cleaner daemon follows behind to compact log Advantages: –LFS is always consistent –LFS performance Much better than Unix FS for small writes At least as good for reads and large writes