Download presentation
Presentation is loading. Please wait.
1
Lecture 17: Storage Systems
COSC6376 Cloud Computing Lecture 17: Storage Systems Instructor: Weidong Shi (Larry), PhD Computer Science Department University of Houston
2
Storage Area Network and Storage Virtualization
3
Introduction To Storage: Overview of the storage subsystem
Storage types Single Disk Drive Storage Array SCSI device DAS NAS SAN iSCSI Introduction To Storage: Overview of the storage subsystem
4
Direct attached storage (internal)
Computer System CPU Memory Bus I/O - RAID Controller Disk Drives Introduction To Storage: Overview of the storage subsystem
5
Direct attached storage (internal)
12345 John Smith 1424 Main Street Computer System CPU Memory Bus I/O - RAID Controller Disk Drives Data Introduction To Storage: Overview of the storage subsystem
6
Direct attached storage (internal)
12345 John Smith 1424 Main Street Computer System CPU Memory Bus I/O - RAID Controller Disk Drives Introduction To Storage: Overview of the storage subsystem
7
DAS w/internal controller and external storage
12345 John Smith 1424 Main Street CPU Memory Bus I/O - RAID Controller Computer System Disk Drives Disk Enclosure Introduction To Storage: Overview of the storage subsystem
8
Attach large number of disks
9/18/ :34 PM Attach large number of disks 4-Processor AMD Dual Core Opteron can handle 2.5 GB/s or more sequential read and write data traffic Each controller connects 8 point-to-point SATA Drives – Average 400 MB/s/card PCI-X SATA Controller 400 GB PCI-X SATA Controller 400 GB 16 TB Raw Disk Space 12 TB RAID 5 8 TB Mirrored PCI-X SATA Controller 400 GB 400 GB PCI-X SATA Controller 400 GB PCI-X SATA Controller © 2005 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.
9
Backblaze
10
Backblaze Storage Pods: 67 terabyte 4U servers
11
Comparing internal and external storage
RAID controllers and disk drives are internal to the server SCSI, ATA, or SATA protocol between controller and disks Server Storage Server Disk Drives RAID Controller RAID Controller RAID controller is internal SCSI or SATA protocol between controller and disks Disk drives are external Storage Internal Storage Disk Drives SCSI Bus w/ external storage Introduction To Storage: Overview of the storage subsystem
12
Direct attached storage
Direct Attached Storage (DAS) Storage is captive ‘behind’ the server Server CPU must handle user I/O requests, but also: User-database inquiries User file/print serving Data-integrity checking Communication with other devices Data access is file system and platform dependent Costly to scale; complex to manage Clients IP Network Servers Win Linux Win Linux Unix FC FC SCSI Direct-Attached Storage (DAS)
13
DAS w/external controller and external storage
Storage System 12345 John Smith 1424 Main Street Computer System CPU Memory Bus HBA Disk Drives Disk Enclosure RAID Controller Introduction To Storage: Overview of the storage subsystem
14
Introduction To Storage: Overview of the storage subsystem
DAS over Fibre channel Server HBA HBA is internal Fibre Channel protocol between HBAs and external RAID controller Storage Disk Drives Disk drives and RAID controller are external RAID Controller External SAN Array Introduction To Storage: Overview of the storage subsystem
15
Introduction To Storage: Overview of the storage subsystem
I/O Transfer RAID Controller Contains the “smarts” Determines how the data will be written (striping, mirroring, RAID 10, RAID 5, etc.) Host Bus Adapter (HBA) Simply transfers the data to the RAID controller. Doesn’t do any RAID or striping calculations. “Dumb” for speed. Required for external storage. Introduction To Storage: Overview of the storage subsystem
16
SCSI Small Computer System Interface
9/18/ :34 PM SCSI Small Computer System Interface © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
17
Terminology Initiator: A SCSI device that requests an operation to be
performed by another SCSI device. Host Adapter: A Controller Board that translates host I/O requests into SCSI requests. Target: A SCSI device that performs an operation as requested by an initiator. Logical Unit: A Physical or Virtual device addressable through a target. LUN: Logical Unit Number. An encoded three-bit identifier for the logical unit.
18
single initiator, single target
SCSI configuration single initiator, single target LUN0 Initiator Target LUN1 SCSI Bus LUN2
19
Single initiator, multiple targets
SCSI configuration Single initiator, multiple targets T Target 0 Initiator 7 Target 1 SCSI Bus Target n T SCSI Bus Terminator
20
Multiple initiators, multiple targets
SCSI configuration Multiple initiators, multiple targets T Target 0 Initiator 7 Target 1 SCSI Bus Initiator 6 Target n T
21
The SCSI I/O Channel SCSI is the dominant protocol used to communicate between servers and storage devices in open system SCSI I/O channel is a half-duplex pipe for SCSI CDBs and data Parallel bus evolution Bus width: 8, 16 bits Bus speed: 5–80 Mhz Throughput: 5–320 MBps Devices/bus: 2–16 devices Cable length: 1.5m–25m A network approach can scale the I/O channel in many areas (length, devices, speed) Applications Raw File System Block Device SCSI Generic TCP/IP Stack NIC Driver Adapter Driver Ethernet NIC SCSI Adapter Half-Duplex SCSI I/O Channel SCSI SCSI Initiator Ethernet SCSI CDB: SCSI Command Descriptor Block Used to Relay SCSI Commands, Parameters, and Status between SCSI Initiators and SCSI Targets; Typically 6, 10, or 12 Byte Block SCSI Target IP Storage Networking FCIP/iSCSI. Steve Tegeler
22
Networking the I/O Channel
Same SCSI protocol (SCSI-3) carried over a network transport layer via serial implementation Transport must not jeopardize SCSI payload (security, integrity, latency) Two primary transports to choose from today: Fibre Channel and IP A networked I/O channel allows for multiple improvements: Distance limitations greatly increased High number of addressable devices Host System SCSI Initiator Channel Controller Networked I/O Channel Network Target and LUNs IP Storage Networking FCIP/iSCSI. Steve Tegeler
23
Fibre channel networking
Very common method for networking SCSI Fibre Channel provides high-speed transport for SCSI payload Fibre Channel SAN overcomes many shortcomings of DAS including: Addressing for up to 16-million nodes (24 bits) Loop (shared) and Fabric (switched) transport Speeds of 100 or 200 MBps (1 or 2 Gbps) Distance of up to 10km (without extenders) Support for multiple protocols Combines best attributes of a channel and a network Host System SCSI Initiator Fibre Channel HBA Fibre Channel Fabric Target IP Storage Networking FCIP/iSCSI. Steve Tegeler
24
IP: an alternate I/O transport
Host System Viable transport for I/O traffic Similar characteristics to Fibre Channel: Addressing for close to 4 billion nodes (IPv4) Primarily a switched transport (with routing) Ethernet speeds of 1/10 Gbps or various WAN speeds Support for multiple high-level protocols Cost and manageability advantages with IP IP knowledge base widespread in industry SCSI Initiator IP “Channel Adapter” IP Network Target IP Storage Networking FCIP/iSCSI. Steve Tegeler
25
9/18/ :34 PM Fibre Channel © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
26
Fibre channel: what is it?
Fibre Channel is a network protocol implemented specifically for dedicated storage networks Fibre Channel utilizes specialized Switches Host Bus Adapters RAID controllers Cables Introduction To Storage: Overview of the storage subsystem
27
Fibre channel components
Server A Server B Server C HBA HBA HBA HBA HBA HBA switch switch Servers Host Bus Adapters Cables Fiber optic or copper Fibre Channel Switches Two switches for redundancy Fibre Channel Storage Array Two RAID Controllers for redundancy 4–100+ disk drives per array A true storage network Multiple servers Multiple switches Multiple Storage Arrays Disk Drives RAID Controller RAID Controller FC Storage Array Introduction To Storage: Overview of the storage subsystem
28
What is fibre channel? A high-speed transmission technology used as a peripheral channel or network backbone. FC1: Over 1G bits per second FC2: Over 2G bits per second It supports several common transport protocols like Internet Protocol (IP) and SCSI. It operates over copper and fiber optic cables at distances of up to 10 Kilometers. Up to 127 devices (SCSI: 15) Up to 10 km of cabling (3-15 ft. for SCSI) Physical interconnect can be copper or fiber optic It is supported by many suppliers.
29
Fibre channel – (continued)
How does it work? Serial interface Data is transferred across a single piece of medium at the fastest speed supported No complex signaling required Hot-pluggable Devices can be removed or added at will with no ill effects to data communications Provides a data link layer above the physical interconnect, analogous to Ethernet Sophisticated error detection at the frame level Data is checked and resent if necessary Introduction To Storage: Overview of the storage subsystem
30
Current standards: Performance from 266 megabits/second to over 4 gigabits/second High-bandwidth utilization with distance insensitivity. Support for multiple cost/performance levels, from small systems to super computers Ability to carry multiple existing interface command sets, including Internet Protocol (IP), SCSI, and audio and video
31
Fibre channel – frame dissection
Up to 2048 byte payload 4 byte checksum for each frame Introduction To Storage: Overview of the storage subsystem
32
Introduction To Storage: Overview of the storage subsystem
Fibre channel What’s with the funny name? Some background history required Originally developed to only support fiber optic cabling When copper cabling support was added, ISO decided not to rename the technology ISO changed to the French spelling to reduce association with fiber optics only medium Introduction To Storage: Overview of the storage subsystem
33
Fibre channel interface layers
RAID Controller Disk Drive Fibre Channel Device Driver SCSI Protocol Fibre Channel Fiber Optic or Copper Cabling Introduction To Storage: Overview of the storage subsystem
34
Introduction To Storage: Overview of the storage subsystem
SCSI vs. Fibre Channel Used with SAN Lots of built-in redundancy with connections Redundant network HBA is fibre channel hardware Standards: FC1: 100 MB/sec FC2: 200 MB/sec Provides a data link layer above the physical interconnect Analogous to Ethernet FC is a network of devices It can be media independent- copper or fibre optic Fibre Channel limitations: Cable length: Up to 10 kilometers (more a limitation of cable than FC itself) Up to 127 devices # of disk drives Interface for internal storage to external disks Potential down time w/ SCSI Single bus RAID controller is SCSI hardware Standards: Ultra2 (80 MB/sec) Ultra 160 (160 MB/sec) Ultra 320 (320 MB/sec) Media specific (copper only) SCSI Limitations: Cables can’t be any longer than 3 feet for single ended; 15 feet for LVD (low voltage differential) No more than 15 devices on a SCSI bus # of disk drives Introduction To Storage: Overview of the storage subsystem
35
9/18/ :34 PM iSCSI © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
36
Introduction To Storage: Overview of the storage subsystem
iSCSI: what is it? An alternate form of networked storage Utilizes a TCP/IP network Encapsulates native SCSI commands in TCP/IP packets Supported in Windows Server and Linux TCP/IP Offload Engines (TOEs) on NICs speed up packet encapsulation Introduction To Storage: Overview of the storage subsystem
37
Public or Private Ethernet network
iSCSI storage Public or Private Ethernet network iSCSI Storage NIC or iSCSI HBA Server NIC or iSCSI HBA SCSI commands are encapsulated in TCP/IP packets Disk Drives Server has a Network Interface Card or iSCSI HBA iSCSI HBAs use TCP/IP Offload Engine (TOE) RAID Controller Introduction To Storage: Overview of the storage subsystem
38
IP storage networking IP storage networking provides solution to carry storage traffic within IP Uses TCP: a reliable transport for delivery Applicable to local data center and long-haul applications Two primary protocols: iSCSI—Internet-SCSI—used to transport SCSI CDBs and data within TCP/IP connections FCIP—Fibre-Channel-over-IP—used to transport Fibre Channel frames within TCP/IP connections—any FC frame—not just SCSI IP TCP iSCSI SCSI Data IP TCP FCIP FC SCSI Data IP Storage Networking FCIP/iSCSI. Steve Tegeler
39
SCSI architectural model transports
SCSI Block Commands SCSI Stream Commands Parallel SCSI Transport SCSI Applications (File Systems, Databases) Parallel SCSI Interfaces SCSI Device-Type Commands SCSI Generic Commands SCSI Transport Protocols Layer 3 Network Transport Layer 2 Network Fibre Channel Ethernet, PPP, HDLC… Other SCSI Commands IP TCP SCSI Commands, Data, and Status FCP SCSI over FC iSCSI SCSI over TCP/IP IP Storage Networking FCIP/iSCSI. Steve Tegeler
40
iSCSI for storage consolidation
IP access to open systems iSCSI and Fibre Channel storage iSCSI driver is loaded onto hosts on Ethernet network Able to consolidate servers via iSCSI onto existing storage arrays Able to build Ethernet-based SANs using iSCSI arrays iSCSI iSCSI-Enabled Hosts (Initiators) iSCSI iSCSI iSCSI iSCSI Array (Target) IP Network iSCSI Gateway FC Fabric FC HBA-Attached Host (Initiator) Storage Pool (Target) IP Storage Networking FCIP/iSCSI. Steve Tegeler
41
iSCSI for remote block access
Block access to remote storage over IP Application must tolerate latency for long distances Metro Ethernet services offer lower-latency transport alternative Remote backup over IP WAN Centralized management from centralized storage iSCSI iSCSI-Enabled Host Site A IP WAN Site B iSCSI Device Remote Mirrors FC Fabric Storage Pool IP Storage Networking FCIP/iSCSI. Steve Tegeler
42
iSCSI architecture: software driver
Host Applications File System iSCSI GW Device iSCSI Host Driver Block Device iSCSI GW Module SCSI Generic SCSI Driver TCP/IP Driver iSCSI TCP/IP Stack FC HBA GigE NIC NIC Driver Adapter Driver IP Network NIC SCSI Adapter Fibre Channel iSCSI Path Conventional SCSI Path IP Storage Networking FCIP/iSCSI. Steve Tegeler
43
OS support OS vendors support native iSCSI drivers Applications
iSCSI Software Driver File System Block Device SCSI Generic iSCSI TCP/IP Stack Adapter Driver NIC Driver NIC Adapter SCSI Adapter IP Storage Networking FCIP/iSCSI. Steve Tegeler
44
iSCSI HBAs and TCP offload engines (TOEs)
Offloads TCP and, optionally, iSCSI processing into hardware Relieves host CPU from: TCP processing—16-bit checksum per packet iSCSI—optional 32-bit header and data digests (CRC32C) Applications File System Block Device SCSI Generic TCP/IP Stack NIC Driver iSCSI Driver TOE Driver HBA Driver Adapter Driver TCP/IP Stack TCP/IP Stack iSCSI SCSI Adapter TCP Offload iSCSI and TCP Offload IP Storage Networking FCIP/iSCSI. Steve Tegeler
45
Introduction To Storage: Overview of the storage subsystem
Fibre channel vs. iSCSI Fibre Channel The current market leader for shared storage technologies Provides the highest performance levels Designed for mission-critical applications Cost of components is relatively high, particularly per server HBA costs Relatively difficult to implement and manage iSCSI Performance can approach Fibre Channel speeds A better fit for databases than NAS A good fit for Small to Medium Size Businesses Relatively inexpensive, compared to Fibre Channel Relatively easy to implement and manage . Introduction To Storage: Overview of the storage subsystem
46
Example performance impact on CPU utility %
FC vs. iSCSI TOE vs. iSCSI SW Driver iSCSI SW Driver iSCSI TOE FC HBA CPU % Throughput MB/s
47
9/18/ :34 PM SAN © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
48
Introduction To Storage: Overview of the storage subsystem
SAN: what is it? Storage Area Network A network whose primary purpose is the transfer of data between storage systems and computer systems Fibre Channel is the primary technology utilized for SANs Can be implemented with dedicated iSCSI networks Introduction To Storage: Overview of the storage subsystem
49
Benefits of SAN/consolidated storage
Reduce cost of external storage Increase performance Centralized and improved tape backup LAN-less backup High-speed Consolidation with > 70TB of storage Introduction To Storage: Overview of the storage subsystem
50
Introduction To Storage: Overview of the storage subsystem
NAS: what is it? Network Attached Storage Utilizes a TCP/IP network to “share” data Uses file sharing protocols like Unix NFS Storage “Appliances” utilize a stripped-down OS that optimizes file protocol performance Introduction To Storage: Overview of the storage subsystem
51
Networked attached storage
Public or Private Ethernet network NAS Server NIC Server NIC All data converted to file protocol for transmission (may slow down database transactions) Storage Disk Drives Server has a Network Interface Card No RAID Controller or HBA in the server RAID Controller Introduction To Storage: Overview of the storage subsystem
52
NAS vs. SAN NAS SAN Access Methods File access Disk block access
Access Medium Ethernet Fiber Channel Architecture Decentralized Centralized Transport Protocol Layer over TCP/IP SCSI/FC and SCSI/IP Efficiency Less More Sharing and Access Control Good Poor Typical Applications Web Database Typical Clients Workstations Database servers
53
Storage Virtualization
9/18/ :34 PM Storage Virtualization © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
54
What and where can storage Potential Areas of Virtualization
be virtualized? SNIA Storage Model Storage Networking Industry Association Potential Areas of Virtualization Storage Virtualization aims to provide a layer of abstraction to manage storage and reduce complexity !!! 3 File Level Virtualization 2 Host Level Virtualization * 6 Network Virtualization Block Virtualization 4 * * 5 Device Virtualization 1 Storage Level Virtualization Source: The Storage Networking Tutorials, SNIAVIRT- Page 20 * Host aka Server ** Device=aggregation of Host and Network (Meta Data)
55
SNIA storage virtualization taxonomy
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. SNIA storage virtualization taxonomy Storage Virtualization Block Disk File System, File/record Other Device Tape, Tape Drive, Tape Library Network Based Virtualization Storage Device/Storage Subsystem Virtualization Host Based In-band Out-of-band What is created Where it is done How it is implemented The SNIA (Storage Networking Industry Association) storage virtualization taxonomy provides a systematic classification of storage virtualization, with three levels defining what, where, and how storage can be virtualized. The first level of the storage virtualization taxonomy addresses “what” is created. It specifies the types of virtualization: block virtualization, file virtualization, disk virtualization, tape virtualization, or any other device virtualization. Block-level and file-level virtualization are the core focus areas covered later in this module. Storage Virtualization. EMC. 2009 Storage Virtualization Storage Virtualization
56
Storage virtualization requires a multi-level approach
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Storage virtualization requires a multi-level approach Path management Server Volume management Replication Storage Path redirection Network Load balancing - ISL trucking Access control - Zoning Volume management - LUNs Access control Storage Replication RAID Storage Virtualization Storage Virtualization. EMC. 2009 Storage Virtualization
57
Storage virtualization configuration
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Storage virtualization configuration Servers Storage Arrays Virtualization Appliance Out-of-Band (a) Network Servers Storage Arrays In-Band (b) Network Virtualization Appliance (a) In out-of-band implementation, the virtualized environment configuration is stored external to the data path (b) The in-band implementation places the virtualization function in the data path Storage Virtualization Storage Virtualization. EMC. 2009 Storage Virtualization
58
Block-level storage virtualization
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Block-level storage virtualization Ties together multiple independent storage arrays Presented to host as a single storage device Mapping used to redirect I/O on this device to underlying physical arrays Deployed in a SAN environment Non-disruptive data mobility and data migration Enable significant cost and resource optimization Servers Heterogeneous Storage Arrays Virtualization Applied at SAN Level Storage Virtualization Storage Virtualization. EMC. 2009 Storage Virtualization
59
File-level virtualization
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. File-level virtualization Every NAS device is an independent entity, physically and logically Underutilized storage resources Downtime caused by data migrations NAS Devices/Platforms Before File-Level Virtualization IP Network Storage Array File Server Clients Break dependencies between end-user access and data location Storage utilization is optimized Nondisruptive migrations NAS Devices/Platforms After File-Level Virtualization IP Network Clients Storage Array File Server Virtualization Appliance Storage Virtualization Storage Virtualization. EMC. 2009 Storage Virtualization
60
How is storage virtualized at the enterprise level?
Currently Networks are virtualized using Metadata or Storage Volume Controllers. There are two types of network virtualization… Metadata or Storage Volume Controllers are placed (out of band) outside the path of data flow. Metadata or Storage Volume Controllers (SVC) are placed (in-band) or in the path of data flow. Source: IBM Redbook Page 8
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.