1. Storage Wide-Area Networks (SWANs)
Randy H. Katz
Computer Science Division
Electrical Engineering and Computer Science Department
University of California, Berkeley
Berkeley, CA

2. Storage Networks
Primary goal is to share storage among computers in complex, heterogeneous environments, with PCs, workstations, file servers, and mainframes
Storage can be:
Direct attached (e.g., host bus adapter/HBA)
Network attached (via a file server)
Channel attached (primarily fibre channel, but also IBM SSA)

3. Storage Networks E.g, Hitachi, EMC, IBM Storage Arrays Mainframe
Fibre Channel Loops
Dual Ported Disks
And Controllers
Remote
Storage
Manager DI DI DI DI
Cache
Workstation
Cache
Mainframe
Crossbar
Interconnect
Cache
Fibre Channel
Or ESCON
Cache HI HI HI HI
Multiple Host
Interconnections

4. Storage Networks Major development in 1990s: storage networks
Native FC is a point-to-point or loop/string-oriented (“arbitrated”) method of interconnection; 1 Gbps, extend up to 10 km
Emergence of FC “fabrics”: FC switches arranged hierarchically to enable connectivity between any host and any storage device (e.g., Brocade Networks dominates this product space)
Standardize FC protocol stack lives on top of such fabrics: device naming, transport, CoS, etc.
SCSI-3 protocol over FC fabrics
Gigabit ethernet now emerging in SAN environment

5. Basic Attached Storage
Host OS
Disk Interface (DI)
Allocation
Table
Disk,
Cylinder,Track,
Sector
Device attached by SCSI HBA or channel interface
Host manages the file-to-block mapping

6. Network-Attached Storage (NAS) aka Network File Service
Host
Network Interface
(NI)
Network
File
Server
Host
LAN
File Name,
Offset, Length OS
Network Interface
(NI)
Host
Network Interface
(NI)
Mapping from File to Block done in network-attached File Server, not host

7. Network-Attached Secure Devices (NASD)
Host
Research project on
device embedded protocol stack,
authentication OS
Network Interface
(NI)
Network
File
Server
Host
LAN
Disk,
Cylinder,
Track,
Sector
File Name,
Offset, Length OS
Network Interface (NI)
Host
Network Interface
(NI)
Network-attached
Secure Device (NASD)

8. Storage Platforms “Storage Virtualization”
Main
Frame
LUN = Logical Unit
Logical disk mapping onto underlying physical disks on logical block to physical
block basis OS
LUN,
Offset,
Length
Channel Interface
Main
Frame
Disk
Storage
Subsystem
Work
Station
LUN To
PHY

9. NAS NAS distinguished by an exported
Network File System interface over a standard
Local Area Network-based transport
File
Server
Host
Network Interface
(NI)
Host
LAN
File
Server
File Name,
Offset, Length
Network Interface
(NI)
Host
Network Interface
(NI)
File
Server

10. NAS + SAN
SAN distinguished by a block-oriented interface; Usually implemented across a channel-oriented fabric
Main
Frame
PHY Device,
Cyl, Trk, Sector
File
Server
Host
Disk
Storage
Subsystem
Network Interface
(NI)
Channel
Interface CI
Host
File
Server
SAN
LUN,
Offset, Length
LAN
Tape
Storage
Subsystem
LUN,
Offset,
Length
File Name,
Offset, Length CI
Network Interface
(NI) CI CI
Host
Network Interface
(NI)
Optical
Disk
Storage
Subsystem
File
Server
Main
Frame

11. NAS + SAN + SWAN Now extend the NAS or the SAN over a wide-area
network transport …
NOTE: wide-area
SAN is new idea
Optical
Disk
Storage
Subsystem
SAN
Main
Frame
Tape
Channel
Interface
LAN
Host
Network Interface
(NI)
File Name,
Offset, Length
File
Server CI
LUN,
Offset,
Length
Gate
way
WAN FS
DSS
Remote SAN
PHY Device,
Cyl, Trk, Sector

12. Shared Storage Reference Model
Application
File/Record Subsystem
Host
Resource Mgmt, Configuration
Redundancy Mgmt, Back-up
Service Subsystem
High Availability, Fail-over
Discovery, Monitoring
Security, Billing
Capacity Planning
Block
Aggregation
Storage Domain
SAN
Device
Block Subsystem

13. SAN Reference Model Block-oriented Application SAN File FS Host-based
SAN-based
Block
Device-based DA

14. SAN Reference Model Application NAS Storage Host Host LAN File FS NAS
Block
Device

15. SAN Reference Model Application Heterogeneous Storage Environment Host
s/w
raid
Host
Host
Host
LAN
File FS
NAS
Head
NAS
Host
SAN
SAN
Block
Device DA

16. FC-1 Encoding, Link Control
Seven Layer Stack IP
NFS, CIFS
FTP, SNMP,
TFTP, Telnet,
FCP, SCSI-3
TCP, UDP
LAN, MAN, WAN
Phy
Sw GigaE
NFS, CIFS
FTP, SNMP,
TFTP, Telnet,
FCP, SCSI-3
TCP, UDP IP
Mac Client/ Control
Phy FC
SCSI-3 VI IP
FC-4 Protocol I/F FC-3 Encrypt/ Authentication FC-2 Framing, FC, Class of Service
FC-1 Encoding, Link Control
FC-0 Phy 7 6 5 4 3 2 1
Application
Presentation
Session
Transport
Network
Data Link
Physical

17. Fibre Channel Protocol Stack
FC-0: gigabit physical layer
FC-1: data encoding and link layer control
FC-2: segmentation/reassembly of data frames, flow control, class of service
FC-3: common services, e.g., encryption
FC-4: “upper layer protocol” upon which SCSI-3 or IP can run
Contrast with Gigabit Ethernet
Ethernet framing, VLAN tagging, frame prioritization (8 levels), link aggregation, 1.25 gbps
IP + Gigabit Ethernet emerging for SANs

18. (e.g., Fibre Channel or Serial SCSI over IP)
SCSI-3 Terminology
Target
Initiator
Request
Device
Server
LUNs
Application
Client
SCSI
Client-Server
Model
Response
Delivery Subsystem
(e.g., Fibre Channel or Serial SCSI over IP)

19. FC over IP (FCIP) IP Network WAN Tunnel Session
Server
Tape
Storage
Subsystem
Tape
Storage
Subsystem
Server
WAN
IP Network FC
Switch FC
Over IP FC
Over IP FC
Switch
Server
JBOD
Server
JBOD
Tunnel Session
IETF IP Storage (IPS) working group
Recall FC fabrics developed in context of machine room/building-scale interconnect (e.g., no congestion control!)
FC time outs in wide-area? Flow control interaction? QoS?
How does bridging actually work for FC e2e management?

20. Internet FC Protocol (iFCP)
FC_Device
N_Port
F_Port
iFCP layer
FCP Portal
FC Traffic
IP Network
FC Device
Address
IP Address
Mapping
Control Data
iFCP gateway
region
iFCP Frames
Gateway to gateway protocol, sessions rather than tunnels
TCP for congestion control, error detection, recovery
Plug FC devices directly into iFCP switches
Session and naming semantics

21. iFCP Services 24-bit N-Port Address: <Domain, Area, Port>
iSNS: Internet Storage Name Service—discovery and management protocol for IP storage networks (IPNSP)
Protocol specification includes address translation feature to allow remote storage devices to be assigned a local, FC fabric compliant address
Local commands executed locally on the fabric
Remote commands executed on top of TCP connections
Error Detection/Time Outs
Security

22. Other Protocols Metro Fibre Channel Protocol (mFCP)
FCP over IP using UDP rather than TCP (link layer flow control and pacing)
Internet SCSI (iSCSI)
IP to the storage device
Serial SCSI block data transfer over IP (SCSI Access Method Command Set—SAM)
IPSec, command/data ordering, steering to application memory

23. iSNS Discovery Process iSNS objects Device registration
WWN or iSCSI names
Zoning/discovery domains
iSNS objects
Portals
Storage Port
Storage Nodes

24. Storage Applications Data Centers IP Storage for Remote Applications
Server clustering
Storage centralization, consolidation, management
LAN-free back-up
IP Storage for Remote Applications
Remote back-up
Remote mirroring
Disaster recovery
Content distribution

25. Rhapsody Networks, Inc. Storage Application Director
“Alteon box for storage networks”
Peek into storage packets traversing fabric (“deep frame classification”) and invoke code—for encryption/decryption, mirroring, LUN mappings, etc.
Per port software processing and cut-through fabric routing
Data copy engine, table lookup engine, in-transit I/O mods, data escrow/trap to software for complex errors or event processing
Intelligent queue management