Beyond NAS and SAN: The Evolution of Storage Marc Farley Author Building Storage Networks

What We’ll Do Analyze storage network functions Apply these functions in NAS and SANs Observe market pressures forcing changes Identify evolving architectures

Fundamental Functions of Storage Networks Wiring Fibre Channel. Parallel SCSI, Ethernet, IP Storing Devices & Subsystems, I/O Controllers, Virtualization SW & HW (RAID, Volume Managers, Mirroring) Filing File systems, Databases, Backup, Replication Building Blocks

Wiring Storing Filing Traditional SAN Functional Distribution Host System Storage Subsystem Fibre Channel Network HBA Building Blocks Controller

Wiring Storing Traditional NAS Functional Distribution Client System File Server System LAN Filing NIC Internal Bus Building Blocks

Wiring: Physical & Logical Parts Physical = Cabling and Network Hardware signaling, encoding, error detection Logical = Algorithms, Addressing, Protocols and Services access methods, flow control (buffer management) addressing, naming, topology management, routing filtering, zoning gateway, bridging and tunneling services Building Blocks

Wiring: Fibre Channel and i-SCSI Functionally equivalent as wiring technologies More of a business issue than a technology issue Question: Will i-SCSI SANs be implemented as distinct networks or grow as part of the existing Ethernet/IP infrastructure? i-SCSI leverage might be less than expected - but it might be good enough Building Blocks

Wiring: Separate SANs or Bigger LANs? LAN SAN Good Idea

An Question for the Audience….. What is the best routing method for storage traffic? To date, storage networks have borrowed from IP networking OSPF in IP networks (NAS) & FSPF in Fibre Channel What do you want? Fast recovery & optimal availability –OR Compatibility with IP routing An Idea

Storing Block level operations storage block addressing store, retrieve, status, resource reservation Subsystems and Virtual Devices block translation, mirroring, multiple ports, LUN masking I/O commands between initiators and targets SCSI, serial SCSI, SCSI adaptations, IDE really Error correction really happens here Building Blocks

Storing on Steroids: Virtualization Virtualization techniques expand and extend the capabilities of devices and subsystems Virtualization as a storing level function has no ability to manage anything about its contents Virtual Storage Physical Devices

Filing Representation of data files, directories, tables, records Storage data structure (block address layout) equalized performance, maintain free blocks System kernel integration kernel manages time, file system manages space File semantics Open, close, update, delete, block range operations Building Blocks

The Fundamental Role of Filing Manage the use of the storing address space Building Blocks Storing Filing

Internal Functions in Filing

NAS & SAN REDUX

SANs are the Application of Storing Functions Over a Network The Caveat: SANS do not convey any knowledge of data structures & there is no way to provide data-level management Gotcha Storing Storing Functions Initiators Targets Storing Network

Virtualization Creates Storing Structures  Block Data Initiators Virtual Targets Storing Network Storing Physical Devices ] But not data structures

NAS: The Application of Filing Functions over a Network The Caveat: ‘Loose wiring’ for I/O intensive computing Gotcha Filing Functions Clients Servers Filing Network

Challenge for NAS: Distribute Filing Functions & Reinforce the Wiring File Data Clients Servers Network

Storage Network Requirements

Market Needs Integrity Recoverability Security Availability Manageability (Scalability) Best Practice

Data Integrity Must be Maintained Error free transmissions Fiber optic cabling Data segregation is a good thing Who wants a Battle Royale over data? Any to any connectivity is not required Storage networks differ a lot from data networks Zoning, masking and fencing are band-aids for existing architectural weaknesses Filing enforces data integrity through locking Gotcha

Data Recoverability = Redundancy Data redundancy  system redundancy Mirroring is fast, but relatively inefficient Extended distance mirrors are “open territory” System buffer management is a caveat Data structures must have integrity Journalled file systems, database commitments File system intelligence is most efficient Serverless backup, data snapshots, replication Gotcha

Security - the Great Afterthought Protecting Data From Theft File systems have done this for decades System-login is the gatekeeper There are no constructs for storing-level security Wiring-level security can be done, but is slow Encryption is S-L-O-W for I/O IP-SEC may provide a method Gotcha

Remember the Good Old Days of SCSI? Data integrity was less exposed with Parallel SCSI Human error was far less likely Storage was not accessible to hackers Processing power on the storage end is needed to provide security functions

Data Availability Remove the system as a bottleneck (And integrity and security become more suspect) Availability doesn’t necessarily mean “seamless” Single servers in SANs can failover using redundant paths NAS network failures must re-connect Data sharing can provide availability NAS file locking can ensure integrity for server farms Data sharing in SANs is expensive and complex

The Fine Print in Scalability Maxed-out NAS requires additional mount points Users and application configurations must change SANs can use virtualization Does “V” ensure integrity? Is “V” recoverable? (will it support serverless backup?) File system data structures must be included for non-stop scalability! Secret

Storing Filing The Fine Print in Scalability How does the filing function know about the new storing address space?

Evolving Architectures

Evolving Storage Network Architectures Filing technologies are intimately involved with: Integrity Recoverability Security Availability (sharing) Scalability Performance (efficiency - working smart) Hmmmmm? Maybe filing could be important! Good Idea

DAFS Network Appliance (and others?) Puts rigor into NAS wiring Important proof of concept for NAS Eliminate doubts about NAS for database I/O

Relocating Filing Functions in the Storage Network Systems need to have a data view Can be achieved by aggregating views from distributed Metadata is moved out of host systems Distributed or network-central Data structures can be managed by the subsystem Enables subsystem-based recoverability Delivers enormous scalability

Aggregating Data Views Virtualization at the file level Data views from many subsystems are merged

Independence of Metadata from Hosts Integrity (locking) Scalability Metadata Systems Storage 1 2 3

Independence of Metadata from Hosts Systems Storage Distributed Metadata

Subsystem-Managed Data Structures System without data structure function Data structure is distributed across multiple subsystems